Auto-Refresh Daily data

[music_p13]

Hello,

The issue that I am experiencing from time to time (especially when using multiple workspaces on different instruments) - sometimes daily data is not getting refreshed - and I have to manually download it through historical data load process and then hit F5 on all charts.

Indicator that calculates values based on daily values and plots them on intraday chart would sometimes show wrong values (because daily data was not properly refreshed).

Is there a way to force Daily data refresh (to guarantee that all possible daily data is there) when loading particular indicator or changing workspace?

There are 2 scenarios:
Scenario 1 - indicator keeps on running with workspace/instrument loaded for few days. I want to make sure I don't have to re-load daily data manually each day and I don't have to press F5 on the chart to reload indicators values.

Scenario 2 - I open new workspace/instrument with indicator associated to that workspace, and I want to ensure that it will auto-refresh the daily data and plot indicator and its calculations based on updated daily values.

Any tricks to force auto-download/auto-refresh of daily data (assuming connectivity was running and was never really disconnected from datafeed)?

Thanks

[NinjaTrader_ChrisL]

Hello music_p13,

Thank you for the post.

It sounds like the real-time data is not coming in properly.

First, reset your cache by following these steps:

1. Shut down NinjaTrader.
2. Open the Documents > NinjaTrader 8 > db folder.
3. Delete the sub-folder named 'cache'.
4. Restart NinjaTrader and test.

If this does not resolve the issue, can you make a daily chart in the workspace, then switch the workspace then switch back and see if real-time data is coming in?

Are you seeing any errors in the log tab of the Control Center?

I look forward to your reply.

[NinjaTrader_ChrisL]

Hello music_p13,

As an addendum to my last post, could you also post your workspace file here? Please also let me know what data provider you are connected to.

You can find your workspace file under the following directory:

..\Documents\NinjaTrader 8\workspaces

I look forward to your reply.

[music_p13]

Thanks,

I will try loading daily charts and keeping them on to see if it can help with daily data updates.

I am using kinetick, don't think I have any cache issues...

[NinjaTrader_ChrisL]

Hello music_p13,

Thanks for the reply.

Could you kindly post your workspace file here as well so I can try to reproduce this on my end?

You can also send it into platformsupport[at]ninjatrader[dot]com and put "ATTN ChrisL" in the body of the email.

Thank you very much for your patience.

[brucerobinson]

To add info of what seems like similar, FWIW

I use only the OHL & PriorOHLC indicators.
Sometimes I notice one of them is not displaying on a chart.
Usually, reloading the chart's template is required, I think. IIRC F5 and and Reload Historical Data don't do it.
For info:
I use various charts on same instrument e.g. Multiple time and tick. To give an example, the high may be there on all charts, and conspicuous by its absence on one, and upon investigation to find on that chart it is displaying yesterday's High.

I tried manually re-downloading data, clearing cache and all that good stuff to no avail.
I have the same chart template applied to all. I restart my pc every day. I have multiple workspaces etc

Sometimes one chart/indicator/window/instance of a data series goes rogue and gets stuck on yesterday's values where the same indicator/instrument data series in the same chart (I.e. I have several instances of the same indicator in the same chart and only one might be stuck on yesterday's values, the rest in that chart and other charts ok).

I'm not wishing to hi-jack or for my scenario to be addressed directly. Just throwing the info into the mix if it offers any clues. It happens repeatedly but not frequently. It's an irritation not life threatening. If it does, I reload the template and order is restored.

But it seems not dissimilar in nature to the OP.

Hope it helps & solution for OP applies to mine

[NinjaTrader_ChrisL]

Hello music_p13,

I wanted to try and ask for your workspace once more before I close this ticket.

If you could it would help us find the cause of the suspended daily bar in Kinetick. Thanks in advance.

[music_p13]

Hello,

I have reproduced the issue again.

I left my Ninja connected on Wednesday September 19, before the Daily Futures closed for CL Nov contract.

I left "Daily" workspace to grab daily data at the end of the day, and my other workspace that has indicator that utilizes daily data (multiple timeframes) but plots lines intraday.

On September 20th morning I observed that daily data was not populated into historical data "Last -> Day".

Going back to intraday chart and just hitting F5 - did refresh my indicator and it was able to plot values correctly, even though the daily close was still not pupulated into historical data manager.


Going back to my "Daily" workspace and then right click - and refresh all historical data - was able to populate "Daily" data into historical data manager.

I will send you an email in few minutes with my workspaces, my trace file and screenshots.

Please keep this ticket open...

Thanks

[yeshe]

i don't think this issue is solved
this is how i think nt deals with daily candles:
it uses the minute data to make the close of the daily candle
with futures there is a settlement close that is the real close of the daily candle (not the same as the last minute bar of the day)
nt does not update the daily candle in real time with the exact close price of the daily candle . (Why not?)
only by reloading historical data everyday after the settlementclose is available will the daily candle gets the right close price of the day
keeping your daily charts open in real time will thus lead to an incorrect chart, and incorrect indicators etc.

for traders using daily candles they need to reload historical data everyday after the settlement close is available
in other words you cannot automatically have a daily trading strategy running in nt

this misses the purpose of automatic trading and the purpose of nt.

first of all, are my assumptions right.
if so how do traders deal with this issue. there are a lot of traders that use daily candles in their auto strategies.
so how do you deal with this
please your answers

[NinjaTrader_ChrisL]

Hello yeshe,

Thank you for the post.

I will test out a daily CL chart today and update this thread with what I find. Just to confirm, you are expecting a daily chart to hold the non-adjusted close price until I reload the historical data, then it will have the adjusted settlement close price?

I look forward to assisting further.

[brucerobinson]

Hello yeshe & ChrisL.
(In advance, if I'm teaching grandma to suck eggs, apologies to ChrisL - just sharing knowledge/experience)
What yeshe describes is along the lines of how NT7 worked.

So unless it has specifically been changed in 8 to deal differently with Futures Settlement Close, that is how it has been historically.

The reason I know for sure for 7 comes from the development path for Pivot indicators. NT's own used Daily Close, but Settlement Close (obtained from mypivots.com etc) could be entered manually.

Harry FatTails' Pivot indicators could pull in the Settlement Close if it was in your Historical Data. However, you had to put it in your historical data - it didn't appear automatically after the Close & then the question of whether it refreshed automatically if left connected, or have to manually refresh that yeshe asks. To get it in your Historical, the easiest(!) way was to connect to Kinetic EOD which had Settlement Close & reload your historical data to over-write the Daily Close value with the Settlement Close in the database on your PC, which the indicator could then use for proper Pivot calcs. Disconnect Kinetic/connect to your broker feed & Bob's your Uncle (but don't reload your historical before tomorrow or you'll be back on Daily Close).

I don't use pivots or Close any longer, so don't have duplicate experience on 8 to draw on/share hence can't comment further on what happens today on 8, but curious to know. Never hurts.

Hope it helps.

Best,

[NinjaTrader_ChrisL]

Hello yeshe,

I couldn't see this happen running Kinetick over a few sessions. I will make a workspace that holds many daily charts to see if I can make it happen there. Since multiple users are seeing this on R16.3 I assume I will be able to reproduce this. If possible, please post your workspace as well in case I can not reproduce on my workspace.

Thanks in advance for your patience.

[yeshe]

thank you brucerobinson for your reply. You confirm what i found out .
Chris L. i am not sure what you are testing or what you are asking me.
to repeat: what i find out (and the other users in this thread) is that the daily settlementclose does not automatically update on the chart and therefore not in my historical database. (only by manually reloading historical data after the settlementclose is available). Because nt does not update the daily candles my indicators and strategies are in error and my historical database is corrupted.
I have to remember everyday to update my daily charts manually, which beats the purpose of an automatic trading platform.

my suggestion to nt:
please let open charts update to the right daily candle by itself
if that is not possible:
please make a method RELOADHISTORICALBARS(X);
like this the user can in the code get the daily settlementclose when available and correct the chart and database from within the strategy.

if (Bars.FirstBarOfSession) RELOADHISTORICALBARS(1); // yesterdays' settlementclose will be available on the first (minute) bar of the next session

in this way you don't need to manually reload all the data and you don't need to manually stop the strategy and restart it.

please your response

[NinjaTrader_ChrisL]

Hello yeshe,

Thank you for your patience.

I was able to reproduce the daily bar update issue. I will let our product management team know of this item. I will also submit a feature request to add a historical data reload method in NinjaScript.

Please let me know if I can assist further.

[brucerobinson]

Hello Yeshe & ChrisL,
Although I don't use this, curiosity to know how 8 is dealing with Settlement Close got the better of me.
It was a contentious point in 7, insofar as it wasn't dealing with the issue, and given 7 was focused toward Futures, caused some discussion and third-party solutions.

I found this thread on the subject for 8 https://ninjatrader.com/support/foru...-to-last-price

and this in Help:

Note

: Intraday time based charts are built off the

Trading Hours

definitions set for the individual chart's

DataSeries

. For daily charts and higher this is not the case though, here the Trading Hours are governed by the provider recording the data. For the NinjaTrader Historical Data Servers daily bars will be recorded using the ETH (Electronic Trading Hours) definitions for the respective instrument. Further for providers which support accessing the official settlement value, NinjaTrader will use this value as the daily bar close - for more information regarding your specific provider please consult

this link

.

To my reading the language is somewhat awkward hindering clarity but what it seems to me to be saying is that if your provider supports accessing the Settlement Close, then Ninja will use that, whatever the chart, intra, daily or above, real time or HDS & not need refresh/load. But you're saying in practice, not.

Unless either/both using a provider that doesn't give Settlement (unlikely).

Oh, well, good luck.

Kind regards