TDA historical price feed not supplying the last day of historical data

[bertochi]

Last Week I started experiencing a problem with my strategies, and after digging and debugging, I realized that historical prices are feed but not for the last day of historical data. I noticed this is fixed by right clicking on a chart and selecting "reload all historical data". I checked the trace files and the request is made fine until the last day, but prices of the last day of historical data are not feed into strategies or charts until you do what i said above (right click, reload all historical data).
Are there any other TDA users out there who could check if you have the same issue?
So far, I tried different things, deleted the database and allowed NT8 to start with a fresh one, and tried on 3 different computers. The issue is the same, you call a chart for the last five days and you see data until the 27/06/2019, nothing for 28/06/2019, being today Saturday 29/06/2019.
I am thinking of a way to use ReloadAllHistoricalData() and I do understand it can only be used after an OnConnectionStatusUpdate event, but maybe with an external bool inside a file.

Any thoughts or helps from anyone will be much appreciated.

Thanks

[bertochi]

Temporary fixes:

I added the ReloadAllHistoricalData() inside my strategies in state.configure and checking for a false bool from an external file, which is updated to true after a first run, and every time NT8 is started such bool is reset to false. This workaround worked, but could take 2 or 3x times for everything to load specially if you have many instruments traded and many days to load.

Therefore, I kept trying to find the root cause of the problem. I tried NT7 and there was no issue there, all charts showing historical data until the last day without needing to manually reload all historical data. That made me think, what would be the main difference between Nt7 and Nt8, and I found it, the bloody time zone setting added to NT8. I am in Europe and my time zone is set to "UTC-0". When I set this to "UTC -5" for US, the charts/strategies are all plotting/working correctly until the last minute of historical data. This is the root cause of the problem. In my case, I do not see the benefit of having this feature which as you can see causes other major problems to the platform. Keep it simple.

Conclusion: If you have set your time zone to anything different than "UTC -5 US", your historical data will not load correctly in charts and strategies.

[NinjaTrader_ChelseaB]

Hello bertochi,

To confirm, NinjaTrader 8 is not automatically downloading historical data from TD Ameritrade up to date, is this correct?
This is minute or daily data, and is not tick data, correct?

Is the Tools > Options > Market data > Historical > 'Get data from server' checked and enabled?


Please try clearing the historical data and cache and see if this automatically downloads data up to the current after.

• Close NinjaTrader
• Open (My) Documents\NinjaTrader 8\db\cache\
• Delete all files within this folder
• Go up one directory to the \db\ folder
• Repeat this procedure for the minute, and day folders as well to delete your downloaded historical data
• Restart NinjaTrader
• Connect

[bertochi]

Dear Chelsea,

I found the root of the problem. If you see my second reply on this thread you will see it.
I tried what you said and many more alternatives. Tried on different pcs and with fresh installations too.
Then, to rule out the problem was not TDA, I opened NT7 and historical intraday prices were displayed correctly there (30m 60m any intraday prices). So the problem was and still is with NT8.

The problem lies in the time zone setting within NT8. I had it setup correctly as utc=0 as I am in Europe. But changed it to utc=-5 (US), and then the charts all started displaying intraday data until the last day, in this case, last friday 28/6. This behaviour not only affects charts but also strategies. With my correct time zone setup of utc=0 I was not getting prices for the last day unless I did a "right click on the chart and reloadallhistoricaldata", and on the strategies I had to include the statement ReloadAllHistoricalData() as mentioned earlier.

Regards

[NinjaTrader_ChelseaB]

Hello bertochi,

May I confirm you are using 8.0.18.1?

This may be related to similar open issue with time zones and downloading data from Interactive Brokers I would like to inquire with our development about.

Would our development be able to temporarily use your account to test for the behavior?

If so, please send an email to platformsupport [at] ninjatrader [dot] com.
In the email please include a link to this forum thread.

[bertochi]

Dear Chelsea,

Please let me tell you I had issues with my strategies for the last 2 weeks and had to spend a lot of time to get to the bottom of this. I do not want to say any more. This "time zone" feature is in my view absolutely useless and opens up the possibility of issues such as the one I am describing. Had there not be any time zone feature, this issue would have never happened. And yes, I am using the latest NT8 version 8.0.18.1

[bertochi]

Dear Chelsea,

My 2cent suggestion. To your dev team, get rid of the time zone feature, which I am sure is causing issues to historical price feeds with other sources such as IB. Better solid than fancy. Besides, what is so important about a time zone feature to have in the platform? People can code their strategies depending where they are/live. It's always been like this. No time zone f*&^%, no chances of issues. You can test it yourself, set your time zone to different zones other than US, and start calling intraday charts. BTW the price feed is not the issue, the issue lies with NT8 not displaying the prices that the price feed has already supplied. Think about why right clicking "reloadallhistoricaldata" fixes the problem.

Regards