Kinetick data

[Harry]

I have had a similar problem some time ago, which was fixed:

https://ninjatrader.com/support/forum/showthread.php?t=98235&highlight=Kinetick+hour+missing

However, downloading trade data is still a real pain with NinjaTrader 8.

Attached is sample chart from today: Data for August 3 is missing. Data from August 9 and 10 is also missing. NinjaTrader is incapable of filling such gaps. I am often spending half an hour to feed the missing data into the data base by reloading charts for 30 to 40 instruments.

Today there was not a single instrument that did properly show. I believe that this time the problem was linked to the market analyzer. By mistake I had opened a market analyzer with an instrument list before updating my historical data. Now I have to reload charts manually for 30 instruments.

Another huge problem is that NinjaTrader regularly throws exeptions when I try to download data for an instrument list via the historical data manager The data adapter then freezes and I need to shut down and restart NinjaTrader before I may continue to download data. Checking the trace file tells me which instruments have possibly been correctly updated, and I need to load the other ones one by one, as downloading the entire instrument list throws exceptions.

Getting correct data is the first thing to do before trading and it should be much simpler.

[NinjaTrader_PatrickG]

Harry, I split your post into its own thread as I do not believe it is related to the original post of the previous thread.

What version of NinjaTrader 8 are you using? Please provide the entire version number. This can be found under Help -> About (Example: 8.0.X.0)

The following help guide page details how NinjaTrader loads historical data:

When does NinjaTrader load historical data?

NinjaTrader loads data from your data provider whenever it determines it could potentially not have all the data pertaining to the requested time period.

NinjaTrader will load data when:

1. The End date parameter of the Data Series window contains the current day
2. The End date parameter of the Data Series window contains the last day of data available in your data repository
3. The data repository contains no data within 3 days of the first day loaded in the chart

Based on the information you've provided, it sounds like NinjaTrader is not attempting to backfill the missing data because of item #3.

I think you may be onto something regarding your Market Analyzer effecting the downloads. Do any of your Market Analyzer columns have a 'days to load' property? If so, what are those set to?

Also, what process/steps do you follow which result in an unhandled exception message? How do recover from this message?

[Harry]

Hi Patrick,

Thank you for your answer.

I am using NT 8.0.15.1 (64 bit).

Actually, it is easy to get clean data. Reloading a chart and - in severe cases - deleting the data base case cache will always fix it.

The problem here is the market analyzer. As there are many instruments on the market analyzer, I use it with a shorter lookback period than my charts. Therefore when the market analyzer loads data it will leave gaps in the data base which do not get filled later when using the historcial data manager or loading a chart.

Usually when I open NinjaTrader I first update historical data via the historical data manager and and then open the market analyzer. In this case there is no problem.

However, when I close NinjaTrader and have a primary workspace with the market analyzer open, this market analyzer will reappear when I open NinjaTrader again. The market analyzer will load histrorical data as needed, and then will leave the gaps in the historical data base.

Then I have to fill all the gaps manually by opening 30-40 charts one by one and reload the data.

[NinjaTrader_PatrickG]

This sounds expected based on the information you've provided.

I spoke with the Product Development Team at great length regarding what you're experiencing, and we are unsure how this behavior could be improved for cases like yours without negatively affecting performance.

I'll forward a suggestion to the Development Team to try to find a way to improve the behavior without sacrificing performance for cases like you're describing. The tracking number is SFT-1627.

[Harry]

Thank you for your answer.

The problem here is that NinjaTrader cannot detect giant holes in the backfill.

I understand that any algorithm that checks for completeness of historical data will have a negative impact on chart loading times.