Skipped Data - Bars TD Ameritrade

[antrux]

I have a custom strategy that runs on minute bars and on each OnBarUpdate writes to a CSV log file. This strategy is running on many instruments and I have just noticed that some of the log files are showing skips in bars / data. Below is a copy/paste from one that isn't hitting every bar. Most of the logs show very reliable minute entries but around 7% of them are not.

I know that if NT doesn't get data then OnBarUpdate cannot run so this must be from TDA's side of things. As I doubt that I'd be able to get a clear answer from TDA on this I was wondering if anyone else has ever seen or heard of this before?

7/25/2016 10:11:00 AM
7/25/2016 10:12:00 AM
7/25/2016 10:14:00 AM
7/25/2016 10:15:00 AM
7/25/2016 10:16:00 AM
7/25/2016 10:17:00 AM
7/25/2016 10:18:00 AM
7/25/2016 10:37:00 AM
7/25/2016 10:38:00 AM
7/25/2016 10:39:00 AM
7/25/2016 10:40:00 AM
7/25/2016 10:42:00 AM
7/25/2016 10:45:00 AM
7/25/2016 10:47:00 AM
7/25/2016 10:52:00 AM
7/25/2016 10:55:00 AM
7/25/2016 11:00:00 AM
7/25/2016 11:03:00 AM
7/25/2016 11:04:00 AM
7/25/2016 11:05:00 AM
7/25/2016 11:07:00 AM
7/25/2016 11:08:00 AM
7/25/2016 11:13:00 AM
7/25/2016 11:14:00 AM
7/25/2016 11:27:00 AM
7/25/2016 11:33:00 AM
7/25/2016 11:35:00 AM
7/25/2016 11:42:00 AM
7/25/2016 11:45:00 AM

[NinjaTrader_PatrickH]

Hello antrux,

Thank you for your post.

If you go to Tools > Historical Data Manager > Edit, do you see the data listed that is missing from the files?
You can view the data by expanding the instrument, price series, and then data type and finally date.

For information on the Historical Data Manager please visit the following link: http://ninjatrader.com/support/helpG...t7/editing.htm

[antrux]

Of the instruments not updating through the strategy the Historical Data Manager is missing that information. One example is ARG, Last, Minute, July 2016, 7/25/2016(1 Item) Lists 8:01am with 0's for all the fields and that is it. I can however open a new minute chart with the symbol ARG for the last 262 days and the chart appears to load and look normal. This is NT8 64Bit v8.0.0.12.

[NinjaTrader_PatrickH]

antrux,

Do you see the 1 Minute bar at 8:01 am for the ARG on the chart?

[antrux]

No, After further looking at the chart the data ends around May 22 just after 1600. Then after a Google this it appears this symbol no longer exists as AirGas had a merger with AIQUY in Paris.
Another one was AZO and after looking at it's Historical Data it is also showing missing minutes of data. Shows 11:27am, 33,35,42,45,48, and etc.

[antrux]

And it appears that the spots that are missing from the historical manager are somehow occasionally offset on the chart and the chart also shows missing time. Using the cross-hairs to see the minute time better I can see the chart time go from 11:36 to 11:39.

[NinjaTrader_PatrickH]

Hello antrux,

Thank you for your response.

Can you provide the date and time and instrument name for the missing bars or incorrect bars?

[antrux]

AZO 7/25/2016 Last Minute

Attached is today's Historical Data exported for AZO. There are several others but this is a good example.

Before export I also went to Load, selected AZO, chose today's date, Minute, Last, and pressed Download. No update and no Log window entries regarding an error with AZO.

[antrux]

Also as an FYI, NT8 bug/issue, there is definitely an issue with NT8 and syncing historical data with TDA.

From a fresh new install, add account TDA, connect, goto Historical Data Manager, Load TAB, Select DOW 30, date range a month or etc, Minute, Last, and press Download.

The bottom status bar will only cycle/display 1-3 instruments being downloaded however once the Download button returns to a normal state goto the Edit TAB and click Historical. Simply no data. Have done this multiple times with NT8 32 and 64. I can watch a bandwidth meter and see some small amount of traffic but none that would be enough to fill that data. Found that after I import NT7 data to NT8 then repeat the process above several times while also selecting Minute and Daily data will the data eventually make it in. Was tested on a Win7 and Win10 machine. Once more than a few dozen instruments data has been synced will other new instruments be able to download as usual. No errors in log file are displayed.

[NinjaTrader_PatrickH]

What is incorrect in the AZO file? Is it that data is missing from it?

[NinjaTrader_PatrickH]

On the NT 8 downloading of historical data from TD Ameritrade, can you detail the instruments that data is not downloading for in NinjaTrader 8 when using TD Ameritrade?

[antrux]

What is incorrect in the AZO file? Is it that data is missing from it?

Yes of course the missing data would be what is incorrect. I wasn't looking for Pokemon in it. <grin> Either way I'm chocking it up to the TDA data stream but funds are being transferred this week for a new IB account who shouldn't have this issue then. The NT8 bug mentioned earlier also may just be from TDA's side as well. Not sure but will test again from a fresh install after the IB account is online.