File format of historical data and processing for analysis

[rickym]

I am reading through the forums and not finding a clear answer how to proceed.
Amazing work and people here, BTW...
I have downloaded historical data from a subscription service and have them in my NT8 folder.
NT8 takes a long time to open the files, and when it does, they are by day.
Where do I find developer information on historical data file format so I can open and parse the files for analysis with my neural network program?
My goal is to parse each instrument for the same day at high of resolution as possible so I can look for trading opportunities.
I build something like this for forex trading many years ago but it was very slow.
I want to do the same for futures trading in as close to real-time as possible.
Thanks in advance...

[NinjaTrader_ChrisL]

Hello rickym,

Thanks for the post.

You will need to export the historical data before you can parse it.

Go to Tools> Historical Data> Load> Expand the "Export" section of the load tab. From there you will select the time frame you want to export then click Export. When you click export, you will be prompted to save the .txt file on your hard drive. This file is human readable and you can devise a parsing algorithm to load that data into your program.

https://ninjatrader.com/support/help...?exporting.htm - Exporting historical data

Please let us know if you have any questions.

[rickym]

How do I "chain" together my Market Replay data files?

Thank you Chris!
I have a few years worth of Market Replay data that I cannot seem to get to open with NT8.
How do I "chain" them together and get them exported at the tick level?
They have Level II data but when I go into their individual folders and open them, I do not see any coherent file structure.
Is there something else I am missing?
Are they not going to work because I also have some overlap with my Historical Data?
Thanks in advance...

[NinjaTrader_ChrisL]

Hello rickym,

Thanks for the reply.

Market Replay data is not available for export. When you download Market Replay data, it comes from NinjaTrader servers. When you download historical data it will download from your data provider. Level II data would need to be recorded by a script that you write. You could use this price ladder sample script and store level II the data that comes in.

https://ninjatrader.com/support/forum/showthread.php?t=3478

Please let us know if we may be of any further assistance.