Historical Data Export Formats

[Gandalf5]

There is no direct documentation on the exported historical data formats in NT8. The export data page refers to the import page.

https://ninjatrader.com/support/helpGuides/nt8/en-us/?exporting.htm

"Tick Replay Format (Sub Second Granularity)
Each tick must be on its own line and fields must be separated by semicolon (.

The format is:
yyyyMMdd HHmmss fffffff;last price; bid price; ask price;volume

Sample data: (Note: If you wanted to import in millisecond granularity data then each line must have the remaining "0"'s behind it to import correctly.)
20061107 000431 1000000;1383.00;1383.00;1383.25;1
20061107 000456 1000000;1383.25;1382.50;1382.25;25
20061107 000456 2000000;1383.25;1383.25;1383.50;36
20061107 000537 7000000;1383.25;1383.25;1383.50;14"

But there are 3 options to export tick data, last, bid, and ask. Each option on the export screen gives a different format for the same time period. Below I show samples from each option? What is the format when exporting with each option?

Last

$ head spxl-last-nt8.txt
20160725 120101 8340000;98;97.72;98.16;1800
20160725 122016 7050000;97.98;97.9;98.07;100
20160725 123419 2180000;98;97.87;98.05;150
20160725 132833 3480000;97.65;97.65;97.69;100
20160725 132835 3020000;97.65;97.65;97.68;200
20160725 132835 4470000;97.65;97.65;97.68;200
20160725 132835 7840000;97.65;97.65;97.68;200
20160725 132840 6490000;97.65;97.65;97.68;400
20160725 132840 6490000;97.65;97.65;97.68;100
20160725 132840 6500000;97.65;97.65;97.68;400


Bid

$ head spxl-bid-nt8.txt
20160725 120101 8340000;97.72;97.72;98.16;0
20160725 122016 7050000;97.9;97.9;98.07;0
20160725 123419 2180000;97.87;97.87;98.05;0
20160725 132833 3480000;97.65;97.65;97.69;0
20160725 132835 3020000;97.65;97.65;97.68;0
20160725 132835 4470000;97.65;97.65;97.68;0
20160725 132835 7840000;97.65;97.65;97.68;0
20160725 132840 6490000;97.65;97.65;97.68;0
20160725 132840 6490000;97.65;97.65;97.68;0
20160725 132840 6500000;97.65;97.65;97.68;0


Ask

$ head spxl-nt8-ask.txt
20160725 120101 8340000;98.16;97.72;98.16;0
20160725 122016 7050000;98.07;97.9;98.07;0
20160725 123419 2180000;98.05;97.87;98.05;0
20160725 132833 3480000;97.69;97.65;97.69;0
20160725 132835 3020000;97.68;97.65;97.68;0
20160725 132835 4470000;97.68;97.65;97.68;0
20160725 132835 7840000;97.68;97.65;97.68;0
20160725 132840 6490000;97.68;97.65;97.68;0
20160725 132840 6490000;97.68;97.65;97.68;0
20160725 132840 6500000;97.68;97.65;97.68;0

[NinjaTrader_ChelseaB]

Hello Gandalf5,

There are no differences of the text format in the file for Ask, Bid, or Last.

Instead, the data type in the file is controlled by the file name:

From the help guide:
"File Name
When using the NinjaTrader format, the name of the text file to be imported must be the NinjaTrader instrument name followed by a period and "Last", "Bid", or "Ask" depending on the data type. For example:

MSFT.Last.txt for Microsoft stock last price data
ES 12-09.Bid.txt for the S&P E-mini December contract bid price data
EURUSD.Ask.txt for the Euro/U.S. dollar currency pair ask price data"

https://ninjatrader.com/support/helpGuides/nt8/en-us/index.html?importing.htm#UnderstandingImportFileAndDataFormats

[Gandalf5]

The data from the same time frame is formatted differently based upon the export option for last, bid, and ask. See the formatted data in bold? What is the format here?

Last

$ head spxl-last-nt8.txt
20160725 120101 8340000;98;97.72;98.16;1800

Bid

$ head spxl-bid-nt8.txt
20160725 120101 8340000;97.72;97.72;98.16;0

Ask

$ head spxl-nt8-ask.txt
20160725 120101 8340000;98.16;97.72;98.16;0

[tradesmart]

Looks like:
Last = last;bid;ask
Bid = bid;bid;ask
Ask = ask;bid;ask

[Gandalf5]

Agreed. But that is only us inferring that. I'd like NT to supply the answer for NT8.

[NinjaTrader_ChelseaB]

Hello Gandalf5,

This is not the proper text format for Daily, Minute, or Tick.

Your first post does show your are reading that section of the help guide, however, I was advising you on how to import historical data as requested and not tick replay data.

Are you actually wanting to import this as TickReplay data?

If so, you cannot make 3 separate files for ask, bid, and last. These cannot be separated when exporting tick replay data (and not historical data). These must be exported as one file and must be imported as one file.

[Gandalf5]

Here is the original question:

> There is no direct documentation on the exported historical data formats in NT8. The export data page refers to the import page.

> But there are 3 options to export tick data, last, bid, and ask. Each option on the export screen gives a different format for the same time period. Below I show samples from each option? What is the format when exporting with each option?

[NinjaTrader_ChelseaB]

Hello Gandalf5,

There are 3 options to export historical data not tick replay data.

There is no option in the Historical Data window to export Tick Replay data, this window only allows for the export of historical data.

What are you trying to create an importable file for, Historical Data or Tick Replay data?

[Gandalf5]

Export -> Historical Tick Data

(I would attach a screen shot but it only allows for a link?)

[NinjaTrader_ChelseaB]

Hello Gandalf5,

To attached a screenshot click the paperclip icon to attach a file, then select the screenshot file.

If you are trying to export historical data and not tick replay data, then you are viewing the wrong part of the help guide page.


There are no differences of the text format in the file for Ask, Bid, or Last historical data (not tick replay data).

Instead, the data type in the file is controlled by the file name:

From the help guide:
"File Name
When using the NinjaTrader format, the name of the text file to be imported must be the NinjaTrader instrument name followed by a period and "Last", "Bid", or "Ask" depending on the data type. For example:

MSFT.Last.txt for Microsoft stock last price data
ES 12-09.Bid.txt for the S&P E-mini December contract bid price data
EURUSD.Ask.txt for the Euro/U.S. dollar currency pair ask price data"

"Tick Format (Second Granularity)
Each tick must be on its own line and fields must be separated by semicolon (.

The format is:
yyyyMMdd HHmmss;price;volume

Sample data:
20061107 000431;1383.00;1
20061107 000456;1383.25;25
20061107 000456;1383.25;36
20061107 000537;1383.25;14

Tick Format (Sub Second Granularity)
You can also import tick granularity to the ten millionth of a second. Each tick must be on its own line and fields must be separated by semicolon (.

The format is:
yyyyMMdd HHmmss fffffff;price;volume

Sample data: (Note: If you wanted to import in millisecond granularity data then each line must have the remaining "0"'s behind it to import correctly.)
20061107 000431 1000000;1383.00;1
20061107 000456 1000000;1383.25;25
20061107 000456 2000000;1383.25;36
20061107 000537 7000000;1383.25;14"

http://ninjatrader.com/support/helpGuides/nt8/en-us/index.html?importing.htm#UnderstandingImportFileAndDataFormats

[Gandalf5]

None of those match the formats I am getting. See below....

[NinjaTrader_ChelseaB]

Hello Gandalf5

I'm not understanding.

I am expected that you are exporting tick data.

If you are exporting historical data, the format of the text will not change.
All 3, ask, bid, and last will use the same format for the text.

The format for ask tick data is:
yyyyMMdd HHmmss;price;volume

The format for bid tick data is:
yyyyMMdd HHmmss;price;volume

The format for last tick data is:
yyyyMMdd HHmmss;price;volume

The filenames for these will be different.

The filename for ask data is:
InstrumentName.Ask.txt

The filename for bid data is:
InstrumentName.Bid.txt

The filename for last data is:
InstrumentName.Last.txt

[Gandalf5]

Right, it doesn't seem like you are able to understand and resolve the question.

[NinjaTrader_ChelseaB]

Hello Gandalf5,

What is the exact importable file you are trying to make?

What is the instrument, interval, and type?

Is this tick replay data or historical data?

I will tell you the exact file name and contents you will need to use.

[trader_rick]

A clear answer was not provided to Gandalf's question. Please could you clarify...

I have exported data from NT8 - interval type is 'Tick' and data type is 'Last'.

The resulting text file produced has the same layout as Gandalf's example above, e.g..;

20160725 120101 8340000;98;97.72;98.16;1800
20160725 122016 7050000;97.98;97.9;98.07;100
20160725 123419 2180000;98;97.87;98.05;150
20160725 132833 3480000;97.65;97.65;97.69;100
20160725 132835 3020000;97.65;97.65;97.68;200
20160725 132835 4470000;97.65;97.65;97.68;200
20160725 132835 7840000;97.65;97.65;97.68;200
20160725 132840 6490000;97.65;97.65;97.68;400
20160725 132840 6490000;97.65;97.65;97.68;100
20160725 132840 6500000;97.65;97.65;97.68;400

So I am surmising the following format;

YYYYMMDD hhmmss ddddddd;price1;price2;price3;volume

The question then becomes;

What are price1, price 2 and price3?

I presume they map to 'Bid', 'Ask' and 'Last', but which is which?

thanks