NScript to Import Historical Data

[aventeren]

Howdy--

Is it possible to import historical data via NScript as opposed to the NT GUI (i.e., Tools > Historical Data Manager... > Import)?

I am going to write a StreamReader and then StreamWriter indicator to pull daily futures data all the way back to the 1980s, format that data to the NT import format type and then write the properly formatted data to an appropriately named .txt file. Given my current understanding, I would then need to import each contract (i.e., GC 06-86.Last.txt) into NT via the Historical Data Manager GUI, which as you might imagine, will be rather labor intensive for several instruments going back to the 1980s (i.e., energy, metals, ag, currencies, etc). Therefore, I am interested if I can program the data importation process via NScript, which would save me a TON of screen time.

Thanks,

Aventeren

[NinjaTrader_PatrickH]

Hello Aventeren,

Thank you for your note.

Unfortunately, there are no supported means to achieve this in code. And if it is possible it would not be supported.

[aventeren]

Follow Up Question:
The source for my daily data reports the Volume as a string that looks like "555.0" instead of "555" (i.e., it looks like a double as opposed to an int).

When I attempt to import, will the ".0" portion of the volume cause the import to fail?

Do I first need to remove the ".0" from the volume before importing?

Thanks,

Aventeren

[aventeren]

Date Order

Also, does it matter if the lines are ordered by descending dates (i.e., 12/31/13, 12/30/13, 12/29/13, etc) or does the data need to be sorted in an ascending fashion (i.e., 12/1/13, 12/2/13, 12/3/13, etc)?

Thanks,

Aventeren

[aventeren]

Year Format of .txt file

Howdy--

NT asks for a month and year in the .txt file per the Help Guide (i.e., "ES 09-12"). Obviously this is for the ES 09 contract in 2012.

I am looking to import historical data as far back as 1959 (Wheat). So for instance, I am trying to import the ZW 12 contract in 1959. Would the correct file name be called "ZW 12-59.Last.txt"?

Thanks,

Aventeren

[aventeren]

I parsed out the ".0" from the volume and I sorted the data to be descending prior to creating the .txt file. I then created a new ZW contract called 12-59 with a rollover date of 01/14/1960 (which I will change later).

I was able to get the "ZW 12-59.Last.txt" data to import, which I confirmed via the Historical Data Manager -> Edit opened "ZW 12-59" and confirmed the data was there and it matched the source.

I then opened a ZW 12-14 contract chart and loaded 25,000 days (2014 - 1959 == 55 * 365 = 20,075 days --> so 25,000 days covers us). However the data still only goes back to 12-09 and the 12-59 data does not show up.

Given that there is a 1959 to 2009 data gap in my db, is there a setting that NT is looking to that would preclude the 12-59 contract data from showing on the chart?

Do I first need to import all of the other contracts prior to the historical data showing up?

Thanks,

Aventeren

[NinjaTrader_CalH]

Aventeren,

Have you defined a contract for that period, 12-59?

You would need to set the contract and the dates for it in the contracts defined section -
Tools -> Instrument Manager -> ZW -> Edit -> Misc Tab and scroll to the bottom.

[aventeren]

Aventeren,

Have you defined a contract for that period, 12-59?

You would need to set the contract and the dates for it in the contracts defined section -
Tools -> Instrument Manager -> ZW -> Edit -> Misc Tab and scroll to the bottom.

Yes, I did. Rollover date was 1/14/60...but maybe it should have been 9/14/59?

[NinjaTrader_CalH]

Aventeren,

Yes, the December contract typically rolls over in September of that year.

[aventeren]

So I tried to load just the ZW 12-59 contract and I got an error that the "NT data server does not support this instrument, no data available".

To confirm, I can see the data in the Historical Data Manager's Edit tab (under daily) and I created a new ZW 12-59 contract with a Rollover Date of 9.11.1959.

I am connected to Continuum.

I then disconnected from Continuum and connected to Kinetic's Free EOD connection, and I got an error message saying that the "Unable to load data, unknown instrument (panic)".

Any ideas on why I can't chart this data?

Thanks,

Aventeren

[aventeren]

I've done some digging around, and it appears that NT is looking to populate the chart data from the data server and NT will not look at the imported data if the data server does not have the instrument/contract. Therefore, ZW only goes back until 2005--and from my understanding I would need to delete all of the ZW db data via the Historical Data Manager -> Edit tab--and then NT will look first to the imported data and then back fill the missing data from the server.

I'm confused. This doesn't make sense that I would have to delete data provided by the data server to get historical data to chart.

Help please.

[aventeren]

Here are some screenshots of ZW and ZW 12-59 data, settings and error messages.

[aventeren]

So by disconnecting from all data feeds, I am able to get the data to plot.



Is there a way to have the data plot while being connected?

[NinjaTrader_PatrickH]

Hello aventeren,

Thank you for your response.

It is possible by going to Tools > Options > Data > uncheck the 'Get data from server' option > OK. Then connect to your data feed provider.

[aventeren]

Hello aventeren,

Thank you for your response.

It is possible by going to Tools > Options > Data > uncheck the 'Get data from server' option > OK. Then connect to your data feed provider.

Patrick--

I tried unchecking this box, restarted NT (with connection to Continuum) and then reloaded the ZW 12-14 daily contract with 30,000 days of data starting today. The ZW 12-59 daily data still does not show up on the chart.

Any other ideas?

Thanks,

Aventeren