Corrupt data files

[8DTK8]

I haven't done any backtesting since the upgrade to NT7.
After a few days of getting back into it for the first time in a while, I noticed that large chunks of my data are missing. I never checked to see if the database migrated properly when I went from 6.5 to 7.

The last backtests I ran were working great with both daily and minute data since 1995. Now There are random periods as long as a month missing and it will only go back to 2007.

Any ideas what happened or more imoportantly how I can fix this?

Alternatively, if someone can inform me as to where I can get day and minute data for (ES, NQ, GC, ZB, ZN, and most of the currency) futures, I would greatly appreciate it.

Thanks in advance.

[NinjaTrader_RyanM1]

Hello 8DTK8,

Unfortunately there's no way to know what happened to your data. Historical data is provided by our supported brokers/data providers or can be imported if you already have txt files available You will get the most historical data with a data only provider like Kinetick, eSignal, or BarChart.

A complete list of our providers and the level of data supported for each is available here:

http://www.ninjatrader.com/support/helpGuides/nt7/historical_data.htm

If you have text files available, you can import if the're formatted correctly for NinjaTrader:

http://www.ninjatrader.com/support/helpGuides/nt7/importing.htm

To prevent running into situations where you can potentially lose important data, we advise to run regular backups via our inbuild backup feature in NinjaTrader. You can include historical chart data in this backup. http://www.ninjatrader.com/support/h...up_archive.htm

[8DTK8]

Thanks RyanM.

I just spoke with someone at eSig that told me that their daily data goes back 25 years, but their min. data only goes back to 2007.

Realizing tha my laziness will come shining though with my next request, does anyone know who has the most comprehensive (goes the furthest back) daily and minute data for futures out of the providers listed? http://www.ninjatrader.com/support/h...rical_data.htm

Thanks in advance.

[NinjaTrader_Tim]

Hi 8DTK8,

Kinetick offers extensive historical data. Including at least 120 days of historical tick data, two years of minute data, and ten years of daily data. Much more for popular instruments, for example the ES minutes data goes back to 2005.

[Camdo]

My data file was corrupted in similiar manner. The cause was traced to reloading data from a chart with a range beyond what the data provider provides (ie a three year chart was reloaded with right click, but the data provided only supplies 6 months of minute data). The solution is to reduce the range of the chart, then reload, then expand the range back to what it was. My provider is MB Trading.

This is a bug in NT which should be addressed.

[NinjaTrader_JoshP]

Camdo,

This is not a bug, but unfortunately is indeed a limitation to the way NinjaTrader requests data. When you explicitly try to do a download of historical data based on a date range, NT achieves this in several steps.

1. Removes old data from the range you chose
2. Downloads what it can download from the provider
3. Repopulates the range with what it was able to download

If you request a range longer than what the data provider offers it would not repopulate those sections.

There is no logic in place to allow us to detect what the boundary is for what a certain data provider does or doesn't offer.

[Camdo]

AhHA !
In that case I would suggest a warning message pop up to remind the user of the range issue and lost data consequenses you mentioned above, and allow the user a chance to opt out before the reload is executed. It is too easy to forget chart range could corrupt a database.

[NinjaTrader_JoshP]

Camdo,

Thank you for the suggestion. This limitation is already on our list for things we would like to remove in a future version of NT.