Does data just disappear?

[bluelou]

I've been running some backtests using bid/ask/last tick data and suddenly the 'last' tick data series is missing 8 trading days. I confirmed by using Historical Data Manager and I tried reloading the database but the data is "missing". I've tried restarting NT, too, but can't find the missing data.

Any ideas on where to look or can it literally just get deleted out of the database by itself?

[NinjaTrader_ChelseaB]

Hi bluelou,

Who are you connected to? (This is indicated in green in the lower left corner of the Control Center)

What instrument are you attempting to load?

What expiry are you using with this instrument?

Are there any errors in the Log tab of the Control Center?

[bluelou]

Not connected to any data vendor. This is my research PC. The missing data is 6E continuous contract data from the 1st week of 2013. There are no errors in the log. The bid and ask series are there. The Last series was there up until about an hour ago.

Any ideas?

Hi bluelou,

Who are you connected to? (This is indicated in green in the lower left corner of the Control Center)

What instrument are you attempting to load?

What expiry are you using with this instrument?

Are there any errors in the Log tab of the Control Center?

[NinjaTrader_ChelseaB]

Hello bluelou,

If you are not connected to anyone for data, where are you expecting this data to come from?

[bluelou]

Hello bluelou,

If you are not connected to anyone for data, where are you expecting this data to come from?

What? I imported the data a long time ago. It's level 1 data from 2013. It was there from 2013 up until an hour ago. I just want to know what happened to it and if disappearing data is a known issue.

[NinjaTrader_ChelseaB]

Hello bluelou,

NinjaTrader will not remove previously downloaded historical data unless you are connected to a datafeed and trying to reload all historical data and the data feed does not have the data.

No, it is not expected that if you are not connected to any connection, that historical data would be deleted from NinjaTrader unless this was deleted on purpose from the Historical Data Manager or from the Documents\NinjaTrader 7\db\tick folder.

The only time NinjaTrader will remove data automatically, is when reloading data and about to replace this with newly downloaded data from the server.

[bluelou]

Chelsea,
Regardless, it looks like about 10 days of the 'Last' tick data series disappeared. I didn't change anything. I was running backtests on that date range and suddenly trades and charts became empty.

I took empty charts to be a data problem and sure enough the data was no longer in the historical data manager nor in the folder you mentioned. It's fine since I have a backup of the data but it sucks that it appears the data is lost and there's no answer for why this happened. Hopefully, it's only a small portion of the database that's been lost.

Thx,
Lou

[NinjaTrader_ChelseaB]

Hello Lou,

Is this behavior that you can reproduce?

Do you have backup software or antivirus software that can undo detected changes?

I've not seen any other reports of historical data becoming missing without the data trying to be reloaded or erased by anti-virus software that undoes changes made by programs.

However, if you can reproduce the behavior I would be able to send this in to development to be investigated further.

[bluelou]

Chelsea,
I've already 'fixed' the problem by reimporting a copy of the data. No idea what caused the deletion since I was just running Strategy Analyzer. Also, I don't think I could reproduce it since all I did was run a backtest.

Maybe the answer is to make sure you backup your data

Thx,
Lou