NT7 Error: "Historical data in your database is corrupted"

[avimop]

Hi,

When I'm trying to perform a strategy analysis, a dialog with error "Historical data in your database is corrupted" pops up.

This occurs when using the "US Equities ETH" or "US Equities RTH" session templates and doesn't occur when using the"Default 24/7" session template.

From analyzing the exception as shown in the trace file attached below, I can see that it happens due to illegal date/time format in 2012-03-30 which is exactly the date where my local time has switched to DST. My local time zone is GMT+2 and my strategy is designed to work in GMT-5 (US Equities).

What can I do to avoid this issue? currently as a workaround I'm working with the "Default 24/7" session template but I would really like to set the correct session template which is "US Equities ETH".

Please Advise
Thanks.


Log file:
9/23/2012 12:08:10|0|4|Historical data in your database is corrupted.
Please reload historical minute data for 'SPY' from 8/19/2011 to 9/15/2012.
Tools->Historical Data->Reload...

Trace file:
2012-09-23 12:08:09:639 ERROR: Data.BarsBytes.FromBytes: System.ArgumentException: The supplied DateTime represents an invalid time. For example, when the clock is adjusted forward, any time in the period that is skipped is invalid.
Parameter name: dateTime
at System.TimeZoneInfo.ConvertTime(DateTime dateTime, TimeZoneInfo sourceTimeZone, TimeZoneInfo destinationTimeZone, TimeZoneInfoOptions flags)
at System.TimeZoneInfo.ConvertTime(DateTime dateTime, TimeZoneInfo sourceTimeZone, TimeZoneInfo destinationTimeZone)
at NinjaTrader.Data.BarsBytes.FromBytes(Bytes bytes, Boolean fromEst, Bars bars, DateTime minTime, Boolean suppressSessionHandling) idx=624 (count-1)=729 buf.InLength=5101 dateTimeMaxDay=2012-03-30 instrument='SPY' from='2011-08-19' to='2012-09-15' period=1 Min splitAdjusted=False dividendAdjusted=False bars=103280
2012-09-23 12:08:09:639 ERROR: Data.Bars.Load2: System.ArgumentException: The supplied DateTime represents an invalid time. For example, when the clock is adjusted forward, any time in the period that is skipped is invalid.
Parameter name: dateTime
at NinjaTrader.Data.BarsBytes.FromBytes(Bytes bytes, Boolean fromEst, Bars bars, DateTime minTime, Boolean suppressSessionHandling)
at NinjaTrader.Data.Bars.Load()

[NinjaTrader_AdamP]

Hello,

Who is your data provider here? I'd like to see if I can replicate this.

[avimop]

Hi Adam,
I'm using Kinetick. The instrument is SPY.
Thanks

[NinjaTrader_AdamP]

avimop,

Unfortunately I am not able to replicate this, however it could be my steps are not matching up.

I basically downloaded SPY data from 8/19/2011 to 9/15/2012 and then did a backtest using the simple SampleMACrossover strategy. I used Default 24/7, as well as US Equities RTH/ETH.

I would suggest doing two things here.

1) Open a new SPY chart, load this date range and then right click and go to "Reload all historical data"

2) Go to Tools > Options > Data > Repair Database

Let me know if the issue persists, or if you have a set of replication steps you can repeat (and please let me know these steps).

[avimop]

Hi Adam,
The problem occurs when I set the time zone of my desktop to (UTC+02:00) Jerusalem and use the session template US Equities RTH/ETH. When I use the Default 24/7 session template it doesn't happen.
From the exception you can see that it occurred when the time was changed to DST on 03-30-2012.
Can you please try to replicate this behavior on your PC?
Thanks
Avi

[NinjaTrader_AdamP]

avimop,

Thanks for the replication steps. Unfortunately I am still not able to replicate.

1. I set my timezone to UTC + 2 - Jerusalem

2. I open a chart for SPY through the date ranges you specified and "reload all historical data" (this is done to refresh what I have stored on my PC)

3. I open a strategy analyzer and backtest the range 8/1/2011 to 9/24/2012

4. I backtest using Default 24/7 , US Equities RTH, and US Equities ETH.

However I am not able to get the same result.

What date format do you have setup on your PC? I know here in the US we use MM/DD/YYYY

[avimop]

Hi Adam
Please find attached the time format I use - I believe its the same like yours more or less.
The only thing I did different is ti use the historical data manager to download the SPY historical data, please find attached the configuration I've used.
Does it make any difference?
Thanks for you help,
Avi

[avimop]

Adam,
I've tried the same procedure with AA and got the same exception. It happened only when using "US Equities ETH" (US Equities RTH worked fine for both SPY and AA).
Avi

[NinjaTrader_Joydeep]

Hello avimop,
Thanks for your note.

I can replicate the scenario at my end and will send it to development for further investigation.

Thanks for your patience.

[avimop]

Thanks,
Looking forward for a solution to this issue.
Avi

[NinjaTrader_Brett]

Hello,

Thanks for your patience while we researched this.

This is due to the time zone having a daylight saving time in the middle of the week(Friday). We will resolve this in our next major release of NinjaTrader however as a workaround for now I suggest you change the Time Zone on your PC to be a time zone with the same UTC offset that does not have DST in the middle of a week such as (Istanbul, Nikosia) for example.

-Brett