Different results on Strategy Analyzer

[Robertp75]

Dear All,

I backtested a strategy using the Strategy Analyzer. I compiled the strategy without any errors. I am using a 5min ES-Emini set up. The strategy is optimized for a 5min chart set up. What I do not understand why I receive different results neither changing the code of the strategy nor changing the setting parameters? After pushing the "Run Backtest" after several minutes in the row I receive different results although no changes were done?

Is there a Ninjatrader cache which has to be deleted before running the next backtest?

Many thanks for your support.

Cheers!

Rob

[NinjaTrader_Bertrand]

Rob, do you test in connected or disconnected state here? Which broker / provider do you use?

[Robertp75]

Rob, do you test in connected or disconnected state here? Which broker / provider do you use?

Dear NinjaTrader_Bertrand,

I used it in connected state and my provider is CQG.

Cheers!

Rob

[NinjaTrader_Bertrand]

Thanks, that can explain a variation as you would reload parts of the data on subsequent testings - therefore our recommendation would be to download the data and then test in a non connected state.

[Robertp75]

Dear All,

Unfortunately backtesting the strategy in non connected state did not solve the problem.
I receive different results after pushing the 'back test' button short times in the row. Nothing changed so far. I am really despaired as this
issue does not make any logical sense. Why results are changing all time long if I either changed the underlying data nor the program code.
Below you will find an extract of the output window:
-----------------------------------------------------------------------------------------------------------------
21.05.2013 13:30:00 NT7 - Version - XYStrategy

21.05.2013 13:30:00 cstTime: 25.06.2013 18:20:35

21.05.2013 13:30:00 cetTime: 26.06.2013 01:20:35

21.05.2013 13:30:00 ts1: -6

21.05.2013 13:30:00 BarsInProgress = : 418

**NT** Error on calling 'OnBarUpdate' method for strategy 'OHLCStrategy/a7eeefb37e1842b6954e8df0ae6f8fca':
You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo]
with a value of 5 when there are only 4 bars on the chart.
-----------------------------------------------------------------------------------------------------------------
The back test before contained BarsInProgress = : 3213?? at the output window


Has anyone an idea? I really require reliable results in order to optimize the strategy.


Many thanks in advance.

Rob

[NinjaTrader_Bertrand]

Rob, I would be concerned with this OnBarUpdate() error from your script generated -

**NT** Error on calling 'OnBarUpdate' method for strategy 'OHLCStrategy/a7eeefb37e1842b6954e8df0ae6f8fca':
You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo]
with a value of 5 when there are only 4 bars on the chart.

Do you work with a single series strategy here?

[Robertp75]

Rob, I would be concerned with this OnBarUpdate() error from your script generated -

**NT** Error on calling 'OnBarUpdate' method for strategy 'OHLCStrategy/a7eeefb37e1842b6954e8df0ae6f8fca':
You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo]
with a value of 5 when there are only 4 bars on the chart.

Do you work with a single series strategy here?

Dear Bertrand,

Many thanks for your reply.
No I am not swichting betweem different series. The time frame for the calculation of the Strategy is a 5 min bar setup:

protected override void Initialize()
{ ...
Add(contract, PeriodType.Minute, 5);
...
}

but I programmed a request sequence for ticks as follows:

protected override void OnBarUpdate()
{
....
if (Bars == null)
return;

if (BarsInProgress == 1)
{...
if (TickSize < 1) rangeTotalTicks = rangeTotal * (1 / TickSize);
else if (TickSize > 1) rangeTotalTicks = rangeTotal / TickSize;
.....}

Is it possible that NT7 switches automatically to another series because
of the request to tick related paramters as described above? Could this be the issue and as a result the errors occurs?

Many thanks.

Rob

[NinjaTrader_Bertrand]

Hi Rob, not as that part is only called in the context of your first added series, the 5 min (BIP == 1).

Do you have a check for enough bars present for all your series in this script?

http://www.ninjatrader.com/support/helpGuides/nt7/currentbars.htm?zoom_highlightsub=CurrentBar

[Robertp75]

Hi Rob, not as that part is only called in the context of your first added series, the 5 min (BIP == 1).

Do you have a check for enough bars present for all your series in this script?

http://www.ninjatrader.com/support/h...sub=CurrentBar

Dear Bertrand,

Yes I have a check for enough bars present in place at my strategy code.
It seems like this is not the reason for receiving different results.
I commented out a lot of lines in order to find the code snippet which is the reason for the issue. Without any succes. I am more and more convinced that it is not a matter of the strategy code but of the data series which I have updated by third party provider three months ago.

Another evidence are the results of the output-window below:
-----------------------------------------------------------------------------------------
10.01.2013 15:05:00 NT7 - Version - OHLCStrategy

10.01.2013 15:05:00 cstTime: 02.07.2013 19:16:20

10.01.2013 15:05:00 cetTime: 03.07.2013 02:16:20

10.01.2013 15:05:00 ts1: -6

10.01.2013 15:05:00 BarsInProgress = : 1878

10.01.2013 15:05:00 TickS = : 0,25

**NT** Error on calling 'OnBarUpdate' method for strategy 'OHLCStrategy/2feeff1af25b4968a8952df1ec331844': You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
-------------------------------------------------------------------------------------------------------------------------------

Why the "PrintWithTimeStamp" date displays 10.01.2013 as the real time of backtest is/was cstTime: 02.07.2013 19:16:20 or better cetTime: 03.07.2013 02:16:20. It seems like the time stamp is frozen to a date as of 10.01.2013! Why?
Is it possible that the updated data series of the third party provider is damaged?
In case could you please provide me with a test data series of the ES futures contract?

Is it possible that NT7 is not able to calculate more than 1878 five minute bars?

Many thanks.

Cheers!

Rob

[NinjaTrader_Bertrand]

Hi Rob,

'Is it possible that NT7 is not able to calculate more than 1878 five minute bars?'

No, that would not be a limitation - for backtests on continuous contracts usually much more bars are successfully processed.

Do you have another set of data / instrument you can test against?

How would results for example for our SampleMACrossOver script look?

If you check the data for your contract in the Historical Data Manager, is it continuous or would there be gaps present?

If you like to check for comparison I could provide you with a limited data set sample for testing, please get in touch at support at ninjatrader dot com directly if interested.

[Robertp75]

Hi Rob,

'Is it possible that NT7 is not able to calculate more than 1878 five minute bars?'

No, that would not be a limitation - for backtests on continuous contracts usually much more bars are successfully processed.

Do you have another set of data / instrument you can test against?

How would results for example for our SampleMACrossOver script look?

If you check the data for your contract in the Historical Data Manager, is it continuous or would there be gaps present?

If you like to check for comparison I could provide you with a limited data set sample for testing, please get in touch at support at ninjatrader dot com directly if interested.

Dear Bertrand,

I have tested the example SampleMACrossOver and it works fine all the time. The results I
received while backtesting the SampleMACrossOver are always the same as it should be. As a result I assume that the historical data of the third party provider is correct.
After the process of elimination of my code I finally found the code snippet which is responsible for the error message as follows:

"25.01.2013 01:15:00 TickS = : 0,25

**NT** Error on calling 'OnBarUpdate' method for strategy 'OHLCStrategy/0cdd4a6977334020a2cd2ef611c3f21b': You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart."

If I ignore this code as follows:

protected override void OnBarUpdate()
{....
StartHour = (int)((double)STT / 100);
StartMinute = STT - (100 * StartHour);

DateTime st_T = new DateTime(Time[0].Year, Time[0].Month,
Time[0].Day, StartHour, StartMinute, 0, 0);
DateTime ENT = st_T;
ENT = ENT.AddMinutes(Delay);
.....}

Delay paramter is defined as int above at the #region Variables section.

the entire backtesting process will run til the ultimate date without any interruption and errors.

Do you have any idea what to change regarding the code snippet above?

Many thanks.

bye!

Rob

[NinjaTrader_Bertrand]

Would unfortunately not have any ideas here out of the box, but great you could drill it down. To actually run this here, which inputs do you work the snippet with?

[Robertp75]

Dear Bertrand,

Regarding the snippet code which is responsible for the error during the backtest:

protected override void OnBarUpdate()
{....
StartHour = (int)((double)STT / 100);
StartMinute = STT - (100 * StartHour);

DateTime st_T = new DateTime(Time[0].Year, Time[0].Month,
Time[0].Day, StartHour, StartMinute, 0, 0);
DateTime ENT = st_T;
ENT = ENT.AddMinutes(Delay);
.....}

some of the the parameters are generated at the procedure below e.g.
STT = Start_Time;
EXT = ExitTime;
Start_Time and ExitTime are defined at the Properties which are default defined as INT
The procedure below should check summer time or winter time and adjust the current time e.g.
CME opening, etc.


private void timecheck() // Summer time winter time Timecheck
{
DateTime cetTime = DateTime.Now;
DateTime timeUtc = DateTime.UtcNow;
try
{
TimeZoneInfo cstZone = TimeZoneInfo.FindSystemTimeZoneById("Central Standard Time");
DateTime cstTime = TimeZoneInfo.ConvertTimeFromUtc(timeUtc, cstZone);
PrintWithTimeStamp("cstTime: " + cstTime);
PrintWithTimeStamp("cetTime: " + cetTime);
TimeSpan ts1 = cstTime - cetTime;
PrintWithTimeStamp("ts1: " + ts1.Hours);
if (ts1.Hours == -7)
{
STT = Start_Time;
EXT = ExitTime;
//STT = 1530;
//EXT = 221000;
}
else
{
hhSTT = ((Start_Time) - (Start_Time) % 10000) / 10000;
if (hhSTT == 00)
{
STT = Start_Time + 230000;
}
else
{
STT = Start_Time - 10000; //wintertime minus one hour
}

hhEXT = ((ExitTime) - (ExitTime) % 10000) / 10000;
if (hhEXT == 00)
{
EXT = ExitTime + 230000;
}
else
{
EXT = ExitTime - 10000;
}
}
}
catch (TimeZoneNotFoundException)
{
PrintWithTimeStamp("The registry does not define the Central Standard Time zone.");
}
catch (InvalidTimeZoneException)
{
PrintWithTimeStamp("Registry data on the Central STandard Time zone has been corrupted.");
}

}

I used another technique before in order to define start_time and exit_time but in my eyes this the more elegant version. NT7 do not display an errors during compiling the code above but as I mentioned in the threads before there is an issure during backtesting.

If you find a alternative how to adjust or change the code above I would be very grateful.

Many thanks.

Bye

Rob

[NinjaTrader_JC]

Hello Rob,

Thanks for your patience, in taking a look at this nothing stands out to me.

Can you send a toy* NinjaScript code replicating the behavior so that I may look into this. You can Export it as a Source File using the step from our Help Guide at the link below.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?export.htm

You may either post the code here, or send it to support [at] ninjatrader [dot] com and put a reference to this thread in the body: http://www.ninjatrader.com/support/f...ad.php?t=58382 .
In the subject line put Attn: JC.

Happy to be of further assistance.

*Toy - basically a stripped down version of code that isn't necessarily the whole logic; making it easier to identify lines of code.