Backtesting and indicators

[arnonmoscona]

Hi!

This may be simple, but I cannot find the answer...

I have a strategy that relies on several indicators. Some of them need a large number of bars (about 4000) at minimum to function. When I load the strategy in a chart or I run it directly from the strategies tab, I have a Days to load field that allows me to ensure that there is enough data to calculate my indicators.

However, in back testing, there is no such field. There is a field called Min. bars required, which implies the same thing but does not seem to work the same way. Regardless of what I put in the field, it seems that the strategy analyzer does not load bars for days that are not covered by the time period.

So if for instance my indicator needs 2 weeks of 1min data and I set the backtest period to just a few days, then my indicator never gets a CurrentBar value that is high enough for it... and the strategy appears not to work (the indicator in this case will always return zero, which the strategy does not like and so does not enter). If I increase the Min. bars required to 4000, I get the exact same result (including the indicator getting the exact same CurrentBar value as it does when I leave the field in the default 20. If I extend the period to be something like 2.5 weeks, then the backtest actually produces correct results for the last 2 days of the period (anything before that does not have a high enough value of CurrentBar)

So how do I setup a back test such that every day that is specified in the backtest period will get enough bars for the indicators to calculate correctly?

[NinjaTrader_AdamP]

arnonmoscona,

2 weeks of minute data would be 20160 bars.

I would suggest using the following property to see if you can remedy your issue.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?maximumbarslookback.htm

You can set maximum bars to look back to infinite if you wish.

[arnonmoscona]

I already set MaximumBarsLookBack to infinite (quite regularly), and it is also set to infinite in the backtest settings (for the strategy). Now, this is a strategy on a US Stocks. So it has 390 minutes/session. So two weeks (10 sessions) is only 3900 bars.

So bottom line - the answer appears to be incorrect.

[NinjaTrader_RyanM1]

arnonmoscona,

The field you're looking for in backtesting is the TimeFrame settings. Instead of days to load, set the From date far enough back to load all the data you need.

[arnonmoscona]

That was exactly my point. When I select a time frame, this is the time frame that the strategy should actually function.

What you are saying is that I need to calculate a wider time frame than what I really want to make sure that all dependent indicators have enough data. Basically what you're saying is that my observation was accurate in that NT provides no facility in backtesting to ensure that enough data is available to indicators that the strategy depends on at the beginning (or any day) in the requested time frame and that the user needs to cushion the time frame with enough days such that indicators will return correct values.

In many ways this entirely invalidates the simulation (as well as any optimization) and should be considered a serious bug. When you develop a strategy and run it in backtesting or optimization, you are exposed to indicators (possibly unknown to you) returning some incorrect value during some period at the beginning of the time frame simply due to the fact that they do not have enough data. In turn that can cause the strategy to behave differently that it would have behaved in real time during the same day.

The whole point of a simulation or optimization is that the strategy should behave as close to its real time behavior as possible. While it is true that there must be differences due to fulfillment being simulated. But there should not be differences due to how NT provisions data. In real time the data provided is governed by Days to load parameter of the chart or strategy (when it's instantiated in the strategy tab). NT's failure to provide this parameter in backtesting and optimization, and instead choosing to infer the value (incorrectly) from the time frame, is a serious bug that invalidates the results of the simulation. This is not a feature, but a bug that must be fixed as we have absolutely no work around for it. I cannot see any way that I can ensure that indicators running under a strategy in backtesting can be ensured to get the same data that they would get in real time...

Please submit this as a serious bug to your development team.

[NinjaTrader_Brett]

We are looking into this for our next major release of our software to see if anything can be done here.

As for now Ryan is correct. You need to make sure that you load enough days back to load all secondary indicators/time frames so the strategy will run correctly.

You can guard against this in your own code with Correct CurrentBar checks currently.

http://www.ninjatrader.com/support/helpGuides/nt7/currentbars.htm

[GuppyDRV]

An old thread I know. Has there been any update to this in NT8? I am guessing that this issue if still out there would effect an Walk Forward Optimization on each and every OOS period calculated.

Any help would be appreciated.

GuppyDRV

[NinjaTrader_Brett]

We did not in principle change how this worked in NinjaTrader 8. We however did improve and resolve an issue around 'bleedover' where a session in NinjaTrader 7 could bleed into the next based on certain data loading time periods and time zones which would be a compounded problem with walk forward optimizations. With NinjaTrader 8 we consistently load trading sessions based on the trading hour templates now so you get a clean cut of data from start to end on walk forward optimizations.

I recommend just ensuring that you have the data back from your provider by doing a standard backtest/chart to ensure you have all the data you need for running optimizations.