Bug: Walk-Forward Optimization and BarsRequiredToTrade

[GrumpyTrader]

The walk-forward optimizer is not working correctly when you take into account the BarsRequiredToTrade parameter. The BarsRequiredToTrade parameter seems to be applied to the beginning of the "Test Period" when it should only be applied to the "Optimization Period".

For example, if I run a WFO with the following settings:
a) optimization period of 365 days
b) test period of 90 days
c) and the strategy runs on daily bars, with a BarsRequiredToTrade setting of 30

Currently, the dataseries gets processed like this:
<- BarsRequiredToTrade (30 bars) -><- Optimization Period (365 minus 30) -><- BarsRequiredToTrade (30 bars) -><- test period (90 - 30 ) ->

The "Test Period" actual executions should be contiguous to the "Optimization Period" by having the Test Period's dataseries padded with more bars before the start of the Test Period equal to the BarsRequiredToTrade parameter so trades can be taken immediately following the Optimization Period.

As it works now, if my BarsRequiredToTrade for the example above was 90. You would never get any executions and the walk-forward optimization would return zero trades.

[NinjaTrader_ZacharyG]

Hello GrumpyTrader,

Thank you for writing in.

I will be investigating this further and will return with my findings.

[GrumpyTrader]

Hi Zachary any luck with this? I'm at a standstill on this.

[NinjaTrader_ZacharyG]

Hello GrumpyTrader,

I am still looking into this.

Thank you for your patience.

[NinjaTrader_ZacharyG]

Hello GrumpyTrader,

Thank you for your patience.

The BarsRequiredToTrade parameter is actually a part of the strategy logic rather than the optimizer logic itself.

Therefore, this parameter is in play during each optimization iteration as well as on the test period.

Please, let us know if we may be of further assistance.

[GrumpyTrader]

Sorry Zach but the mechanics for how the walkforward optimizer works is wrong. Did you check the example I gave? If you have a long BarsRequiredToTrade parameter you don't get any results !

Think about it this way. A walk forward test is suppose to simulate the process of
a) optimizing your strategy using in-sample data
b) running the strategy using out-of-sample data (or running your system live)
c) then after out-of-sample period is over, re-optimizing using new in-sample

The out-of-sample periods have to run adjacent to each other as if you are running a strategy live. As it works now (unless BarsRequiredToTrade is zero) you always have a gap between the out-of-sample periods, which makes no sense in the real world.

Additionally, there should be an equality between running a single backtest with one set of parameters and a walk forward test using one set of parameters. As it works now they will never be equal because of the BarsRequireToTrade gap.

The error lies in the optimizer logic because of how it breaks up the dataseries. It should be referencing the entire dataseries with logic applied to filter the out of sample period for trades. That way the BarsRequiredToTrade would only come into play once.

The quickest way to run a walk-forward test is actually to run a single optimization on the dataseries, then filter the trades by walkforward period and calculate each step. There is no need to actually break up the dataseries or run iterative backtests.

[GrumpyTrader]

Zach,
Will at least acknowledge that if I am running a daily strategy that requires 200 days of bars before trading (because for example it might be using the 200 day average in it's rules) that I will receive no trades for the first 200 days of each out of sample period.. Further more, if each out of sample period is set to 200 days (almost a year of trading) I will receive no executions for this walk forward test.

This limitation is a serious flaw for anyone doing these types of backtests. It is clearly not in the documentation and quite honestly invalidates any walk-forward test results. Your response so far is "yes thats how it works". What I am trying to discuss "Is that how it should work?"

[NinjaTrader_ZacharyG]

Hello GrumpyTrader,

Thank you for the additional information.

I'm looking into this further and will return with more information.

Thank you for your patience.