WalkForward - fwd-testing top 5 backtested sets instead of top 1?

[kc0793]

Hi,

Walk forward does this - runs the strategy on the parameter space for the 'in-sample' period, selects the best performing set of parameters, and uses that set of parameters for the 'out-of-sample' period test. Is there any way to customize the walk-forward process such that instead of taking the best performing set of parameters, we select the top 5 best performing set of parameters, and do the 'out-of-sample' test on all 5 of them?

Thanks

[NinjaTrader_PatrickH]

Hello kc0793,

Thank you for your post and welcome to the NinjaTrader Support Forum!

The 'Keep best # results' option will allow you to set the number of results you wish to keep as the best. However, all combinations will be tested in the WFO backtest.

For more information on WFO please visit the following link: http://www.ninjatrader.com/support/h...e_a_strate.htm

Please let me know if you have any questions.

[kc0793]

Not sure I understand. Lets take an example - lets say I am testing a strategy for the period Jan 13 to Jun 14 (1.5 years). The WFO periods are set at 1 year and 6 months - i.e, optimize for Jan 13 to Dec 13, and then using the optimized parameter, forward test Jan 14 to Jun 14.

Are you saying that if I set 'Keep best # results' to 5, the forward test from Jan 14 to Jun 14 will use the optimized parameters for each of the 5 best optimized results, and essentially do 5 forward tests?

So for each optimization period, there will be 5 forward tests? If using the same period of data (Jan 13 to Jun 14), I now set the periods at 6 months, 3 months, is this how the WFO will behave -

1) Optimize for period Jan 13 to Jun 13, forward test Jul 13 to Sep 13 five times using the 5 best optimized parameter sets.
2) Optimize for period Apr 13 to Sep13, forward test Oct13 to Dec13 five times using the 5 best optimized parameter sets.
3) and so on and so forth............

[NinjaTrader_PatrickH]

Hello kc0793,

Thank you for your response.

The WFO will run an optimization for each combination of parameters and then take the top performing optimizations based on the number set for 'Keep best # results' and run this over the walk forward period.

[kc0793]

Hi Patrick, thanks for coming back so quickly. I think you seem to be agreeing with my previous post, but I in fact don't believe that is what Ninjatrader actually does. As an example, I am attaching 2 screenshots of a WFO - 1st one being the optimizer section, 2nd one Walk Forward. I have set 'keep best no of results' to 2. So, in the first screenshot, you can see the 2 best optimized parameter sets are 110/10/87/100 and 110/10/91/100. But in the walk forward, only the top parameter set (110/10/87/100) is used.My question originally was - how can I do the walk forward for both my parameter sets (110/10/87/100 and 110/10/91/100)?

[NinjaTrader_PatrickH]

Hello kc0793,

Thank you for your response.

The WFO will run the optimizations base don the optimization period set, then it will keep the best results based on the 'Keep best # results' setting for each run of the optimization period. Then the WFO tab will show the best out of sample runs of that collection based on your setting for Optimize On.

So the answer to your questions would be no, you can not customize what parameters would be ran over the out of sample period solely based on the parameters. They would need to be the best results over the optimization period.