Walk Forward Optimization Results Interpretation

[TOOLMachine462]

I am not understanding the results of WFO. I think the "Instruments" tab is showing the overall Profit, etc from the 'walk forward' Tab, but it has settings that do not match any of the 'Walk Forward' tab settings. Are these the new settings to use moving forward into the future?

What is the 'Instruments' tab DD from? Is this an avg of the 'Walk Forward' tab results? Where does this come from and what is the use if an avg?
Why cant you view the graphs of the overall walk forward results?

Is WFO reliable enough to trust to go live with a strategy? (After testing live in SIM) Should running strategies in Strategy Analyzer just be for fun?

Has anyone ever created a successful strategy from successful results in this WFO?

Do you get your next set of settings from just running a backtest optimization for your preferred time period?

Thanks!

[NinjaTrader_PatrickH]

Hello TOOLMachine462,

Thank you for your post.

The Instruments tab would show the weighted average of the Walk Forward Optimization.

In general, past performance would not indicate future results. While backtesting is helpful there is no guarantee a system that is profitable in backtesting will be in the real-time market. NinjaTrader Support cannot make any specific recommendations on profitable backtesting or real-time trading.

[TOOLMachine462]

Thank you for your reply! So theoretically if I am optimizing over a 14 day period and testing over a 7 day period and having favorable results, I would just run the optimizer over the last 14 days and then use those settings for the next 7 days to obtain a continuation of the favorable results, theoretically, right? Thanks!

[NinjaTrader_PatrickH]

TOOLMachine462,

You could use any set of parameters. I could not say one way or the other if one set of parameters via a test would be the best idea or not.

[TOOLMachine462]

If I wanted to manually continue into the future what the WFO was doing, I would do what I stated previously?

[koganam]

If I wanted to manually continue into the future what the WFO was doing, I would do what I stated previously?

Getting an optimization schedule is the point of running a WFO.

ref: http://ninjatrader.com/support/forum...d.php?p=412485

[TOOLMachine462]

Getting an optimization schedule is the point of running a WFO.

ref: http://ninjatrader.com/support/forum...d.php?p=412485

Thank you for your help! I have read that post several times but I'm still confused. So a person just wants to find what schedule works best on the WFO? We are talking about the optimization period and test period, right? Is it more common to use the settings it gives on the Instruments tab or to just do the most recent test period optimization and use those moving forward? I'm sorry, I'm still confused on the best practices for utilizing this info.

Thanks,
Landon

[koganam]

Thank you for your help! I have read that post several times but I'm still confused. So a person just wants to find what schedule works best on the WFO? We are talking about the optimization period and test period, right? Is it more common to use the settings it gives on the Instruments tab or to just do the most recent test period optimization and use those moving forward? I'm sorry, I'm still confused on the best practices for utilizing this info.

Thanks,
Landon

That question is answered by the last 2 responses on the original link that I gave you.

11. which strategy parameters should I use once I'm done with WFO. how can I know since they may be different at each of the WFO testing periods. - You will use the parameters that are spit out on your optimization schedule. The purpose of WFO is to find the training period/execution period combo for an optimization schedule. You use that schedule to optimize, and you use the optimization parameters that you get for the next forward trading execution period.
12. Should I take the last one and then do a backtest using those final parameters on the entire data set of several years? - That is pretty pointless, unless you have such a stable set of parameters, that it already points to the lack of need to reoptimize in the first place.

I am not sure how I can make it any clearer, but I am open to suggestions.

[TOOLMachine462]

LMAO! ok thanks

[bltdavid]

I am not sure how I can make it any clearer, but I am open to suggestions.

What, exactly, is an "optimization schedule"?

[koganam]

What, exactly, is an "optimization schedule"?

Answered in this very thread.

ref: https://ninjatrader.com/support/foru...07&postcount=3

This part:

The purpose of WFO is to find the training period/execution period combo for an optimization schedule. You use that schedule to optimize, and you use the optimization parameters that you get for the next forward trading execution period.

[bltdavid]

Answered in this very thread.

ref: https://ninjatrader.com/support/foru...07&postcount=3

This part:

Thanks, but how do I "use" the "schedule" to optimize?

I do Optimizing in Strategy Analyzer all the time, but I still don't know how the word "schedule" has anything to do with my optimizing steps in the Optimizer

Are you trying to say the From/To dates?

> The purpose of WFO is to find the training period/execution period combo
> for an optimization schedule. You use that schedule to optimize, and you use the
> optimization parameters that you get for the next forward trading
> execution period.

This is still pretty darn confusing to me.
What is a "training period/execution period" if not the From/To dates?
If these are parts of the "optimization schedule" what else does it contain?
What does "use that schedule to optimize" really mean?

(Almost sounds like we roll forward previous best parameters into a new
date period. Lol, I think I still have a lot to learn on WFO.)

What am I missing?

[koganam]

Thanks, but how do I "use" the "schedule" to optimize?

I do Optimizing in Strategy Analyzer all the time, but I still don't know how the word "schedule" has anything to do with my optimizing steps in the Optimizer

Are you trying to say the From/To dates?

> The purpose of WFO is to find the training period/execution period combo
> for an optimization schedule. You use that schedule to optimize, and you use the
> optimization parameters that you get for the next forward trading
> execution period.

This is still pretty darn confusing to me.
What is a "training period/execution period" if not the From/To dates?
If these are parts of the "optimization schedule" what else does it contain?
What does "use that schedule to optimize" really mean?

(Almost sounds like we roll forward previous best parameters into a new
date period. Lol, I think I still have a lot to learn on WFO.)

What am I missing?

WFO consists of optimizing over a period, and then using those, putatively, optimized parameters, for trading over some next, immediate period. Hence, "walk forward".The in-sample period is most often not equal to the out-of-sample period. So then, if, for example, your WFO shows that you should optimize over 28 days, and then use the parameters to trade for the next 7 days, that, by definition of the word "schedule", is a schedule for optimizing: Optimize every 28 days, and use the parameters for trading the next 7 days.

ref: https://en.wikipedia.org/wiki/Walk_forward_optimization. First paragraph, Though the shift forward of the test period is more usually done so as to include the previous out-of-sample period, not to exclude it, as described in the article.

Edit: The Wikipedia article is inconsistent. It starts by saying the training and test periods do not overlap, then goes on to describe them as overlapping, as they should.

[staycool3_a]

hi,

running WFO in n8.

does WFO have a limitation that it cannot "walk forward" on shorter periods?

in other words:

opt period=180 days
test period = 7 days

WFO doesn't even run.

if i change my test period to >= 28 days, I get results.

further,

opt period = 60 days
test period = 7 days

Gives me zero trades, it still gives me (REF 1) "optimized parameter values" for that particular period. Then, if I drill into each period's corresponding "10 best optimized values for that period" by right click and selecting, (REF 2) "view optimization results" for period X to Y: I do see trades within that period. what's strange is that, REF 1 results, suppose my optimized parameters for period X to Y (that resulted in zero trades) were for example, a = 20; b= 30 (a and b are variables); once I drill in (REF 2), the "top results" for that period, indeed is the "top" result for that period in REF 1, however, this particular backtest does generate trades using variable values a=20; b=30.

Why is this happening? Basically it seems if I shorten the "test" period to less than 28 days, I get zero trades, however, the optimizer is indeed generating the "optimized" value for that period, and once I drill into that period's optimized 10 best results (ref 2), the top performer for that period generates trades and the parameter values for that iteration match the WFO parameter values that generated zero trades.

Additionally, it seems if I do:

optmized period = 180
test period = 7

The optimizer doesn't even run. does there need to be a certain "ratio" of optimized period/test period for n8 to handle it correctly? note.... the data is there


attaching screenshots

Thanks


*EDIT;

I ran the above WFO settings

opt period = 180
test period = 7 days

on the sampleSMA strategy for n8; i did get trades. so it seems this particular issue is regarding my strategy. however, please take a look at screenshots and let me know why n8 is behaving in such way

***EDIT 2:

looks like this may be happening due to how my strategy is processing data. i'm noticing that my strategy needs atleast 12 days worth of data post my "start" period to generate a single trade. however, what's strange is that, even if I change my "barsrequiredtotrade" to 1, and the "lookback" period to infiniti, the strategy is still needing about 12 days of data to generate. within my strategy, I cannot think of anything else that might be causing this as I don't have any limits of "x bars" required etc to generate a trade. it's basic strategy.. entering based on linear regression strategies values. also I have

			if (CurrentBars[0] < 1
			|| CurrentBars[1] < 1
			|| CurrentBars[2] < 1
			|| CurrentBars[3] < 1)
			return;

***EDIT 3:

use n8 "print" function to print several variables related to my strategy, it seems every variable needs at least 12 days of data before it prints a value. so for example, if i backtest period 6/1/2017 to 6/30/2017; the first printed value in the output is on 6/12/2017. so 12 days, which varies month to month.. if i do this on 7/1/2017 to 7/30/2017, first printed value is on 7/16/2017.

this is a multi period strategy. however, i do not have any limitation on bars required to generate a trade that expands to 12 days etc. kinda confused on what to do....doesn't n8 calculate values of indicators based on historical values? in other words, if i run a backtest from 6/1/2017 to 7/30/2017, on the day of 6/1/2017, is n8 able to provide an output for ...say ... 9 day moving average of close.... using data from May 2017?

[NinjaTrader_PatrickH]

Hello calhawk01,

Thank you for your post.

The entire data set needed for the calculations must be included in the Start and End Date of the Backtest and in the case of Walk Forward Optimizations you must ensure each Optimization and Test Period would include the needed data to calculate.

Please let me know if you have any questions.