WalkForward issue...

[kenb2004]

I've noticed the the WalkForward Optimization always includes weekend days in it's Optimization Period. I have been using a custom 24/5 session template which starts at Monday 12:00AM and ends at Monday 11:59PM. This is repeated Monday thru Friday. But when I looked at the Optimization tab, the weekend days are alway included. My Optimization Period is 4 and my Test period is 1. I also use a time filter 9:30AM to 4:15PM in my strategy. So the end result is that is that if Monday is the Test Period day it is really only Optimizing 2 days instead of 4.

So how can I exclude weekend days in the WalkForward Optimization, since the 24/5 session is having no affect on excluding 2 weekend days like it should?
Thanks

[NinjaTrader_JC]

Hello kenb2004,

That is to be expected. If you are using custom code you can do something like:

 if (Time[0].DayOfWeek == DayOfWeek.Saturday ||  Time[0].DayOfWeek == DayOfWeek.Sunday)
        return;

To be able to filter out the weekend days.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?dayofweek.htm

[kenb2004]

JC
When my strategy runs it only runs Monday thru Friday ES 930am to 415pm now. How will this have any effect to Walk Forward Optimization if a session template doesn't? I'll give it a try, maybe you should also to verify your suggestion. I think this may be a bug that needs to be addressed. It seems as though days of the week are NOT RECOGNIZED at all in the Walk Forward Optimization or may the Optimization module also.
Thanks

[NinjaTrader_JC]

Hello kenb2004,

To clarify, what you are seeing is how a Session Template is being handled. For example using the "Default 24/5" session template it is set to use the "Eastern Time", and if your PC is set to "Central Time" then you may see trades on Sunday after 11:00PM which would be Monday 12:00AM "Eastern Time". We are looking at changing the way that Session Templates are being handled in NT8.

If you like you may change the "Time Zone" of the session template to what your local PC Time Clock is to better line up the times or vise versa.

Please let me know if you have any questions about this.

[kenb2004]

JC

TimeZone has nothing to do with this issue. This is a number of days vs Days of the week issue. WFO uses NUMBER OF DAYS and does not take into account DAYS OF THE WEEK. NUMBER OF DAYS is the ONLY thing that WFO looks at when it sees the 28 (default value) in Optimization Period and the Test Period. This means that of those 28 days WFO is really only optimizing about 20. And when you shorten these period to 4:1 like I am attempting to do, the results range from 2:1 to 4:1. WFO is a great tool but this is a major flaw! Please forward this info to development. The issue remains that WFO needs to be able to exclude weekend days from Optimization Period count.

BTW my PC is set to EST and so is 24/5 template

Thanks

[NinjaTrader_Matthew]

Ken,



Thanks for the clarification.

We are aware that the number of days are not linked to the session template and 28 days would include weekend, thus results in 20 days in that period. This would be improved in the next major release of NinjaTrader where we addressing certain limitations with the session templates.

[kenb2004]

Since NT8, according to Raymond, is only 80% done and Alpha testing hasn't even begun, and you will be supporting NT7 for many years in the future, how about a fix or work around to deal with this present day problem instead of just ignoring it for months, since this issue renders WFO results wrong 30% of the time, based on default 28 day input value and wrong 80% of the time with my 4:1 input?

Thanks

[NinjaTrader_Matthew]

We do not have any any workarounds at this time. Our development team is focusing their efforts on the next major release.

[kenb2004]

If Raymond, NinjaTrader CEO, says you will be supporting NT7 for years to come, how is only focusing development of NT8 when there is a known issue, bug, flaw, whatever you want to call it, with NT7 and NOT fixing the problem, supporting NT7 "for years to come"?

Especially when the solution to the problem could be carried over to NT8 anyway?

Thanks

[NinjaTrader_Matthew]

Ken,

I understand your concerns.

This has been the case since day one of NinjaTrader 7 and WFO. We do not get enough reports of this to focus efforts in changing this in NinjaTader 7, but I can assure you that it is on our list for testing in NinjaTrader 8. Should this carry over to NinjaTrader 8, it should be caught by our development team and fixed during this process as we are rebuilding NinjaTrader 8 from the ground up. This gives us the opportunities to look into these sort of little nuances and approach them from another angle.

If you recall from Ray's webinar, he says that NinjaTrader 7 is in "emergency maintenance mode" which he goes on to explain that we will only be providing major an required fixes in NinjaTrader 7. This would be something like an adapter change to comply with exchange changes, or updates that may be required by Microsoft etc.

While this behavior in our WFO in NinjaTrader 7 is not ideal for you, we simply do not hear enough about it to fix in this version. We do not want to sit on a broken feature and we take customer reports like yours very seriously and this will be looked at in NinjaTrader 8.

Thank you for understanding

[kenb2004]

Daytraders Call To Action!!! Fix WFO in NT7!!!

Mathew

I have spent the last 2 years learning the INs and OUTs of NinjaTrader and developing a solid strategy, inspite of NinjaTraders short-comings as well as my own. As most traders know the Markets are dynamic and in order to have a successful system that system also needs to be dynamic, hence WFO. In order to achieve a dynamic system forward testing is the Key. This is the last leg of the journey and the most critical and you are saying it's a "little nuance". Nothing could be further from the truth.

You also say "we simply do not hear enough about it to fix in this version". I would encourage any others who are reading this forum who understand the utmost importance of Forward Testing, that you CHIME IN PLEASE, and let NinjaTrader Support know that we can't just sit on our hands and wait for a Newer Version sometime in the future ??? that will address this issue.

For anyone who day trades and needs to Forward Test to optimize your strategy, this WFO just DOES NOT WORK the way it is. I'm sure as talented as the NinjaTrader programmers are, this would not take too much to resolve the problem IF it was addressed and as I mentioned before The Fix could be carried over to NT8 anyway.

So, please DayTraders, ask NinjaTrader support to Fix WFO in NT7!!!
They NEED to HEAR from YOU!

Thanks

[koganam]

Mathew

I have spent the last 2 years learning the INs and OUTs of NinjaTrader and developing a solid strategy, inspite of NinjaTraders short-comings as well as my own. As most traders know the Markets are dynamic and in order to have a successful system that system also needs to be dynamic, hence WFO. In order to achieve a dynamic system forward testing is the Key. This is the last leg of the journey and the most critical and you are saying it's a "little nuance". Nothing could be further from the truth.

You also say "we simply do not hear enough about it to fix in this version". I would encourage any others who are reading this forum who understand the utmost importance of Forward Testing, that you CHIME IN PLEASE, and let NinjaTrader Support know that we can't just sit on our hands and wait for a Newer Version sometime in the future ??? that will address this issue.

For anyone who day trades and needs to Forward Test to optimize your strategy, this WFO just DOES NOT WORK the way it is. I'm sure as talented as the NinjaTrader programmers are, this would not take too much to resolve the problem IF it was addressed and as I mentioned before The Fix could be carried over to NT8 anyway.

So, please DayTraders, ask NinjaTrader support to Fix WFO in NT7!!!
They NEED to HEAR from YOU!

Thanks

I understand your concerns because they were mine too, but rather than insist, and have to wait, I just swallowed the pill and do my WF by specifying factors of 7, knowing that that guarantees me factors of 5. So if I want 5 days, I just specify 7.

To my mind, ANY kind of optimization is still curve-fitting of a kind, as it is dealing with past data, with perfect hindsight, so I just think that 4 is close enough to 5 that I will be within the bounds of variability.

That having been said, I am still waiting and hoping that it will be fixed in NT8. I have a high tolerance for hopium, induced by these crazy markets. Not!

[kenb2004]

koganam

Thanks for the consoling thoughts. Call me crazy but I just think that if you do something, you do it right and kicking the can down the road isn't "right".

Thanks

[Camdo]

I too am disappointed with Walk Forward Optimization.

1. The Help section inadequately describes what this module is doing. The video just repeats what the written version says. There needs to be a detailed explaination of how the test results are derived. After spending a lot of time trying to figure it out, my conclusions are shown in the attached pictogram. Any comments as to its accuracy would be appreciated.


2.Trades within the test results overlap, so the total cummulative profit for the WFO period is erroneous. Thus one does not know if the optimized parameters would be effective as an adaptive strategy.

3. The session template is ignored by WFO. All days are used, even weekends. There needs to be an explaination if this is affecting parameter optimazation.

4. Minimun Bars cuts into the optimazation period, but not into the test period. Can someone confirm this? I would like to suggest that minimum bars be prefixed to the optimization period so as not burden the user with housekeeping.

5. There needs to be a graph of cumulative profits for the WFO time frame. Since this is not available, I assume development must be aware of the limitations expressed in point 2 above.

6. For the above reasons WFO is useless.

[koganam]

I too am disappointed with Walk Forward Optimization.

1. The Help section inadequately describes what this module is doing. The video just repeats what the written version says. There needs to be a detailed explaination of how the test results are derived. After spending a lot of time trying to figure it out, my conclusions are shown in the attached pictogram. Any comments as to its accuracy would be appreciated.
[ATTACH]24605[/ATTACH] - That diagram matches my conclusions, and would be a correct description for a continuous WF test.

2.Trades within the test results overlap, so the total cummulative profit for the WFO period is erroneous. Thus one does not know if the optimized parameters would be effective as an adaptive strategy. - I do not see why this is an issue. They would overlap, as you are testing/optimizing on a rolling basis, so the last day of the WF period becomes the last day of optimization for the next period's trades. This actually looks correct to me, but I am willing to be corrected.

3. The session template is ignored by WFO. All days are used, even weekends. There needs to be an explaination if this is affecting parameter optimazation. - I believe that my original response of quantized period selection applies, but I think that if we do not use factors of 7, so as to get factors of 5 for WF period, this may cause a variation in results that depends on the start of the optimization period. However, I have not run a test, and I just feel that in the grand scheme of things, while it may make a difference in absolute values, if it actually made a large difference in returns, then the Strategy is more likely to be defective, that a problem with NT WFO.

4. Minimun Bars cuts into the optimazation period, but not into the test period. Can someone confirm this? I would like to suggest that minimum bars be prefixed to the optimization period so as not burden the user with housekeeping. - Should not be an issue. The idea is to test steady state values, and as we are on an overlapping running basis, we want only one initial period, if any: definitely, not an initial unstable period for each shift forward in test period.

5. There needs to be a graph of cumulative profits for the WFO time frame. - I absolutely agree. Manually adding those to see if one would have been profitable, and by how much, most definitely sucks. Since this is not available, I assume development must be aware of the limitations expressed in point 2 above.

6. For the above reasons WFO is useless. - Hm. Somewhat limited. Maybe even severely limited, but IMHO, not quite useless.

My thoughts in blue.