Walk forward 1 day opt/test periods. Extra trades in test days skewing result [FIXED]

[volkie]

Want to post this in case anyone else runs into a similar problem. A fix was suggested (see below) and works for me.

During my walk forward optimizations on an fx currency pair I was seeing trades from the end of optimization period bleeding into the beginning testing period, which was skewing the results. Sometimes the day following the test period also bled in, resulting in 1-2 extra days per walk forward test period.

Since extra days and seen & optimized-for trades were showing up in the unseen test portion of the Walk Forward test, it made the algorithm look way more profitable than it otherwise would be. This is more pronounced the smaller your window sizes are.

I included some screenshots of the results I was getting. These were for 1 day optimization / 1 day test periods and you can see extra trades bleeding in.

Update: Found a solution thanks to NinjaTrader_Brett. Turns out your time zone plays a role in causing/fixing this one.

He advised me to copy the 'Default 24/7' session template and set it to my timezone (Pacific). Previously I was using the default Forex session template set to Eastern Standard time, which is the same time zone as the historical Fx data I test against. Then in the Walk forward settings you override the session template to use this Modified 24/7.

Probably a good thing to check in your own Walk Forward results if you're like me and your time zone isn't EST.

After using this PST 24/7 template I only see trades in the test period from the test day(s) as expected. Thanks Brett!

[volkie]

Saturdays

Update:

Still seeing one last issue. All Saturdays in my Walk Forward testing include Friday's session/trades. Probably because Fx isn't open on Saturdays so it's grabbing the Friday data because it's being asked to trade for a session that doesn't exist.

Copying the default 24/7 works for fixing 99% of this issue but I think leaving the Saturday session in is causing this last minor bleed over. I will try removing Saturday and see how it goes.

[volkie]

I will try removing Saturday and see how it goes.

This did not fix the error when using 1 day windows. I still get Saturday test periods using Friday's data (which it was optimized against) but I suspect this is probably isolated to 1 day windows and might not occur on more reasonable 2 day+ window sizes. I didn't test though.

I decided to just write a small php script to filter out any Saturday rows to get the true results. I look at all WF result data in my own spreadsheet with graphs etc anyways so this works for me. Attached it if anyone needs, just remove the .txt part, then run it in your terminal with "php fix_sat.php <in.csv> <out.csv>"

[NinjaTrader_Brett]

Volkie,

Thanks for this positive and constructive posts.

Glad to hear you were able to get the fine granularity you were looking for in back testing.

-Brett

[volkie]

I have yet another update, think I found the last thing I needed to iron out:

I did some Walk forwards with 180 opt period and 30 day test period, a much more normal test case. In this situation did notice an extra stray trading day from the optimization period ocassionally showing up in the 30 day test period if it started on a Saturday.

I did one final tweak to my Session Template and I no longer see any stray trades and am very happy with results I'm getting back. I've included a screenshot of the Fx Session Template I now use that produces no strays / bleeding over trades.

So the final fix was using an Fx Session template setup as my screenshot shows, and using my local time zone (Pacific), as Brett suggested, instead of Eastern. Thanks for the help finding the optimal settings. Results are looking good now

[NinjaTrader_AdamP]

volkie,

Please don't hesitate to contact us should you require additional assistance.

[mumuto1988]

How to copy those time zones

I have the same issue, how can I adjust my timezone.

[NinjaTrader_ChelseaB]

Hello mumuto1988,

You can make a copy of the Default 24/7 template by:

• Click Tools -> Session Manager...
• Select Default 24/7
• Click the Copy... button -> give the new template a unique name -> click OK
• Set the time zone to your time zone in the 'The sessions above are based on the following time zone...' drop-down
• Click the Save button -> click OK

[mumuto1988]

Thank you very much.

[mumuto1988]

After the I create my own session, the result is one day before the walk forward date.

[NinjaTrader_ChelseaB]

Hello,

This is likely due to the session bleed over effect.

What are the dates and times of your custom session template?

What parameters did you use for the backtest?

[mumuto1988]

Here are the analyzer and session image

[NinjaTrader_ChelseaB]

Hello mumuto1988,

Are these the parameters that were used for the previous backtest?

I ask because the dates chosen in the parameters of the screenshot are from 1/1/2014 to 2/6/2014 while the orders seem to start on 4/11/2013.

Also, the bleed over effect would be expected here.

Lets say that you have chosen 2/4/2014 (Tuesday) as the start date. The session template is from Monday at 12:00 AM to Tuesday at 12:00 AM. The analyzer will load the full calendar day for all days that the sessions bleed into. So by adding Tuesday this will include Monday as Mondays session ends on and includes Tuesday.

To prevent Monday from being selected, your session template would need to be from Monday 12:00 AM to Monday at 11:59 PM, Tuesday 12:00 AM to Tuesday 11:59 PM etc.