Results of Walk Forward and Backtest are Different

[SamIam]

I am using the walk forward results to do the back test with optimized variable parameter for test period but getting different results. The strategy has one variable with parameter range of 5 to 100 with 5 step. The walk forward process suggested 45 as optimized variable value for the period of 1/15/19 thru 1/21/19 with net profit of $5,890. But when I run the back test on 1/15/19 thru 1/21/19 with 45 variable parameter, the net profit comes out to be $4,330. Can someone explain why the back test and walk forward results are different for same time period and variable parameter? How much of chart data is used during the walk forward process? Is the from the start of walk forward date or just the duration of optimization period?

[NinjaTrader_Jesse]

Hello SamIam,

Thank you for the post.

From the provided details I couldn't really say what may have been different in those tests to provide much here. Are you able to see this discrepancy with the SampleMACrossOver? If so that would be helpful as we can both use that for testing to find the reason. I did a quick test on my side with the SampleMA and used one of the parameters for the walk forward. I compared the results to a backtest and see the same result for the selected period, can you try this type of test as well?

In your test what was different between the two results? The difference in profit is quite a lot which may suggest different executions or the strategy was executed differently in its logic somehow between tests. You may try running both tests in two windows and then comparing the chart for the test period to see what specifically was different, that may provide some clues as well.

I look forward to being of further assistance.

[SamIam]

Jesse,

Thank you for the response. Attached are examples of the walk forward and back test discrepancies. I have used SampleMACrossOver strategy and ran walk forward and then used optimized results for back test. The walk forward seems to have three additional trades as compared to back test for same period 4/23/19 thru 4/29/19. Its visible in both CSV and JPG files, the chart for walk forward shows trades made even before the MA crossover on 4/22/19 and 4/23/19 while in back test scenario, first trade was on 4/24/19 after the first MA cross over. Why walk forward have trade executions before the first MA crossover?

[NinjaTrader_Jesse]

Hello SamIam,

Thank you for the additional details.

I was able to see this on my end, and it appears this is mainly caused by how the walk forward works in NT7. I tested this also in NT8 and see matching results. From what I can see the BarsRequired is not used with trades in the walk forward individual results where the backtest will use this at the start of the test. Because of that we can see differences in this comparison. For this item really all I can suggest would be to upgrade to NT8 as it appears to have been corrected in this area.

I look forward to being of further assistance.

[SamIam]

Jesse,

Thank you for additional feedback. If I understood you correctly, NT7 walk forward results are inaccurate while NT8 are accurate, is that correct? An additional question on same topic, if I do upgrade to NT8 and if I want to test a strategy in live environment, how many days of data I should load while executing the strategy? New Strategy -> Time Frame -> Days to Load. If I optimize a strategy for past 14 days and want to use the optimized parameters for next 7 days, should the days to load be 14 or 1 or something else? Trying to understand how walk forward process works during optimization and testing periods, how many bars of historical data is loaded during those two phases?

[NinjaTrader_Jesse]

Hello SamIam,

I couldn't confirm that, they are simply different in this respect. There are other items which have changed between platforms since NT8 has been released. The analyzer in NT8 has had some improvements from items which were previously reported in NT7 so they will not work identically in all areas based on improvements/changes. I only noted this because the test was matching in NT8 and was using the BarsRequiredToTrade as you were expecting.

In regard to loading data, you would load the amount you need to use for your goals. This is not something that I could answer as this relates to what you personally want and what you created. For this question I could suggest to review the results of both historical and forward testing and then use your best judgment to move forward with your goal in mind.

If you are trying to compare a walk forward/backtest to have an expectation of how the script will work in realtime I would suggest to instead actually use the script in realtime and then compare the results. There are expected differences between historical tests and realtime tests so finding the optimal parameters for your custom script will be something you need to work through and develop toward your goals. https://ninjatrader.com/support/help...lightsub=discr



I look forward to being of further assistance.