Walk Forward Bugs, Display and Data Series

[EC_Chris]

Hello,

I am running into some issues with the Walk Forward Optimizer. This is a pretty basic walk forward, these are the parameters:

Optimize on: Max Profit Factor
Optimize Data Series: True

So I am only optimizing on the data series over the walk forward.

Data Series. Minute 5-60 min, in 5 steps
Optimization Period: 90
Test period (days): 30

Issue #1. The first issue is the display on the Walk Forward is showing a different parameter than the optimizer actually used. The Walk Forward is saying that the data series used was 5 min, but in fact when you pull up the chart it is using 60 min. I anticipate this doesn't impact results much, but the next issue does.

Issue#2. When I run the last optimization window of 90 days for the current parameters (3/3/18 - 5/2/18) based on the same optimization of the data series alone, the optimal value is totally different from what the Walk Forward gave. The optimizer is saying 35 minutes is the best data series and is actually saying 60 minutes was the worst performer.

Any idea what may be causing these bugs? Are there any open tickets on this? Just not sure how to structure a proper walk forward when I am seeing such discrepancies.

Thanks

[NinjaTrader_ChelseaB]

Hello EC_Chris,

Thank you for reporting the behavior.

With the first issue I am able to reproduce and I have reported the behavior to our development.

Once I have more information I will update this thread.

With the second issue I'd like to clarify.

Are you comparing a regular optimization with a walk forward optimization and these are producing different best results?
(This would be expected)

Are you comparing the pnl of a specific result with a backtest using the same parameters and this is giving a different pnl?

[EC_Chris]

Clarifying your questions on the second issue. I had a slight error in my dates, but there is still an issue and I think it has to do with the period bug. I am attaching new screen shots.Please use the CYAN colored markers to reference the image.

Here is what is happening:

In Image #1, I am showing the Walk Forward Optimization ran from 1/1/2007-5/15/2018. Again, we are only optimizing on data series for Profit Factor with these parameters:

Data Series. Minute 5-60 min, in 5 steps
Optimization Period: 90
Test period (days): 30

For the last period in the walk forward, we would be using the Optimized Profit Factor for the data series for an Optimized Period of 2/1/2018-5/2/2018.

For Image #2, I am running the optimization for that same window, with the exact same settings optimizing for Profit Factor.

As you see the optimized value ends up being the 5 minute timeframe. So this would relate to the prior bug in issue #1 where it saying the right 5 minute value but using an incorrect value for the actual results.

Does this make sense?

[NinjaTrader_ChelseaB]

Hello EC_Chris,

I afraid I'm still not quite following..

Is this a separate issue that is different than the first issue I have reported?

Are these images both walk forward optimizations?

Is the issue that you've run a walk forward optimization twice using the same parameters and data and the results have changed?

[EC_Chris]

The issue is related, and maybe is the same thing.

The first image is a walk forward optimization, the second image a stand optimization.

The issue is that the walk forward optimization is using a 60 minute value in it's results for the last window, when based on the optimization it should be using a 5 minute value. So this may be the same thing you already reported.

[NinjaTrader_PatrickH]

Hello EC_Chris,

Thank you for your report.

We have reported this to our development team. We will update you when we have any details.

[NinjaTrader_ChelseaB]

Hello EC_Chris,

I've received tracking ID #NTEIGHT-12830 for this behavior.

As new releases of NinjaTrader 8 become available please check the release notes for this ID from the publicly available link below.

https://ninjatrader.com/support/helpGuides/nt8/en-us/release_notes.htm