Strategy Analyzer variables not correct after 1st optimization run and save

[PN720]

Hey fellas -

I wanted to report a bug I've run into several times while running Optimizations. After the 1st Optimization is run and you switch to Standard to run a different set of variables, your inputs are ignored as the variables reset to the first Optimization result.

You can replicate this using the Sample MA Crossover:
1. Optimization
2. Set a range for Fast and Slow EMAs. I used 5;30;5 and 10;60;5 with a date range of 01/01/2018-01/31/2018.
3. Tick Range: 100-500 Tick
4. Run.
5. After it completes its run, notice the optimum values that it shows in parenthesis.
6. Switch to Standard.
7. Plug in a different set of values other than the optimum values.
8. Template | Save. Choose the Default, if it's already there and overwrite.
9. Run.
10. Notice the input parameters reset to the optimum values and not the ones you put in. The report results are for the optimum values, not the ones you put in.
11. Now try your revised input values again and Run.
12. It sticks and successfully runs the Report.

[NinjaTrader_ChelseaB]

Hello PN720,

Thank you for reporting the behavior.

I will test and if I am able to reproduce I will forward a report to our development.

I appreciate your patience.

[NinjaTrader_ChelseaB]

Hello PN720,

Are you able to reproduce this behavior with less optimization iterations (meaning less range between inputs)?

Is the Tick range from 100 to 500 absolutely necessary to reproduce this behavior?

If you test with the Fast from 10;20;1, the Slow 25;25;1, and the data series not optimized over (10 iterations) can the behavior not be reproduced?

I've tested with the parameters you have mentioned but I was not able to reproduce.

https://drive.google.com/file/d/1k9QdV7FaqXf_KaWJUv1uWuGhAWhJzzxk/view?usp=drivesdk

What version of NinjaTrader 8 are you using? (Help > About)

[PN720]

Hi Chelsea,

Sorry, I missed adding in a Step 100 to the 100-500 Ticks. I don't think it matters but what I noticed in your video is that you didn't save a Template (step 8). Here's a video showing you what I'm seeing.

https://youtu.be/DQZWYQhsNvk

[NinjaTrader_ChelseaB]

Hello PN720,

(edited to say "saving")
If you are saving a template named Default, this will automatically be loaded.

Are you able to reproduce this behavior if you name the template something other than Default so it is not automatically loaded?

[PN720]

Hi, yes, I tried 'Default1' on the Save and it did the same thing. This is the Save operation, not the Load.

[NinjaTrader_ChelseaB]

Hello PN720,

I was not able to confirm, if you test with the Fast from 10;20;1, the Slow 25;25;1, and the data series not optimized over (10 iterations) can the behavior not be reproduced?

[PN720]

Hi again,

Yes, I can duplicate it using your settings. You can see it reset to the Optimized values (20,25) right after my Save (5,10).

https://youtu.be/Xt4GXRpgBos

[NinjaTrader_ChelseaB]

Hello PN720,

Thank you for confirming you are able to reproduce the behavior with a much smaller range (allowing for a must faster reproduction of the issue).

I was able to reproduce and I have reported the behavior to our development.

Once I have a tracking ID for this behavior I will forward this to you for future reference.

[PN720]

Hi, I'm just following up.

Did this get assigned a Tracking ID so it can be fixed?

[NinjaTrader_ChelseaB]

Hello PN720,

Thanks for checking in.

This does not have an NTEIGHT ID yet. Its still under review.

Once I have an ID I will post in this thread so you can watch the release notes for the ID to be fixed.

[NinjaTrader_ChelseaB]

Hello PN720,

I have received tracking ID #NTEIGHT-13137 for this behavior where saving a backtest template after an optimization causes standard backtest to change and use optimization parameters.

As new releases of NinjaTrader 8 become available please check the release notes for this ID. Once the ID is listed, please update and then test for the behavior to confirm it was corrected.

Below is a public link to the release notes.

https://ninjatrader.com/support/helpGuides/nt8/en-us/release_notes.htm