[CRITICAL BUG] Strategy Analyzer Optimization - totally random behavior

[login_dejavu]

Hello guys,

I'm attaching simple strategy code and screenshots to articulate the problem. Basically after optimization finished and I'm clicking in each result, but the summary below never reflects what I'm selecting, and actually showing something random which is not even available in the list of results. Please take look, and let me know if you need any other details which can help you to investigate the problem, I will be happy to assist.

[NinjaTrader_RyanS]

Hello,
Thanks for the inquiry.

If you run an optimization using the built-in 'sample-ma-crossover' strategy does the same behavior occur?

I look forward to your reply.

[login_dejavu]

Hello Ryan, with Sample-MA-Crossover it is working as expected. But as you see in simple code I've attached it is very easy to reproduce. Hope it helps.

[NinjaTrader_RyanS]

Hello,

This likely indicates something with the strategy itself rather than the functionality of the strategy analyzer. Can you provide the .cs and .dll file as a zip so I may load this strategy on my platform to try and test?

Thanks in advance-

[NinjaTrader_Jim]

Hello login_dejavu,

This is going to be an expected behavior. NinjaTrader performs a "details run" which processes OnBarUpdate for the specified parameters in the optimization iteration to generate the trade performance results. If you add a print in OnBarUpdate, you will see this happen each time you switch viewing optimization iterations. Since you are getting a new random number in OnBarUpdate, each run will have different random values than what the strategy used to generate the results initially.

Because of the above, it would not be advised to use random numbers in the strategy while maintaining the expectation that the value would be consistent between each "details run" of OnBarUpdate.

If you have any additional questions, please let me know.

[login_dejavu]

Hi Jim, but wait, optimization happens on Input parameter, in my case it is "Iterator". Random number is used to generate entry signal. My expectation to see in summary best results selected based on "Optimized On/OptimizationFitnesses" function. I would understand your explanation if I would use random number to generate "Iterator" property itself, but this is no the case here.
Could you clarify please.

[NinjaTrader_Jim]

Hello login_dejavu,

When the strategy runs the first time, best optimization iterations are listed from the backtest. Whenever you switch to view a new optimization iteration, OnBarUpdate runs again for the details run. Since you are generating random numbers in OnBarUpdate, the details run will be generating different trades than when the backtest was first run. NinjaTrader is expecting the strategy to use parameters set from the optimizer for consistency between backtests and details runs. If the logic used Iterator instead of random.Next, we would see consistent results.

I have made a demo to help clarify.

Demo - https://drive.google.com/file/d/11zK...w?usp=drivesdk

Please let me know if there is anything else we can do to help.

[login_dejavu]

Uhhh, now it make sense. I always believed that when I see list of results and I click through them only UI binding changed to render specific one because I was expecting we already have all the required data to display, I did not know that when I click it kind of re-run it again.

Thanks for the video and your effort! And thanks for the best retail platform available on the market!