Odd Strategy Analyzer Behavior - Analysis != Summary

[wadams]

Hi NT,

I'm getting some strange behavior from a strategy I'm trying optimize. After running an optimization, the results in the top "Analysis" pane differ from the results that I see in the bottom "Summary" pane.

In the attached screenshot, the result selected in the "Analysis" pane shows a Profit Factor of 2.70 with 97 trades. However, in the "Summary" pane, I see a profit factor of 0.89 with 111 trades.

Has anyone seen behavior like this before? And, what else would you need from me?

I did try running the SampleMaCrossOver strategy and the results went back to matching. My first guess is that this strategy has some issues. So, I've attached the code for that as well (it may use some indicators that aren't in the standard NT distribution).

Thanks,
Wayne

[NinjaTrader_ChrisL]

Hello wadams,

Thanks for the post.

I ran your script over 1-minute bars and my optimization results are correct. This could be coming from the use of Renko bars in the Strategy Analyzer, which is not a recommended action to take. Do you see correct results when you run the backtest on 1-minute data of the same time frame?

I look forward to your reply.

[wadams]

Hello ChrisL,

Thanks for the tip!

I did try running with 1-minute data, and the problem persisted .

However, when I restarted NinjaTrader, the issue went away. Then, after 10 - 15 optimization runs, the issue came back. When I restarted NinjaTrader, then the issue cleared up again.

So, my guess is that something funky is going on in memory. While I'm not blocked on my work, this is concerning behavior. Can we open a support ticket?

Also, I'm curious about the guidance against using Renko Bars in the Strategy Analyzer. Is it because of the issues similar to what was brought up here: https://ninjatrader.com/support/foru...&postcount=31? Or are there other issues I should be aware of? And, are these issue for other non-time based charts, like Range Bars?

[NinjaTrader_ChrisL]

wadams,

Thanks for the reply.

If you would like to open a ticket about this, please write into platformsupport[at]ninjatrader[dot]com and paste this forum link in the body of the email. Please attach the newest version of the script you are using if there have been any changes made to it. If there were no changes, I can use the script that you posted originally. I will let you know if I can reproduce the issue you are seeing.

To answer your question, Renko bars on real-time data re-draw the open based on the direction of the bar. This can not be simulated historically. That is why Renko bars are unreliable in a backtest. If you would like to know more about this I invite you to review the code of the standard Renko bar (or the code of your custom Renko bar if it is available to you).

I look forward to assisting further.