Strategy Optimizer and analyser result doesn't match

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your patience.

I was able to narrow things down a bit and have found a specific date and ATR setting that will cause a chart to freeze. Running the strategy on a 5 minute chart for March 13th 2020 with the ATR period set to 4 appears to cause the chart to freeze. This is just loading a regular chart, you don't have to actually run an optimization over that date for the effect to be seen. You should be able to reproduce this by opening a MES 03-21 chart, then in the data series window, setting the end date to 3/13/2020. Once the chart loads, apply the strategy with the ATR period of 4. It spends quite a bit of time attempting to calculate, and the whole thing will freeze up.

Do you see the same when running that strategy on a chart of the same date?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Thank you for your suggestion.

Yes, I make sure that all open orders are closed when we don't have any active position. This should be ok as it is working fine for lower period.
I doubt if the cancel order is a problem here as even if I reduce the ATR Opt. period to say 1-10, It'll stuck at 9th optimization.

Looking forward to hearing back from you soon.

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

First, you'd want to be using the MES 3-21 contract for that date range, since the MES 12-20 settled on 12/18/20.

Next, I was able to reproduce this and it appears it's getting stuck when trying to cancel all the orders in the backtest, which indicates a stuck order is occurring. If you check your Log tab of the Control Center, you'll likely see a lot of prints regarding cancelling backtest orders, is this correct?

I'm looking further into exactly what may be occurring and will let you know when I have more information.

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Thank you for you reply.
It works fine for a month or two but when calculating on slightly higher period it always fails. Please find the screenshots attached for reference.


Looking forward to your views on it.

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

I am unable to reproduce this error on my end over about a month's worth of data - what instrument and time frame are you currently testing on?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

We can find the same issue with the previous one but nevertheless please find the current version attached.

Looking forward to your valuable insights.

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

Could you supply the current version so we may test on our end?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate and Jim,

Thank you for the suggestion above. It worked for me.

Now I am facing another issue with the same script. When I set the ATR optimisation from 1 to 14, It is freezing on the last iteration ( 14 ). I found out even I reduce the range, It keeps freezing.

What could be the issue ?

[NinjaTrader_Jim]

Hello dastaan,

You can check Account.Name to see if the strategy is applied to the Strategy Analyzer, Playback, a Simulation account, or a live account.

The IsInStrategyAnalyzer Help Guide documentation mentions this property is used for Strategy Analyzer charts. There is no Strategy Analyzer chart associated with optimization iterations except for the final run that would get displayed. We are confirming the language with the development team since the name of the property may sound misleading.

IsInStrategyAnalyzer - https://ninjatrader.com/support/help...egyanalyer.htm

If you want to differentiate optimizations and backtests, please see the undocumented detail posted here - https://ninjatrader.com/support/foru...ion#post696223

We will follow up after the Development team has confirmed the Help Guide language.

[dastaan]

Hi Kate,

As I am finding it hard to differentiate the environment ( If the strategy is in Analyzer, Playback or live ) Can you suggest a way so I can do so ?

My goal is to build a dynamic strategy so the knowledge of environment is a must.

Also how long it will take NT to resolve the bug ( isInStrategyAnalyzer ) ?

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

We do not have any kind of bug bounty program, no.

Please let us know if we may be of further assistance to you.

[dastaan]

Hi Kate,

Thank you for your reply.

Glad to hear that. Thanks for the suggestion, I'll do as suggested.

By the way, Is there bug bounty program active in NT ?

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

I realized that I was only running it as a backtest, and not as an optimization (mainly so I didn't have to wade through so many prints). I was able to reproduce this when running an optimization on the current version. I believe this should not be the case and will be reporting it to our development team to see what they say.

In the meantime, I would recommend using your own user input bool that can be marked as true when optimizing a strategy and use that in place of your use of IsInStrategyAnalyzer, at least temporarily. If you do that, do you see the results match as expected?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Thank you for your reply. This is quite strange that it is returning True.

Unfortunately the "IsInStrategyAnalyser" isn't working as expected.

Please find screenshot attached below.



It shouldn't print the statement but as you can see it is.

( I am on latest version of NT so I am not sure if it is creating the issue )

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

In my testing IsInStrategyAnalyzer by printing the value right before it's checked in State.DataLoaded, it returns True. I don't think that part is where the issue is. You're also using Account.CancelAllOrders(Instrument); and in GetInitialLots you are referencing Account.Get(AccountItem.CashValue, Currency.UsDollar); and I suspect that's what's causing the issue. I would see what happens if you use CancelOrder() to explicitly cancel each order and have GetInitialLots return a hardcoded value in a backtest rather than trying to access the account there.

Please let us know if we may be of further assistance to you.