Strategy Optimizer and analyser result doesn't match

[dastaan]

Hi NT,

I need serious help with understanding the results. Please find the attached image for reference.

As stated, The results isn't matching up in optimisation. Can't seem to understand why NT is calculating result twice ( As they both are different ).

Please find the codebase attached.

( Instrument MES-12 , TF : 5 Min )

PS : The IsInstantiatedOnEachOptimizationIteration is set to True(Default)

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your post.

First, what version of the platform are you running? You can find this under Help > About. The current version is 8.0.23.1. If you are on any other version, please update:

• First, copy your license key from NinjaTrader under Help> License Key then exit NinjaTrader
• Click on the link: https://ninjatrader.com/PlatformDirect
• Enter your license key and press Submit
• Select 'NinjaTrader 8'
• Select 'Download'
• Critical: Before running the installer, ensure NinjaTrader is closed.

Once completed, please rerun a fresh new optimization. Do you see the results matching summary details?

If not, please run an optimization using the built in Sample MA Crossover. Do the results match between the summary and what's listed when you click on it?

A few more questions as well. Does the strategy in question:

1. use static variables?
2. use a timer?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Yes, I am on the current version.
There are no static variable and there isn't any use of local timer ( We are using candle time though ).

I have uploaded my codebase and you can test it

[patricia70]

@dastaan: sorry, cannot help but: what kind of strategy should that be? A profit factor of 2.7-3.0 is very unlikely and maybe this is an indication that's something faulty in general with the strategy ?

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

You didn't state whether or not you have tested with the Sample MA Crossover. Can you confirm if running an optimization using that built in strategy returns results as you'd expect?

Further, the sample you previously had on your first post doesn't match the inputs in your screenshot, which means you're using a different version of the Strategy than you've provided. We would want to test the current version of the code. Please attach the version you're using in your reply.

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,
( First, Sorry for late reply. )

The Sample MA Crossover worked perfectly fine to me. Also, I noticed when I set the setting variable "fixed lot size" to a positive integer ( Which disables dynamic position sizing based on portfolio value ) it works completely fine. (Maybe this information can help)

Please find the requested files attached. ( The strategy name is RevMasterV5 )

[NinjaTrader_Kate]

Hi dastaan,

Thank you for your reply.

The provided strategy in your previous post doesn't match the screenshots in your first post either, nor does it contain a Fixed Lot Size parameter, though I do see an Initial Lot Size parameter. Could you provide a screenshot of the correct settings to try to reproduce?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Thank you for pointing that out. The current script highly resembles to the initial script.

Fixed Lot Size display name can be found in properties ( variable name FIXLOTSIZE ).

I have prepared a small optimisation case for you to quickly replicate the scenario. Please set the ATR multiplier to 1.5 to 2.0 with an increment of 0.1 step.

[ For reference please check the screenshot ]

PS : Tested on MES-12-20

Looking forward to your valuable feedback.

[NinjaTrader_Kate]

Hello daastan,

Thank you for your reply.

This is what I see when attempting to optimize the strategy:



I'm not getting any results whatsoever when setting the ATR Multiplier to 1.5; 2; 0.1 (I did set that after taking the screenshot).

Can you provide the same version you are working with along with screenshots of all settings so that I may reproduce successfully?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Good to hear back from you.

About the strategy optimization, I have few suggestions for you :

1) I am under impression that you are on previous version . You can remove all other versions and install the DebugNinja addon which you can find above. Also please note, we are on RevRangerV5 not V5F.

2) Please set the Timeframe to 5 minute ( Like described in the first post )


I am uploading the settings screenshot below.




Looking forward to your valuable insights !!!

[NinjaTrader_Kate]

Hello daastan,

Thank you for your reply.

I was able to replicate this behavior and I note that in the strategy you've accounted for the fact that there's not a specified Account that you can access values for in the Strategy Analyzer, but you're still referring to the Account class elsewhere within the code. If you remove all references to Account values and instead specify those values as hard coded values, do you get the same result?

Thanks in advance; I look forward to assisting you further.

[dastaan]

Hi Kate,

Thank you for your comment. I tried the same and found the issue.

The bool "inStrategyAnalzer" is giving "false" value which is surprising to me.

I'd like your view on it.

How I can differentiate between ( Analyzer (backtest, optimisation, ...), playback and live account) in a strategy ?

Looking forward to hear back from you soon.

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

In my testing IsInStrategyAnalyzer by printing the value right before it's checked in State.DataLoaded, it returns True. I don't think that part is where the issue is. You're also using Account.CancelAllOrders(Instrument); and in GetInitialLots you are referencing Account.Get(AccountItem.CashValue, Currency.UsDollar); and I suspect that's what's causing the issue. I would see what happens if you use CancelOrder() to explicitly cancel each order and have GetInitialLots return a hardcoded value in a backtest rather than trying to access the account there.

Please let us know if we may be of further assistance to you.

[dastaan]

Hi Kate,

Thank you for your reply. This is quite strange that it is returning True.

Unfortunately the "IsInStrategyAnalyser" isn't working as expected.

Please find screenshot attached below.



It shouldn't print the statement but as you can see it is.

( I am on latest version of NT so I am not sure if it is creating the issue )

[NinjaTrader_Kate]

Hello dastaan,

Thank you for your reply.

I realized that I was only running it as a backtest, and not as an optimization (mainly so I didn't have to wade through so many prints). I was able to reproduce this when running an optimization on the current version. I believe this should not be the case and will be reporting it to our development team to see what they say.

In the meantime, I would recommend using your own user input bool that can be marked as true when optimizing a strategy and use that in place of your use of IsInStrategyAnalyzer, at least temporarily. If you do that, do you see the results match as expected?

Thanks in advance; I look forward to assisting you further.