Optimization of strategy with multiple instruments runs unnecessary iterations

[svitaz]

Dear support,

I am developing a strategy that works in a nutshell as follows:

1. I work with all instruments from S&P500
2. On each bar, I check each instrument from S&P500 and see if it matches enter condition e.g. ROC(x) above something
3. Then I select the one that has the highest potential and enter

I tried to optimize the strategy using the "Strategy Analyzer", but it seems like the strategy runs extra optimizations for each instrument in S&P500 while I want to run optimization only for each value of the property e.g. it can be the period for ROC.

I have put together this sample that demonstrates the problem:
0. Using Ninjatrader 8. 64-bit 8.0.15.1 with Kinetic EOD data
1. There is simple strategy (attached) that has one property that should be optimized called "OptimizationProperty"
2. I run the strategy in "Strategy Analyzer" with following settings.



- My expectation is that 2 optimizations would be run for values 1 and 2.
- However, when you execute it, it will run 500+ executions which is far more than expected (see log.txt attached). What is the reason and how can I stop it and make Ninja to execute only 2?

With Regards
Martin

[NinjaTrader_ChelseaB]

Hello svitaz,

I wanted to let you know I'm currently reviewing with our development the expected behavior with strategy instances.
I appreciate your patience.

[svitaz]

Thanks. Have you figured out something? The optimization does not behave as expected. It seems that instead of 2 iterations it runs iteration for each instrument which is not what I want.

[NinjaTrader_ChelseaB]

Hello svitaz,

Our development has let me know:

NinjaTrader will utilize the Processor Count for your CPU to distribute the workload of an optimization. The iterations are not created solely based on parameters and are created in instances of parameter cloning and passing among other scenarios.

While we are not able to provide the internal workings of instance creation, our development has let us know that this behavior is expected.

[svitaz]

I am sorry, but how can this be expected behavior? If I run the optimizations on 500 stocks with 10 parameters, it will result in 5000 optimizations instead of 10. There is obviously a high-performance impact, because if we assume 5 threads and 1 optimization running 1 minute, then instead of (10/5*1) = 2 minutes the optimization will run ( (500 * 10) / 5 * 1) = 1000 minutes. The other thing is that only 10 optimizations will be useful, the other 4990 optimizations will just repeat what the other 10 has done. Not talking about the fact that instead of dealing with 10 results, you have 5000.

[NinjaTrader_Brett]

Strategy instances get created for copying data around, reusing the strategy between optimizations, and in additional multiple instances also get created at the start of an optimization for pooling which allows us to re-use a strategy instance for repeat use, As we try to re-use strategy instances instead of being messy with the memory as allocating memory for each new strategy for each backtest in an optimization and leaving that strategy for the garbage collector to clean it up will cause the optimizer to run much slower in our testing. Strategy pooling is one of the reasons we saw a significant performance improvement in NinjaTrader 8 when we optimized the performance of the strategy optimizer. I suspect your simply noticing the strategy pooling mechanic. Its nothing to worry as those strategy instances are not impacting the runtime performance of the optimization (there not duplicate working but instead just waiting ready for work). Right now they don't do anything as your optimization is rather limited in scope but they're there for when you do something heavier. As in your prints you can see you see the "Processing counter" the print the expected number of times which proves there is no duplicate backtesting going on. If there would be that, of course, would be a bug and not expected.

Let me know if any questions.