Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Optimization of strategy with multiple instruments runs unnecessary iterations

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Optimization of strategy with multiple instruments runs unnecessary iterations

    Dear support,

    I am developing a strategy that works in a nutshell as follows:

    1. I work with all instruments from S&P500
    2. On each bar, I check each instrument from S&P500 and see if it matches enter condition e.g. ROC(x) above something
    3. Then I select the one that has the highest potential and enter

    I tried to optimize the strategy using the "Strategy Analyzer", but it seems like the strategy runs extra optimizations for each instrument in S&P500 while I want to run optimization only for each value of the property e.g. it can be the period for ROC.

    I have put together this sample that demonstrates the problem:
    0. Using Ninjatrader 8. 64-bit 8.0.15.1 with Kinetic EOD data
    1. There is simple strategy (attached) that has one property that should be optimized called "OptimizationProperty"
    2. I run the strategy in "Strategy Analyzer" with following settings.

    Click image for larger version

Name:	settings.png
Views:	50
Size:	48.8 KB
ID:	1042660

    - My expectation is that 2 optimizations would be run for values 1 and 2.
    - However, when you execute it, it will run 500+ executions which is far more than expected (see log.txt attached). What is the reason and how can I stop it and make Ninja to execute only 2?

    With Regards
    Martin

    #2
    Hello svitaz,

    I wanted to let you know I'm currently reviewing with our development the expected behavior with strategy instances.
    I appreciate your patience.
    Chelsea B.NinjaTrader Customer Service

    Comment


      #3
      Thanks. Have you figured out something? The optimization does not behave as expected. It seems that instead of 2 iterations it runs iteration for each instrument which is not what I want.

      Comment


        #4
        Hello svitaz,

        Our development has let me know:

        NinjaTrader will utilize the Processor Count for your CPU to distribute the workload of an optimization. The iterations are not created solely based on parameters and are created in instances of parameter cloning and passing among other scenarios.

        While we are not able to provide the internal workings of instance creation, our development has let us know that this behavior is expected.
        Chelsea B.NinjaTrader Customer Service

        Comment


          #5
          I am sorry, but how can this be expected behavior? If I run the optimizations on 500 stocks with 10 parameters, it will result in 5000 optimizations instead of 10. There is obviously a high-performance impact, because if we assume 5 threads and 1 optimization running 1 minute, then instead of (10/5*1) = 2 minutes the optimization will run ( (500 * 10) / 5 * 1) = 1000 minutes. The other thing is that only 10 optimizations will be useful, the other 4990 optimizations will just repeat what the other 10 has done. Not talking about the fact that instead of dealing with 10 results, you have 5000.

          Comment


            #6
            Strategy instances get created for copying data around, reusing the strategy between optimizations, and in additional multiple instances also get created at the start of an optimization for pooling which allows us to re-use a strategy instance for repeat use, As we try to re-use strategy instances instead of being messy with the memory as allocating memory for each new strategy for each backtest in an optimization and leaving that strategy for the garbage collector to clean it up will cause the optimizer to run much slower in our testing. Strategy pooling is one of the reasons we saw a significant performance improvement in NinjaTrader 8 when we optimized the performance of the strategy optimizer. I suspect your simply noticing the strategy pooling mechanic. Its nothing to worry as those strategy instances are not impacting the runtime performance of the optimization (there not duplicate working but instead just waiting ready for work). Right now they don't do anything as your optimization is rather limited in scope but they're there for when you do something heavier. As in your prints you can see you see the "Processing counter" the print the expected number of times which proves there is no duplicate backtesting going on. If there would be that, of course, would be a bug and not expected.

            Let me know if any questions.
            Last edited by NinjaTrader_Brett; 01-17-2019, 03:11 PM.
            BrettNinjaTrader Product Management

            Comment

            Latest Posts

            Collapse

            Topics Statistics Last Post
            Started by Oracletrades, Today, 09:10 PM
            0 responses
            3 views
            0 likes
            Last Post Oracletrades  
            Started by SteveAU, Today, 08:35 PM
            0 responses
            7 views
            0 likes
            Last Post SteveAU
            by SteveAU
             
            Started by Lloyd143, Today, 05:24 PM
            1 response
            20 views
            0 likes
            Last Post NinjaTrader_ShawnB  
            Started by Hagar, Today, 04:38 PM
            1 response
            14 views
            0 likes
            Last Post NinjaTrader_ShawnB  
            Started by [email protected], Today, 04:49 PM
            0 responses
            12 views
            0 likes
            Last Post bpatel20@gmail.com  
            Working...
            X