Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Memory bug when WFA optimizing

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Memory bug when WFA optimizing

    Hi there, I recently noticed that there may be an issue with memory allocation when running back testing. I noticed that the memory utilisation of my windows VPS increases steadily until the whole application crashes and no memory is left .

    I’ve tried this on different machines and always observe the same outcome.

    One of the main reasons I purchased a license was to be able to do complex backtesting. How can I troubleshoot this?

    Thanks!


    #2
    Hi Oracletrades,
    I'm more than happy to be corrected here, but I'm afraid Optimization can use UNLIMITED memory and crash. It might (still) be possible to limit the number of iterations running in parallel using the (undocumented) WaitForIterationsCompleted() command. However, individual users should not have to deal with "debugging" NT from my point of view. RAM Control is a must have, particularly for such software.
    NT-Roland

    Comment


      #3
      Hello Oracletrades,

      With many backtest iterations of an optimization this may require very large amounts of memory.

      If this is a script you have written, I recommend modifying the script to take advantage of IsInstantiatedOnEachOptimizationIteration being set to false.
      https://ninjatrader.com/support/help...niteration.htm

      Objects should only be initialized in State.DataLoaded when absolutely necessary, objects that can be reset like arrays should be reset instead of new objects re-initialized.

      If this is not a script you have made, I would recommend contacting the vendor or author and requesting these changes.
      Chelsea B.NinjaTrader Customer Service

      Comment


        #4
        Thank you Chelsea, Roland. I wasn't aware of that script and assumed that NT would do that automatically. I have 24 GB of RAM on the machine and was running 100.000 iterations which is quite common when you have 4-5 variables... Will try what you suggest above and revert.

        Comment


          #5
          Hi everyone, just a follow-up to this post. Quite frankly, the proposed solution is too complicated for me to implement and understand. Therefore, I did the following (in case anybody ever has the same issue):

          - increased the Windows Paging file to 16GB
          - as I am on a SSD, the read/write speeds are sufficient for the backtesting (it slows down a bit, but time is not my issue)

          Cheers!

          Comment


            #6
            SOLUTION: to anybody coming across this post, please just remove any "drawing" objects like arrows from the strategy. The memory leakage will stop and your strategy will run, even with 1.000.000 iterations (just doing this on a 16GB RAM machine).

            Took me a few weeks. Support, please take note of this solution, it was mentioned in a few other posts.

            FEATURE REQUEST: for future releases, can you consider "disabling" the drawing of objects in the strategy analyzer when doing optimizations, entirely? There is no value anyway during the optimization process and it solves the memory hog problem without the need of coding anything.

            Comment


              #7
              Hello Oracletrades, thanks for your reply .

              You can use IsInStrategyAnalyzer to ignore drawing draw objects while backtesting/optimizing:

              https://ninjatrader.com/support/help...gyanalyzer.htm
              Chris L.NinjaTrader Customer Service

              Comment


                #8
                Originally posted by Oracletrades View Post
                SOLUTION: to anybody coming across this post, please just remove any "drawing" objects like arrows from the strategy. The memory leakage will stop and your strategy will run, even with 1.000.000 iterations (just doing this on a 16GB RAM machine).
                Took me a few weeks. Support, please take note of this solution, it was mentioned in a few other posts.
                FEATURE REQUEST: for future releases, can you consider "disabling" the drawing of objects in the strategy analyzer when doing optimizations, entirely? There is no value anyway during the optimization process and it solves the memory hog problem without the need of coding anything. https://givology.org/~GabrielBlocker/blog/326896/
                Hi! Thanks for the suggestion: I've managed to stop memory leakage by removing "drawing" objects. But is it some kind of bug? I didn't find it on a list.

                Comment


                  #9
                  Hello mcrobertg7,

                  Drawing object method calls all create new instances of the drawing object which all need need memory locations whether or not they will be drawn.
                  This is not a leak. This is the number of objects being created by the strategy that is calling the methods for these.
                  To avoid the drawing objects from being created, do not call the draw method during historical data or specifically when in the strategy analyzer.

                  Any objects created by a strategy will need memory locations.

                  This also involves using IsInstantiatedOnEachOptimizationIteration set to false where the variables should be initialized only when null in State.DataLoaded and reset when possible to save memory.

                  Below is a link to the help guide.
                  https://ninjatrader.com/support/help...niteration.htm

                  Below is a link to a thread where this is mentioned.
                  https://ninjatrader.com/support/foru...036#post838036
                  Last edited by NinjaTrader_ChelseaB; 11-14-2019, 11:42 AM.
                  Chelsea B.NinjaTrader Customer Service

                  Comment

                  Latest Posts

                  Collapse

                  Topics Statistics Last Post
                  Started by Oracletrades, Today, 09:10 PM
                  0 responses
                  3 views
                  0 likes
                  Last Post Oracletrades  
                  Started by SteveAU, Today, 08:35 PM
                  0 responses
                  8 views
                  0 likes
                  Last Post SteveAU
                  by SteveAU
                   
                  Started by Lloyd143, Today, 05:24 PM
                  1 response
                  21 views
                  0 likes
                  Last Post NinjaTrader_ShawnB  
                  Started by Hagar, Today, 04:38 PM
                  1 response
                  16 views
                  0 likes
                  Last Post NinjaTrader_ShawnB  
                  Started by [email protected], Today, 04:49 PM
                  0 responses
                  13 views
                  0 likes
                  Last Post bpatel20@gmail.com  
                  Working...
                  X