Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Memory Leak in Strategy Analyzer

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    Hello everyone,

    There seems to be a bit of confusion for what is happening here.

    Yes optimizations use a significant amount of memory and yes, the memory is not decommitted right away.

    NinjaTrader 7 had a way of forcing garbage collection and NinjaTrader 8 just lets .NET handle it.

    Peter, when you wrote in over email, you gave a us a video showing your test with the SampleMACrossover strategy where it used a large amount of tick data to build 200 volume bars. That will have a noticeable impact on memory which we do see in your video. I performed this test, and then I continued testing to see if the memory was in fact not getting decommitted. After completing your test, I downloaded the same amount of data for ES 06-20 and before I ran the same optimization backtest on that data, the memory was decommitted. This tells me that .NET garbage collection is working and there is not a memory leak.

    Something else to consider with optimization backtests: By default NinjaTrader will create new instances of the strategy for each optimization iteration. A strategy can be programmed to use IsInstantiatedOnEachOptimizationIteration=false (class level variables will need to be reset in State.DataLoaded) and the optimizer will not create new instances of the strategy for each optimization iteration. The additional optimization iterations will increase the amount of memory utilized, and will increase the the time needed to complete the backtest.

    AI Generate strategies do not generate code that is compliant with IsInstantiatedOnEachOptimizationIteration=false, so AI Generate runs will carry a large processing time in nature.

    SampleMACrossover utilizes IsInstantiatedOnEachOptimizationIteration=false, but I thought this would be worth mentioning for others running optimization backtests.

    IsInstantiatedOnEachOptimizationIteration - https://ninjatrader.com/support/help...niteration.htm

    Peter, if there is a test similar to the video provided where we can observe that memory is not decommitting after a period of time, we can look into that.

    Otherwise, what you would be looking for is to have NinjaTrader implement a way to force garbage collection. We do have a feature request tracking interest behind this with the ticket ID SFT-2111. Let me know if you would like us to add a vote on your behalf.

    I look forward to assisting.
    Last edited by NinjaTrader_Jim; 04-20-2020, 11:24 AM.
    JimNinjaTrader Customer Service

    Comment


      #17
      Ok, yes please add my vote for the ticket ID SFT-2111 to have NinjaTrader implement a way to force garbage collection.

      Thank you for looking into this and explaining it.

      Anyone else who wants this feature please add your vote to this thread.

      Comment


        #18
        Click image for larger version

Name:	Resource exhaustion.PNG
Views:	188
Size:	1.07 MB
ID:	1102158
        I am getting resource exhaustion and ninja hang while doing optimization in Strategy Analyzer still using volume bars.

        This issue has not been resolved!

        I have upgraded to 128 GB DDR4 RAM.

        Ninja still managed to use all of the memory plus exhausted the virtual memory.

        I get a resource exhaustion error. There is also an error with NET UltraView.

        I have included the error message of NET Framework Ultraview.
        There are others of ninja application hang and resource exhaustion error which I will attach in next support emails.

        We have already gone through all of your safe mode steps before and support has verified this is an issue in the previous post. Your response previously was for me to vote for a feature request to force garbage collection, which I did. I think that is a ridiculous response to an issue that you acknowledge.

        How will voting for a feature fix the functional problem? Seriously.





        Attached Files

        Comment


          #19
          I understand this is a garbage collection issue. Here are some specific inputs that seem to affect the resource exhaustion due to limitations of the garbage collection within NET Framework:

          1. Using volume bars
          2. Multiple runs. Exhaustion may occur on the 3rd or 4th run. Do I need to close the strategy analyzer after each run or close the workspace as one user suggests after each run?
          3. Optimizing data series

          For other user's sake how does one effectively work around the limitations of the NET Framework's garbage collection limitations?

          Comment


            #20
            Hello peterslane1,

            You wrote in over email and brought up an issue where resources are not freed when using Optimize On Data Series and using Volume bars. Our Quality Assurance team is investigating that further.

            We will continue to work with you over email and will post any necessary updates here.
            JimNinjaTrader Customer Service

            Comment

            Latest Posts

            Collapse

            Topics Statistics Last Post
            Started by AlessioCianini12, Today, 01:38 AM
            0 responses
            23 views
            0 likes
            Last Post AlessioCianini12  
            Started by nandhumca, Yesterday, 11:54 PM
            0 responses
            26 views
            0 likes
            Last Post nandhumca  
            Started by nandhumca, Yesterday, 11:51 PM
            0 responses
            25 views
            0 likes
            Last Post nandhumca  
            Started by ezrollin, Yesterday, 11:48 PM
            2 responses
            25 views
            0 likes
            Last Post ezrollin  
            Started by jjanguda, Yesterday, 10:15 PM
            1 response
            25 views
            0 likes
            Last Post NinjaTrader_ChrisSch  
            Working...
            X