Memory not released after Strategy Analyzer end of run.

[xTrader1]

Hello,

Because I'm working on a big amount of a tick data, I'm monitoring a memory usage, using Process Explorer from SysInternals. I payed attention that a memory used be Strategy Analyzer is not released on end of run. It is not released when closing Strategy Analyzer Window. When running next instance of Strategy Analyzer, memory usage increases until thrashing occurs.
It's possible that upon a "normal usage" this issue does not show itself. However my strategy uses >4GB memory for 2 years of a tick data, therefore the issue appeared.
If needed I can capture a screen on end of run.
NT 64 bit 8.0.4.0

[NinjaTrader_JustinS]

Hello,

Thank you for your post.

I attempted to replicate the situation you described but have been unable to do so.

As you mentioned a screenshot we would like to compare the memory usage to events on your log and trace files.

Please follow the steps below to manually attach your log and trace files to your response so I may investigate this matter further.

Open your NinjaTrader folder under Documents.
Right click on the 'log' and 'trace' folders and select Send To> Compressed (zipped) Folder.
Send the 2 compressed folders as attachments to [email protected]
Once complete, you can delete these compressed folders.

Additionally, please include a screenshot of Process Explorer so we may have a base to run with.

Please reference the following ticket number in the body of the email: 1672614

[xTrader1]

Hello Justin,

I'm bringing your post and my replay to the forum, because it's possible that our fellow traders/programmers could advice either.
You wrote to me:

We are performing multiple tests to replicate this and have so far been able to replicate using our basic strategies.
I will be getting in touch with our Project Management group to bring this to their attention and find out what can be done or if we have a work around for resolving this.
In the meantime please feel free to update me with additional information.
I will reach out as soon as we have more details.
Thanks in advance for your patience.

I think you have to do something with it, as it's a huge memory leak, and even with a less memory consuming usage than mine, eventually all physical memory will be filled.
Meantime I would advice to look for a workaround , trying to constrain Strategy Analyzer object to perform a garbage collection.


Another guess would be, that Strategy Analyzer object should be destroyed after a run , but you're not doing it to avoid reloading a data in case the strategy will be run analyzed again, using different parameters, either manually or during automatic optimization.

If this is a case, I would advice to create one more property in Strategy Analyzer tab , something like "destroy Strategy Analyzer object after this run" with an explanation that will tell the user that in this case a data will be reloaded during next run, which will be time and resource consuming.

FYI.

[NinjaTrader_JustinS]

Hello,

Thank you for your post.

After some extensive testing and research on our side we were able to determine that this is not being caused by the platform but with the system's garbage collector.

The garbage collector on your PC does its own thing in its own time (NT does not control when memory is collected by garbage collector)
Memory would rise but would also be released as its needed by the system.
The garbage collector would take over when one of the following is true:
-The system has low physical memory.
-The memory that is used by allocated objects on the managed heap surpasses an acceptable threshold. This threshold is continuously adjusted as the process runs.

When running the the strategy analyzer have you encountered any crashes?

[xTrader1]

Thanks, Justin.

No crashes, but thrashing it means, that one certain physical RAM threshold is achieved, the OS (Windows, not .NET) performs more and more page faults in order to reduce a working set of a process, , and the system is effectively frozen.

I performed my checks either: When changing an instrument and parameters, at certain moment RAM usage fails, i.e. Garbage Collection is done.

It was in 8.0.4.0 . I installed 8.0.5.0 , seems that GC is working, when it's needed..

Thanks

[NinjaTrader_ChrisL]

Hi Freedance, thanks for posting.

I am assuming you are seeing a similar memory issue when using the strategy analyzer. If so, we have the IsInstantiatedOnEachOptimizationIteration where you can specify to keep the same strategy object for each optimization run. Also, test on a shorter time frame to see if the issue is hardware-related, and also test the SampleMaCrossover strategy on the same bar type/data range to see if the strategy itself is being inefficient with objects.

Best regards,
-ChrisL