How To Release Memory

[coolmoss]

I have a strategy which is consuming memory every time I disable/enable. I'm not at all familiar with how to release memory consumed by variables and standard Ninja indicators. Any one have any examples of how this is to be done?

My first attempt at improving the situation was to add OnTermination() and include a line to RemoveDrawObjects(). I thought this might be the issue because my strategy draws dozens of fixed text objects. That didn't make any diff at all.

Thanks!

[Radical]

OnTermination() only gets called when you disable the strategy, so I don't think it would help in this situation. It's really hard to give you a recommendation in this situation without knowing what your code looks like, and what you mean by "I have a strategy which is consuming memory every time I disable/enable." The memory consumption goes UP when you disable the strategy?? By how much?

[NinjaTrader_Joydeep]

Hello coolmoss,
Thanks for your note.

The .net framework's garbage collector should clean up things. Are you witnessing the same scenario if you wait for some time.

Also please let me know are you appending data in the log tab using the Log() functions.

[coolmoss]

Joydeep,

I was talking with Matthew via email, and he suggested that I look into cleaning up resources. I've looked at all the forum posts I can find and it's clear a number of folks are/have experienced issues with excessive memory consumption. What happens is that memory consumption steadily rises each time I manually disable and re-enable my strategy (eg. I disable, change a parameter and then enable). Within 6 to 8 times of doing this I will experience a crash of Ninja due to out of memory.

I shared this with Matthew: the strategy uses only variables and built-in Ninja indicators. No manually created IOrder objects, no user arrays, no other C# objects. No logging. Yet, the memory just goes up and up and up. Nothing clever or sophisticated in the code is present. So, I'm simply looking for an example or reference on how to clean up variables and the Ninja indicator objects.

[sledge]

Coolmoss,

How much hard drive space do you have left, and how much memory does your machine have?

Joydeep,

I was talking with Matthew via email, and he suggested that I look into cleaning up resources. I've looked at all the forum posts I can find and it's clear a number of folks are/have experienced issues with excessive memory consumption. What happens is that memory consumption steadily rises each time I manually disable and re-enable my strategy (eg. I disable, change a parameter and then enable). Within 6 to 8 times of doing this I will experience a crash of Ninja due to out of memory.

I shared this with Matthew: the strategy uses only variables and built-in Ninja indicators. No manually created IOrder objects, no user arrays, no other C# objects. No logging. Yet, the memory just goes up and up and up. Nothing clever or sophisticated in the code is present. So, I'm simply looking for an example or reference on how to clean up variables and the Ninja indicator objects.

[coolmoss]

Sledge,

I'm experiencing the same problem on several different machines, including a brand new Win7 pc with 4 gig of ram and a huge hard drive. I literally get Ninja to shut down using a single chart with nothing else running on the computer. I can watch the Task Manager and can see the memory usage creep up at least 20meg every disable/re-enable. The strategy is about 800 lines long, but consists of nothing more than built-in Ninja indicators and primitives (bools, doubles, ints). That's it. I never realized this was happening until I started doing a lot of backtesting (from the chart) by disabling, changing a parameter, re-enabling... repeatedly. I don't know if this makes a big difference but I do have about 30 parameters, but again they're just bools and ints.

[koganam]

Sledge,

I'm experiencing the same problem on several different machines, including a brand new Win7 pc with 4 gig of ram and a huge hard drive. I literally get Ninja to shut down using a single chart with nothing else running on the computer. I can watch the Task Manager and can see the memory usage creep up at least 20meg every disable/re-enable. The strategy is about 800 lines long, but consists of nothing more than built-in Ninja indicators and primitives (bools, doubles, ints). That's it. I never realized this was happening until I started doing a lot of backtesting (from the chart) by disabling, changing a parameter, re-enabling... repeatedly. I don't know if this makes a big difference but I do have about 30 parameters, but again they're just bools and ints.

It still sounds like some kind of memory leak. Are you creating any objects at all? Whether using the "new" keyword, or just creating named instances of classes/indicators.

[coolmoss]

Koganam,

Thanks so much for your input.

To answer your question: absolutely no objects created at all. No use of "new" keyword, no classes, heck I don't even have any user created method/functions. As I mentioned earlier, this strategy is just a bunch of very simple condition sets with entries/exits via the built-in entry and exit methods. I never declare an IOrder object.

The strategy does plot a bunch of text objects, 3 letters per plot, and there are a few dozen of those that typically plot. I am removing those in OnTermination().

I do have two Add commands in Initialize() for a 1 tick series and a 1 minute series. Could this be causing the problem? If so, do you know how I would code to manually dispose of these Added data series?

[sledge]

Koganam,

Thanks so much for your input.

To answer your question: absolutely no objects created at all. No use of "new" keyword, no classes, heck I don't even have any user created method/functions. As I mentioned earlier, this strategy is just a bunch of very simple condition sets with entries/exits via the built-in entry and exit methods. I never declare an IOrder object.

The strategy does plot a bunch of text objects, 3 letters per plot, and there are a few dozen of those that typically plot. I am removing those in OnTermination().

I do have two Add commands in Initialize() for a 1 tick series and a 1 minute series. Could this be causing the problem? If so, do you know how I would code to manually dispose of these Added data series?

Talk about great timing!

Marcows just posted a "crash strategy". I am going to check it out.

http://www.ninjatrader.com/support/forum/showthread.php?p=305810

I suspect it would crash yours also.

[NinjaTrader_Joydeep]

Hello coolmoss,
How many days of data you are loading? One tick bars can be resource heavy. You can refer to this thread for an workaround

http://www.ninjatrader.com/support/forum/showthread.php?t=44797

[coolmoss]

Hi Joydeep,

I am experienced memory consumption with just 3 days of data loaded. The problem is that in real life I need to have 100 days loaded, as this is specifically for historical testing. I have a separate strategy file for running in real time that does not use 1 tick bars, and that works fine.

Please recall exactly what's happening: the memory is consumed when I disable/re-enable to, say, change a parameter. Can you please confirm that disabling/re-enabling would/could cause memory leakage because of the 1 tick data series being present?

If so, how can I clean up that resource in OnTermination() ?

If not, then I'm still at square one with this issue and need to find an answer. This is quickly becoming a deal killer for me using Ninjatrader. Thanks much for your continued help.

[NinjaTrader_Joydeep]

Hello coolmoss,
You have to isolate the scenario. If you remove the 1 tick bar and the draw object codes then is the strategy working fine?

Also if too many orders are submitted (like order on every bars etc) then you can witness a memory hog.

Also please do remember memory utilization is greater when you use the NinjaScript Editor when trading live with automated strategies. Its a good idea not to develop while you are running a NinjaScript strategy.

[coolmoss]

Joydeep,

To reiterate: this is not a problem in live or real time trading. There is no performance issue when I'm running the strategy. I've said this several times now but I have the feeling I'm not being heard: the memory leakage is occurring when I disable and re-enable. Each time I cycle the strategy on and off the memory consumption goes up about 10 to 20 meg.

I've commented out all the text plots, the entire sections where the tick series is added or processed, and I even commented out the entire blocks of code which place and manage orders. All that is left is several dozen lines of code setting bool variables to true if certain conditions are met. And, again, I'm not using ANY objects with the "new" keyword or any objects through an IInterface.

[NinjaTrader_Joydeep]

Hello coolmoss,
Thanks for the clarification.

To assist you further if you try enabling/disabling the SampleMACrossover strategy then can you replicate the same scenario.

[coolmoss]

Joydeep,

I enabled/disabled the SampleMACrossover about 15 to 20 times and Ninjatrader memory consumption increased about 6 meg of Ram. This is much slower leakage than my strategy, but it's definitely there. This is not specific to a single PC. I tried this on 3 different computers (XP and Win7) and they all do the same thing.