Urgent Memory Leak -- How to Reproduce

[wayneFH]

There's a really bad memory leak in NT 6.5.0.10. Have you already fixed this? Can I get a better release, please? ASAP would be great.

I attached a zip file with only the strategy code necessary to reproduce the memory leak.

How to reproduce: The onBarUpdate() does nothing whatsoever. However, the Initialize function simply adds a second data series and an indicator and gobbles memory like crazy!

        protected override void Initialize()
        {
            // This line creates a HUMONGOUS memory leak.
            // It adds on 30 or 40 Megabytes to memory every time
            // you modify the strategy. I tested it by moving the 
            // slider on Market Replay since. This happens when running
            // real time on the market. In a hour or so it eats up 
            // over 1 gig of memory till it kills the PC.
               Add(PeriodType.Second, 1); 
            // If you comment out the above line, this one also creates
            // a memory leak.  But this indicator by itself on the chart
            // never creates any memory leak. This leads me to believe
            // that the Add() function itself mismanages memory.
            Add(Previous10MinsOHLM());
        }

NOTE: The custom indicator runs by itself on the chart without ever causing a memory leak. But when added to the strategy, it creates a 1 meg leak every time you reload ninjascript on the chart.

Even worse, the Add() function that adds a second data source creates a 30 to 40 meg leak every time you reload the script.

Another way to test, when I run the Market Replay, every time you move the slider, it gobbles another 40 meg of memory.

Where I original observed this was during running with real time data. It also periodically gobbles another 30 or 40 meg of memory.

This makes multi-frame totally unusable. Please, I need this feature. When can you fix it? Do you have a bug fix release? Have you fixed it already?

Sincerely,
Wayne

[NinjaTrader_Dierk]

>> Add(PeriodType.Second, 1);
This line causes NT to load tick data for the timeframe backtested.

You likely hit the wall as you run backtest on tick/volume/second based series larger than e.g. 1 month, since just too much data needed to be loaded by NT. Please try smaller time frames

[wayneFH]

Sorry, you assumed incorrectly. I only had this loaded 6 days of data. I tried it only 3 days of data too but it's still gobbles all the memory. It just takes it slightly longer, that's all.

This is not an issue of memory usage. It's a memory LEAK.

To be more specific, Ninja Trader starts only using 70 meg before I load this strategy onto a chart. That's an excellent memory footprint.

After adding this strategy to the chart the first time, NT goes to 120 meg of usage with the same 6 days of data. That's still an excellent footprint for 6 days of tick data.

If you then press F5 or click to reload Ninja Script on the chart, it adds another 40 or 50 meg, AGAIN! so now NinjaTrader takes 170 meg.

I constantly reload changes to my NinjaScript. So after only about 10 times it uses 500 Meg of memory for the same 6 days of data. After 15 times, it blows out the 1 Gig of memory.

So I have to restart NT, connect to the replay tester again, and re-add the strategy. A real hassle.

Also, if I start this strategy running against real time data, it only takes in a few hours to gobble up all 1 GIG of memory. So I can't use it for automated trading.

Please look at you code for the Add() function. I assure you, it is loading the data, sticking it in a list. Then when the strategy gets reloaded, it does it again, and leaves the previous data still in the list. Thereby it creates a horrendous leak.

Please load the zip file I included and try it. You will see for yourself how bad it is. Use only 3 days or 6 days if you want.

It is still horrible.

If it didn't LEAK so bad, I could use it on 2 weeks to 4 weeks of data no problem.

Please, please, please, trust me. I'm a software developer for 20 years Java, C#, COBOL, C, C++ and many more.

I am handing you precise sample code that exactly replicates the problem.

Please fix this ASAP. It's unbearable. It makes Multi-frame data unusable in any time frame except maybe daily.

Even then it still leaks. So if someone loads several years of multiframe daily data, it will still gobble their memory up after reloading the strategy several times on the chart.

Fix the Add() function, please.

Sincerely,
Wayne

[NinjaTrader_Dierk]

Please remove your custom indicator to see if that makes any difference

[wayneFH]

NOTE: Since this is leaking so bad, I have switched to using only a single time frame on the chart of 1 second data. So I have eliminated using the Add() function.

Now, it doesn't leak. I'm able to load several weeks of tick data on 1 second bars and reload the script, run it in market replay, and against real time data and the memory footprint remains unchanged.

So the problem definitely isn't related to the use of too much tick data.

It's the Add() method, failing to release memory.

But it's a real pain looking at 1 second charts.

I want to use your multi-frame feature so that I can look at 1 minute charts but have the historical testing operate on one second data so that it can enter on the open of the bar and trade within the 1 minute bar.

Please fix this ASAP.

Wayne

[wayneFH]

Just saw your post.

I already removed the custom indicator. I commented it out.

When I do that it leaks to the tune of 40 to 50 Meg every time you reload the script.

So I reversed it and commented out the Multi-frame 1 second data and only ran the indicator.

Then it leaked at 1 Meg per reload of the script.

So I ran the Indicator ALL by itself on the chart to see if it was the culprit and NO. The memory changes absolutely not at all with only the custom indicator.

Trust me, I have already tested every scenario. I debug software for a living.

Use ANY indicator you like, the Add() function itself causes the leak.

Obviously, with indicators it's less noticeable since it only eats a small amount each time. But it leaks just the same.

Please just try it out and see for yourself.

Sincerely,
Wayne

[NinjaTrader_Dierk]

This is a bug which will be fixed with next update early next week. Thanks for reporting.

[wayneFH]

Thank you so much for looking into this. I can't wait to get that release. I am really starting to love NinjaTrader over the other platforms I use. NT is more elegantly designed, modern and your tech support is clearly very responsive from all the posts I see on the forum and your attention to this.

Keep up the great work!

Wayne

[NinjaTrader_Dierk]

Thanks for your kind words.