memory usage and garbage collection?

[NinjaCustomer]

Should I be worried when memory usage gets high or does that just mean it is prime for garbage collection and eventually it will be garbage collected before anything bad happens?

Is there any way to manually force GC?

[NinjaTrader_PatrickG]

It is expected that memory would grow based on actions within the platform.

There is no way to force garbage collection via NinjaScript. There is no way to force garbage collection other than restarting the platform.

[NinjaCustomer]

I have been monitoring the usage since I wrote you.. and it is much less once the market has slowed down, it was moving fast when I noticed the huge ram increase. Of course I'm aware that over time memory can grow.. but it's been pretty stable (well under 1gb in task manager) once i restarted nt8 after the market slowed down. But when it was moving fast, it moved so fast that my chart spontaneously re-loaded indicators (twice in a row)... I mean memory usage was 10x more than it is right now when the market was moving fast.

So in this kind of situation there is no danger of crashing? just indicators reloading and maybe gc pause?

[eDanny]

Why would a fast market cause indicators to reload?

[NinjaCustomer]

Why would a fast market cause indicators to reload?

well like I said the memory shot up 10x, and then i noticed the chart reloaded twice (the 2nd time pretty much straight after the first).. so I don't know if it is GC or what.. that said, these are not simple indicators but ones that use a lot of resources compared to normal indicators. The memory usage has really been remarkably stable since I reloaded nt8 after the fast market moves.. but i also did reduce by half the # of days lookback on each chart as well. So I'm guessing had the workspace been the way it is now, it would have shot up maybe 5x (and possibly not have reloaded the indicators on my ES chart) what it is now not 10x like it did.

[NinjaTrader_PatrickG]

It would be hard to diagnose without more specific information. Can you reproduce that large memory increase without any custom indicators or add-ons?

[NinjaCustomer]

It would be hard to diagnose without more specific information. Can you reproduce that large memory increase without any custom indicators or add-ons?

no, because I can't even reproduce it without a fast moving market.. however.. it is not important to diagnose the exact problem. I'm just curious how NT8 handles the situation where a lot of memory gets allocated quickly... what happens? no crashes usually right?

[NinjaTrader_PatrickG]

Again, this is all very general information, but I would never expect NinjaTrader to 'crash' due to high memory usage unless your PC runs out of available RAM.

[NinjaCustomer]

I was just watching it now (freshly opened ninjatrader) - after a while, market was moving and it got up to 4gb used ram (not swap, but used ram - I don't know what the swap was, I wasn't watching it)... then I disconnected for some time and came back .. did not reconnect to cqg... the used ram was back down to 2002MB but the swap was 4116MB ... could this be indicative of a memory leak on one of my indicators?

[NinjaTrader_PatrickG]

I would test in safe mode to be sure, but yes - a custom add-on, or the cumulative effect of your workspace's complexity, could certainly cause this.

Safe Mode will prevent NinjaTrader from:

• Loading workspaces.
• Connecting on start-up.
• Loading custom assemblies.
• Getting instrument updates from the server.

To enable safe mode, please use the following steps:

• Exit NinjaTrader.
• Hold the CONTROL key and double click the NinjaTrader icon.
• Keep the CONTROL key held down until you see the Control Center.
• You can verify you are in safe mode by going to Help -> About.

If there issue doesn't occur here, then we would basically need to pick apart the workspace from the top-down or bottom-up to discover the root issue. Also, I have provided a link below to our Help Guide that goes over performance tips:

• http://ninjatrader.com/support/helpG...ance_tips2.htm

[NinjaCustomer]

I didn't test it in safe mode but what I have done is this:

I copied the workspace to 2 different machines - let's call them A and B

I have been running them the last 2 trading days...

I removed half the indicators from each chart of the workspace on machine A

and the other half from each chart of the workspace on machine B

all charts have the same template so this effectively splits the indicators into running on either machine A or machine B only.

and I also have my main machine running.

However, after monitoring the 3 machines today, it is apparent that is not an indicator issue (or at least if it is an indicator issue there is something else also going on)

I can say this because the ram and swap usage have both grown a lot again today on my main machine but on the other machines, they stay stable.. only a few 10s of MBs more than when they were started.. whereas the main machine has grown from 1200/1550 to 5231/5687 in under a half hour.. and then I restarted it and it grew from 721/1570 to 3942/4176 again in under a half hour.

On Friday it didn't really grow and was more or less stable with ram on all machines including the main one but when I started this thread, i had the same problems as today.

So do you have any suggestions as to what it could be? the test machines have fresh ninjatrader 8 installs whereas the main one has been at least for 6 months ( I might have reset the database once but can't remember, anyhow it would have been months ago)

I would not like to delete my order history if it can be at all avoided but I don't mind removing the tick data.

The "days to load" on all these charts is only maximum 5 days or so.

I believe I fixed the problem

I did this by uninstalling my main NT8 installation, deleting the folder first.. but making backups.. then I imported the backup that included everything but the chart data, logs and trace files (so database was included)

this resulted in an NinjaTrader 8 documents folder size of under 100mb rather than 1.3gb previously

So it seems somehow even though the lookback on the charts was only 5 days, somehow having this large several months worth of chart data caused this ram munching problem. Is this a known issue?

I've attached an image of the ram use today that shows how it became stable after reinstall. When ram drapped before the reinstall, that was from me restarting the platform.

left column is RAM usage, right is SWAP for Ninjatrader 8

[NinjaCustomer]

what is the procedure for clearing all chart data in NT8 but leaving the database (trade history) intact?

[NinjaTrader_PatrickG]

If you wanted to remove locally stored historical data only, you would want to follow these steps:

• Shut down NinjaTrader
• Navigate to Documents > NinjatTrader 8 :> db folder
• Delete all subfolders. DO NOT delete the NinjaTrader.sdf file
• Restart NinjaTrader

[NinjaCustomer]

an the trade history will be safe this way?

[NinjaTrader_PatrickG]

Trade history is contained in the NinjaTrader.sdf file so as long as you don't remove that file your trading history will remain.