NT8 - Jump in Memory Usage

[markbb10]

Hello All,

I am using NT8 8.0.17.2. My workspace has been essentially static for more then six months, meaning no new indicators or growth in windows open or complexity. For the last month or so I have seen a dramtic increase in the amount of memory being used by NT. The NT process used to average around 1 - 1.5 gig of memory during the day, the process is now averaging 3GB or more.

Is there any obvious reason why this would be? Is it a database issue perhaps ?

Thanks,
Mark

[NinjaTrader_ChelseaB]

Hello markbb10,

One thing that may be related is the amount of recent increase in market activity.

Are your charts minute or day charts or are these tick based bars?

Are the charts running with Calculate set to On bar close or are these updating for each tick or price change?

Are you using 3rd party indicators?

Are you able to reproduce this behavior without using 3rd party indicators?

The NinjaScript Utilization monitor can also show scripts that are using large amounts of CPU time which may relate to new memory locations being taken.

• Control Center > New > NinjaScript Output
• Right-click within the NinjaScript output window > Select 'NinjaScript Utilization Monitor...
• This window will begin to populate with NinjaScript items in order of time spent processing

[markbb10]

Hi Chelsea,
My charts are tick based but have been for years, all of my charts/indicators run with "on bar close". I find using anything else can bring me to a standstill. I use several 3rd party indicators but all of these indicators have been configured into the workspace for a minimum of 2 years. My setup and daily work is very static and stable. I doubt the problem would exist if I removed my indicators as the system would not be providing any information. I will give the utilization monitor a try and see if I can get any clues.
Thanks for your response.
Mark

[markbb10]

Chelsea,

I did find a few charts that are set to calculate on price change. I will experiment with that as well.

Thanks again,
Mark

[NinjaTrader_ChelseaB]

Hello Mark,

Its important to keep in mind that different amounts of data that must be processed as well as different indicators and logic will cause the system to have different performance from day to day.
Some indicators are programmed have different logic and resource needs depending on market conditions.
(As a very simple example, an indicator that recognizes a pattern may start rendering visual objects with inefficient code in OnRender(). The amount of resource use becomes high the moment this pattern is recognized.)

The more ticks the exchange sends out, the more CPU cycles that need to be processed by NinjaTrader. This means a workspace/chart setup that was working fine for the last 18~ months may not now have the overhead to keep up with the sheer number of ticks that are being processed in the last few days.

Indicators that process ticks or use tick based bars, create a lot of objects, or are inefficiently coded will require more CPU time and memory.

If the behavior can only be reproduced while using 3rd party indicators, this highlights that these are likely the scripts that are taking those memory locations. If the behavior of high memory usage cannot be reproduced when starting NinjaTrader without any 3rd party scripts, the issue is may not be with the actual platform itself, but instead the amount of demand caused by 3rd party indicator use.

NinjaTrader does provide a warning that using 3rd party addons can have adverse effects on the entire NinjaTrader application.
https://ninjatrader.com/support/help...tallingAdd-Ons

(This is why we first test for the behavior without 3rd party scripts and without background workspaces running, to get an understanding of where the behavior is originating from. This is also why the Utilization Monitor was introduced, to help identify which scripts are using high CPU resources specifically)

Further, the Order Flow+ indicators and Volumetric bar type are premium indicators provided by NinjaTrader that also process tick data and have a higher performance impact than other system indicators and bar types provided by NinjaTrader.


Below I am providing general information about improving performance by reducing the load on the CPU.

Slow performance is typically attributed to an overworked CPU.

The performance when NinjaTrader is loading a workspace or opening a chart is attributed to the processing of historical data. After the workspace or chart has loaded and all historical data has been processed the performance is primarily affected real-time data being processed.


Focusing on real-time performance (after the workspace or chart has loaded):

Using 3rd party add ons that use lots of resources can cause performance freezes, especially during peak trading.

The 'Calculate' setting for indicators can be important. If set to 'On each tick' the indicator will be calculated on each incoming tick. During high volatility this can be very CPU intensive especially if the indicator makes complicated calculations. Please set it to 'On bar close' in the Indicator parameters and the indicator will only be calculated at the end of each price bar which reduces the load on your computer by a large amount. As a heads up, some NinjaScripts are more CPU and resource intensive than other scripts.

The greater the number of charts open with indicators running with the 'Calculate' property set to 'On each tick', the more demand is placed on the CPU core that NinjaTrader is running on.
If the script is not collecting volume or needing to analyze individual ticks, but must be updated in real-time, the 'On price change' setting will use much less CPU than 'On each tick' and still updates in real-time.

Below is a public link to the help guide on Indicator properties. Please see the note for Calculate.
https://ninjatrader.com/support/help...torsParameters

There also may be a recent increase in market activity for this instrument. The more ticks the exchange sends out, the more CPU cycles that need to be processed by NinjaTrader. This means a workspace/chart setup that was working fine for the last few months may not now have the overhead to keep up with the sheer number of ticks that are being processed today or in the last few days.

To see what (if any) NinjaScript items may be contributing to the symptoms, please open the NinjaScript Utilization Monitor:

• Control Center > New > NinjaScript Output
• Right-click within the NinjaScript output window > Select 'NinjaScript Utilization Monitor...
• This window will begin to populate with NinjaScript items in order of time spent processing

Below is a link to the help guide on the NinjaScript Utilization Monitor window.
https://ninjatrader.com/support/help...izationMonitor

Tips to improve real-time performance:

If you have workspaces opened in the background which you do not use, we suggest to close such workspaces.

To do this in NinjaTrader 8:

• On the Control Center open the Workspaces menu and look for any workspaces that show an icon of two squares on the left of the workspace name
• The workspace showing the icon with the green square on top of the gray square is the workspace opened on your monitor
• Any workspaces that are showing icon with the gray square on top of the green square are opened in the background and are using resources
• Hover the mouse above a workspace that is open in the background (with the gray square on top of the green square)
• On the right the open to close will appear, click close
• Repeat this procedure until only the active workspace (with the green square on top of the gray square) is open

Below is a publicly available link to the help guide on closing a workspace in NinjaTrader 8.
https://ninjatrader.com/support/help...singAWorkspace

On a chart, items that will improve real-time performance:

• Remove redundant charts and indicators
• Set the 'Calculate' property in the indicator parameters for as many indicators as possible to 'On bar close' (or 'On price change' if they must be real-time) instead of using 'On each tick'.

Below is a link to the help guide on changing indicator parameters.
https://ninjatrader.com/support/help...torsParameters

Focusing on load (historical data) performance:

When opening a workspace, the amount of data loading and the number of indicators are the biggest factor for how quickly that workspace will open or a chart will load. The Days to load setting in the Data Series window controls how much historical data is loading. A finer the granularity bar type (meaning daily data has less bars in date range than minute data, which has much less bars than tick data, etc..) will load more bars. The more data that loads, the longer it will take for the indicator to process that historical data and load the chart.

All indicators will need to process for every historical data bar on the chart. Some may even need to process all individual historical ticks for that period as well. The more indicators and the more charts there are, the longer the initial load of the workspace or chart will take.

Further, indicators that perform heavy calculations will load slower than indicators that are not as CPU intensive. This is compounded by the amount of historical data a CPU intensive indicator must process.

To see what (if any) NinjaScript items may be contributing to the symptoms, please open the NinjaScript Utilization Monitor:

• Control Center > New > NinjaScript Output
• Right-click within the NinjaScript output window > Select 'NinjaScript Utilization Monitor...
• This window will begin to populate with NinjaScript items in order of time spent processing

Tips to improve load performance:

If you have workspaces opened in the background which you do not use, we suggest to close such workspaces.

To do this in NinjaTrader 8:

• On the Control Center open the Workspaces menu and look for any workspaces that show an icon of two squares on the left of the workspace name
• The workspace showing the icon with the green square on top of the gray square is the workspace opened on your monitor
  Any workspaces that are showing icon with the gray square on top of the green square are opened in the background and are using resources
• Hover the mouse above a workspace that is open in the background (with the gray square on top of the green square)
  On the right the open to close will appear, click close
• Repeat this procedure until only the active workspace (with the green square on top of the gray square) is open

Below is a publicly available link to the help guide on closing a workspace in NinjaTrader 8.
https://ninjatrader.com/support/help...singAWorkspace

On a chart, items that will improve load time performance:

• In the Data Series parameters, reduce the number of 'Days to load' or decrease the date range to improve a charts initial loading time
• In the Data Series parameters, disable Tick Replay (if enabled and not needed) to improve a charts initial loading time
• Remove redundant charts and indicators

More tips and suggestions to improve the performance of NinjaTrader can be found at the link below.
https://ninjatrader.com/support/help...ance_tips2.htm

[markbb10]

Thanks for this info Chelsea. I will continue to investigate. My primary point was this problem did not exist a few weeks ago and I have used this workspace in high volatility markets before with the same indicators and charts. It is also interesting that I have not seen a jump in average CPU usage which has remained constant. I will delve into the monitor some more and see what I find. This is not a critical problem as of yet as platform performance is still acceptable. It was more of curiosity of why memory usage suddenly doubled. Thanks again for your insight.

mark

[markbb10]

So this problem continued to get worse up through Friday when I was unable to work at all. The NT process was now using more then 4gb, even when disconnected from the data server. I have opened up a ticket with support but today (Saturday) I decided to just rebuild Ninja, after deleting NT8 and rebuilding my same environment I seem to be working again with memory now at a more normal 500meg as opposed to the 4.8 gig it was taking this morning when I booted up. I would have liked to have investigated this further but I had a strong feeling it was not my indicators or volatility as my workspace is very static and has been in use for more then 6 months. I did not want to lose another day or two of work hence the drastic rebuild. I will follow up with support on Monday to see if there is any clue as to why this happened

[NinjaTrader_ChelseaB]

Hello markbb10 ,

An easy test would be to make a copy of the workspace and remove all indicators.

With all other workspaces closed, are you able to reproduce with a copy that has all indicators removed?

[markbb10]

Hi Chelsea, I do not think it had anything to do with the indicators. This workspace has been completely static for almost a year and I never saw anything like this. On Saturday with no data connected the NT memory process was growing and was almost 5gb in size. This had nothing to do with indicators. After doing a fresh reload of the NT platform and reinstalling my workspace and templates it was immediately back to normal. Running today under data load it was back to normal with a memory size of around 700 meg which is what I would expect to see. It looks like something was severely corrupted. Thanks for your input, hopefully I will not have to go through a rebuild exercise for some time. Have a great week - Mark

[Cora82]

To do this in NinjaTrader 8:

• On the Control Center open the Workspaces menu and look for any workspaces that show an icon of two squares on the left of the workspace name
• The workspace showing the icon with the green square on top of the gray square is the workspace opened on your monitor
  Any workspaces that are showing icon with the gray square on top of the green square are opened in the background and are using resources
• Hover the mouse above a workspace that is open in the background (with the gray square on top of the green square)
  On the right the open to close will appear, click close tellpopeyes
• Repeat this procedure until only the active workspace (with the green square on top of the gray square) is open