Chart is lagging big since the new update

[Testmaster]

Hello,

I use Windows 10 and Ninjatrader 8.0.26.0 64 bit
Before I updated to this Ninjatrader version I didn't have this problem.
In the morning everything works. In the afternoon it is lagging massivly.
Attached 2 screenshots.
What is the problem?

Thanks in advance

Testmaster

[NinjaTrader_ChelseaB]

Hello Testmaster,

The screenshots appear to show a screenshot taken at 16:31 with 3 more bars than the one taken at 16:24.
This would be expected, in that 6 minutes it is possible from 3 bars to close on a 4 range chart.

Regarding performance, slow performance is typically attributed to an overworked CPU.
Below is a link to a forum post that tips for reducing the load on the CPU to improve performance.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1055200-nt8-jump-in-memory-usage?p=1055235#post1055235

[Testmaster]

I have an AMD Ryzen 9 3900X 12 core processor, 64GB RAM..... I have never had a problem. Since the latest update I have these problems in the afternoon.
Thanks for the link. I will use it, but I don't think it is the problem.

[NinjaTrader_Jim]

Hello Testmaster,

Keep in mind that different addons require different amounts of processing. Order Flow type indicators are also very resource intensive. Custom Rendering can also be resource intensive if not done carefully.

Essentially, we should not simplify the case that our hardware is "too good" to display performance issues.

A good way to see if some software is overburdening on the CPU would be to test a blank workspace that does not have any custom components or templates added. This should perform as fast as we expect because there would not be anything slowing the platform down. If a particular component is slow, perhaps the developer can make changes to make it more efficient. If not, the workspace would need to be scaled back in terms of data and complex components.

Testing with a blank workspace without anything third party is also a good way to communicate NinjaTrader out-of-box-specific behavior to us.

[Lancer]

When NT charts are lagging (fast markets, every day), I have yet to see the CPU anywhere near 100% CPU utilization, and usually 40% max. (overall), with occasional spikes to 70% max on any single thread. Always plenty of unused computer resources, CPU, GPU, and memory, so not a hardware overworked or overburdened issue. Chart lag existed long before 8.0.26.

[NinjaTrader_ChelseaB]

Hello Lancer,

We often see this behavior when using 3rd party indicators and with workspaces that load large amounts of data.


As a test, please take the following steps to force NinjaTrader to start without your saved workspaces. You will have the option to reopen these after our tests.

First, if possible, I suggest making a backup of your workspace so we can restore a backup if any changes need to be undone.

Below I have provided a link to the Backup and Restore section of our help guide.

http://ninjatrader.com/support/helpG...p__restore.htm

Then to start without a saved workspace:

• Shut down NinjaTrader
• Navigate to (My) Documents\NinjaTrader 8\workspaces
• Select all files in this folder -> right-click one of those selected files -> select Cut
• Navigate to the Desktop -> right-click the Desktop -> select New -> Folder
• Open this folder -> right-click in the folder -> select Paste
• After moving these files, restart NinjaTrader which should startup without any workspaces
• Once NinjaTrader has started on just the control center, please open a new chart using 1 minute bars and try reproduce the behavior.
  Do not load a template on this new chart or any 3rd party indicators.
  
  * Please note this will not delete your existing workspaces. However the new blank workspace will be named Untitled1. If you have a workspace already named Untitled1, rename the workspace in the Workspaces menu by clicking Save As > and give a name for the workspace like TestingWorkspace. Then do not save Untitled1 when shutting down NinjaTrader (otherwise it will overwritten).

Please let me know if on this new 1 minute chart in a fast market with high volume, if the behavior continues when all other workspaces are closed.


Once you have tested for the behavior, you can optionally return the workspaces to the workspaces folder and restart NinjaTrader 8. NinjaTrader should only start with the TestingWorkspace (or Untitled1 if you chose not to rename it) with these steps so you can continue using the platform without the workspace automatically opening.

• Navigate to the Desktop\New Folder\
• Delete one file named _Workspaces(.xml)
• (delete this one file. do not delete any other files without that exact name.)
• Select all files in this folder -> right-click one of those selected files -> select Cut
• Navigate to (My) Documents\NinjaTrader 8\workspaces
• Right-click in the folder -> select Paste
• After moving these files, restart NinjaTrader

Once we know if NinjaTrader is still reproducing the behavior without the saved workspaces we can move forward to identifying an issue within the workspace or an issue with the platform itself.

I look forward to further assisting.

[DTSSTS]

I use Task Manager to see how much memory Ninja Trader is using, Later in the day you may see this building up to very high levels. (Pretty much at High levels of memory usage you will need to restart Ninja Trader to clear that memory consumption.) Things I have found overtime that us massive amounts of resources are Drawings and Label Rays Indicator. Of course lots of opens charts with many indicators and if you are running strategies in the control center with lots of Instruments that can build up resource usage. Good Luck

[Testmaster]

Hello today again problems.... see the screenshots

[NinjaTrader_ChelseaB]

Hello Testmaster,

This is likely due to the indicators applied to the chart.

Are you able to reproduce with a blank chart with the instructions in post #6?

[Testmaster]

If I have a blank workspace the chart seems to be ok. But normaly I have more charts open.....

[NinjaTrader_ChelseaB]

Hello Testmaster,

This would confirm the computer hardware is not able to keep up with the market volume when using those indicators.

Post #2 includes a link to a forum post about reducing the load on the CPU.

Sometimes scripts are not written efficiently. If this is a 3rd party script, report the behavior to the author or vendor and inquire if any performance optimizations can be made.

If this is a script you created yourself, see this example that demonstrates the importance of re-using memory locations, creating brushes in OnRenderTargetChanged(), and not doing any calculations in OnRender() which can use computers to be slow in high volume.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/indicator-development/98639-performance-optimization?p=789606#post789606

And a link to the help guide on best practices for performance.

https://ninjatrader.com/support/helpGuides/nt8/index.html?ninjascript_best_practices.htm#Performance

[Lancer]

ChelseaB, How is it that you figure Testmaster's chart lag is due to hardware unable to keep up with market volume when his computer has a 12-core 4.2 GHz CPU at only 11% utilization (89% remaining available), and a GPU at only 34% utilization (66% remaining available), and 64GB memory at only 14% utilization (86% available)? With the great majority of Testmaster's CPU, GPU, and memory unused and available for NinjaTrader to use, what component of computer hardware are you talking about that is unable to keep up with market volume?

[Testmaster]

ChelseaB, How is it that you figure Testmaster's chart lag is due to hardware unable to keep up with market volume when his computer has a 12-core 4.2 GHz CPU at only 11% utilization (89% remaining available), and a GPU at only 34% utilization (66% remaining available), and 64GB memory at only 14% utilization (86% available)? With the great majority of Testmaster's CPU, GPU, and memory unused and available for NinjaTrader to use, what component of computer hardware are you talking about that is unable to keep up with market volume?

great question.... I don't understand this, too. That's why I posted my Task Manager to show that it is not a hardware problem. And I never had this issue before the update.
But I also contacted the author of my indicator.

[NinjaTrader_Jim]

Hello everyone,

NinjaTrader has multiple threads for different tasks. To keep things simple, there are UI threads, there are Instrument threads, and there are thread pools.

Rendering and GUI work happens on UI threads.

Processing data happens on Instrument threads or in thread pools for historical processing.

Instrument processing must be synchronous for NinjaScripts to function properly. This means that resource intensive scripts, like volume profiles, cannot distribute their work among multiple threads.

In the application, all of these threads have jobs and some of them need to wait on one another.

You very rarely will see 100% utilization on the an individual thread when lagging symptoms arise.

We are trying to ask objective questions to help narrow things down, giving our perspective. After all, the question is: "what is making the lagging symptoms occur?"

The way to do that would be perform the blank workspace test, to see the platform operating at full speed, and then to identify which additions are making the most impact. This is best tested when you see the symptoms, i.e. when there is greater volatility, otherwise you may not be able to notice the impact that arises.

If you are looking at Task Manager, look at all logical processes, and keep in mind that you will not see 100% utilization on a single thread.

[Lancer]

So it really isn't that computer hardware is unable to keep up with market volume, it is NinjaTrader's thread processing architecture that restrains NT from utilizing all available hardware resources during high trade volume periods. If the workspace is sufficiently "complex", requiring a lot of thread coordination and wait states, the result is chart lag despite massively underutilized hardware resources.

It is important to know this because so many users with chart lag and adequate hardware first try to solve the problem by unnecessarily spending money on bigger hardware while NT is leaving significant hardware capacity unused as-is.