NT8 Charts Lags and Delays - Need a solution ASAP

[biegea]

I'm not here to rant, but I have to strongly agree with the users here. NinjaTrader really ought to be prioritizing helping users through these performance issues. It is not possible to run the tests you recommend because we cannot make the market dump fast crazy data on us at our discretion. It happens when it happens and is not reproducible on demand. So we are all left clueless as to how many compromises we have to make, or even general order of magnitude, and then there is no way to know if anything helped, because we don't know if or when a lag has been averted.
NinjaTrader should have a more robust infrastructure (either within the software itself or within support) to help identify in a much more targeted way how to resolve these nasty and legitimate lag issues. These are not newly reported issues. This has been being reported for years. This isn't just nerds whining about an imperfect computer program. These are money-draining, and potentially business-crushing issues when they strike with sufficient severity, at the wrong time, and when size is on, and I wish Ninja would prioritize doing everything possible to enhance the software to enable the elimination of these issues and also to enhance support's ability to help users quickly and specifically target the most offensive drivers of the issues when they strike. Thank you for your consideration.

[NinjaTrader_ChelseaB]

Hello biegea,

Unfortunately, as the behavior tends to occur when the volatility increases, it can be necessary to do the testing during high volatility.

To confirm, you are finding the NinjaScript Utilization Monitor detailed in the forum post linked from post # 5 is not sufficient for determining which scripts are causing high resource use?

As far as prioritizing everything to enhance the software, one large performance increase would likely result from removing all 3rd party addons, as well as reduce CPU intensive scripts such as order flow which can process large amounts of data and require a lot of processing power. These would be what is causing the performance impact.

[biegea]

Thanks, Chelsea. Your link didn't work for me, but for sure, I use the Utilization Monitor to guide me to what is driving the highest resource usage. But I don't know things like:
-how much utilization overall is "good" and how much is "too much." Guidelines for good vs shaky baselines would be really helpful. I know this isn't trivial as there are many variables; that's why it would be helpful
-my "Candlestick" chart style number is almost always near the top. Is that normal? Is the amount of utilization ok? If I change a dozen charts to use bars, instead, what overall percentage gain can I expect, for example. And whatever the gain, will it actually have any impact? I have no idea how much gain I need to attain a given type of impact.
-I don't suspect Ninja would remain in business much longer if you disabled support for 3rd party addons or OrderFlow indicators. On that note, however, there are other software packages (also which allow 3rd party addons and custom programming) that process order flow data and indicators without anything near the same degree and consistency of issues that we see in Ninja.
Cheers.

[NinjaTrader_ChelseaB]

Hello biegea,

Long links are abbreviated on the forum. May I confirm you are clicking the link and not attempting to copy and paste it? Or you right-click the link and select 'copy link address'.

I have tested clicking the link in post # 5 and this is working correctly.

how much utilization overall is "good" and how much is "too much."

This would completely depend your system hardware. An extremely fast single threaded CPU computer is going to handle multitudes more operations than a cheap computer with a slow single threaded performance. I would not be able to predict this. A $200 computer from walmart could freeze up with just a few simple indicators. A specialized expensive computer could handle many more instances of much more resource intensive scripts.

In general, if there is something using the CPU way more than something else, that's what is causing the high resource use.

my "Candlestick" chart style number is almost always near the top

The chart bars painting on the chart may have a lot of cpu time as its constantly redrawing the chart. However, if you test with no indicators using minute bars, you will likely see there is no performance impact.

If I change a dozen charts to use bars

Sounds like you have a lot going in your workspace. I highly recommend following the guide I linked in post # 5 and reducing the load.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1055200-nt8-jump-in-memory-usage?p=1055235&_gl=1*beivvf*_gcl_au*MTU3NzgyOTEzOS4xNjgxNDk0ODg1#post1055235

If I change a dozen charts to use bars, instead, what overall percentage gain can I expect, for example

Change chart bars from what?
If you were using Volumetric bars, these are very resource intensive. Changing to minute bars would be magnitudes less resource intensive. Minute bars do not process all of the ask and bid volume of every tick and are super simple.
If you were using range bars, these are also super simple in logic but are created historically with historical tick data instead of historical minute data. There can many many ticks in 1 minute. Meaning tick data is larger than minute data. Changing from range bars to minute bars would not have much a difference in performance impact in real-time, but would have an impact when loading historical data.

I don't suspect Ninja would remain in business much longer if you disabled support for 3rd party addons or OrderFlow indicators.

That's kind of the gist. Because NinjaTrader is platform designed for Addons coded in C#, and runs on the local computer it is subject to the user's hardware limitations. Instead of making the decision to close the ecosystem and remove the ability to design custom addons, thereby completely controlling the user experience and performance impact, NinjaTrader decides to stay open, and try our best to work with developers to make more efficient code. Sometimes the nature of what the addon code does may an unavoidable performance impact, especially when the amount of data to process drastically increases in high volatility. Having a CPU with a fast single threaded performance becomes necessary when placing a high load on the computer.

The NinjaTrader Development does implement any performance enhancements that are able to be implemented, as we all want our users to have a good experience with NinjaTrader. As an example, NinjaTrader 7 was largely single threaded. Meaning even if there were multiple cores were available, all charts, indicators, and other windows were all running on one logical processor. The high volatility mornings severely impacted a much much higher percentage of our users all simultaneously as we receive reports. NinjaTrader 8 was designed to be multi-threaded. This means more threads can run, meaning more instruments, as those threads can run on other logical processors and spread the load. While more threads can run, the individual thread, like the indicators running on a chart runs in that instruments thread, is still limited by the single threaded performance. Fortunately though, with these improvements the amount users that send in reports of being severely impacted in high volatility has drastically decreased. We mostly get these reports from users with insufficient hardware for the amount of demand being put on the computer (meaning too many high resource scripts and too poor of single threaded performance).

[biegea]

Clicking the link brings me to this blank page, full URL https://ninjatrader.com/support/forum/sguncategorized

[biegea]

Thanks for the detailed reply, Chelsea. I appreciate your detailed responses. But most of your response was general response to example questions, with no context. This is why I am saying a somewhat comprehensive guide to using the Utilization monitor that digs into some of these variables and and provides some form of established baselines could be helpful.
For what it's worth, yes, I do have a fair amount going on in my workspace, and I use what is there. My computer is a commercial-grade workstation built specifically to be a trading computer. The highest resource-consuming items are built-in Ninja indicators.

Perhaps I will experiment with some of my charts/indicators to balance between MES and ES. Not ideal, but given how close they trade, perhaps offloading some of the heavier indicators to the one symbol will eliminate lag on the other, since the two symbols should execute on different cores. Might this work the way I am proposing? Any tips in going down this path?

Additionally, if you reply to this before anyone has responded to this other post I wrote, can you please respond to this other one too? Thanks in advance https://ninjatrader.com/support/foru...nd-bar-spacing

[NinjaTrader_ChelseaB]

Hello biegea,

That is not the link address in the link..

May I have you right-click the link and select copy link address, then paste this with your reply?

I made a quick video to show the link works, but the video is also a link. I'm adding the link text so you can copy and paste this into a browser.

https://drive.google.com/file/d/1jb8PLnqeOkh-kvA0Sx65AUO9MQHpbAcB/view

I'm also including the link text to the forum post.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1055200-nt8-jump-in-memory-usage?p=1055235#post1055235

[biegea]

RMC > copy link yields this: https://ninjatrader.com/support/forum/node/5
When I paste it into a browser it goes to the page I showed you before.

Thanks for the long-form link you provided below. I will read that thread shortly.

Hello biegea,

That is not the link address in the link..

May I have you right-click the link and select copy link address, then paste this with your reply?

I made a quick video to show the link works, but the video is also a link. I'm adding the link text so you can copy and paste this into a browser.

https://drive.google.com/file/d/1jb8PLnqeOkh-kvA0Sx65AUO9MQHpbAcB/view

I'm also including the link text to the forum post.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1055200-nt8-jump-in-memory-usage?p=1055235#post1055235

[NinjaTrader_ChelseaB]

Hello biegea,

Try using a different browser. Same behavior?

[biegea]

Hello biegea,

Try using a different browser. Same behavior?

yes same behavior in both chrome and firefox, on two different computers

[biegea]

Also please don't forget post 21 https://ninjatrader.com/support/foru...37#post1219537 (in case it got lost in the thread)

[NinjaTrader_ChelseaB]

Hello biegea,

Are you able to click your own link?
(Your link works for me, and the link in post # 5 also works for me)

I would like to schedule a call with you to investigate on your end.
Please send an email to [email protected] Attn ChelseaB 03389477. In the email please include a link to this forum thread.

Not ideal, but given how close they trade, perhaps offloading some of the heavier indicators to the one symbol will eliminate lag on the other, since the two symbols should execute on different cores.

Yes, different instruments are different threads, which can run on other logic processors. Decreasing the amount of scripts running on the same instrument will alleviate the stress on charts with that instrument.

Your other thread is in the Platform Technical section of the forums and is assigned to a platform support technician who will be following up.

[sunlounger]

Hello everyone, I had a horrible trading experience this week trading RTY and having the same problem. I think that using NT in plug-in mode along with R trader pro could solve the problem. This means you guys start R pro and then connect NT8 to it. From R pro you can set the updates per second(preferences-performance-updates per second). I am going to set them as low as 5 for quotes and DOM and 3 for charts. I really hope this will solve the problem. I will keep you posted.

[backtester831]

I'm sorry to hear about your horrible trading experience with RTY. It can be really frustrating when things don't go as planned. Using NT in plug-in mode along with R trader pro sounds like a possible solution to your problem. Connecting NT8 to R pro and adjusting the updates per second settings could potentially help. Setting them at 5 for quotes and DOM, and 3 for charts sounds like a good plan. Hopefully, this adjustment will resolve the issue for you. Please keep us updated on how it goes. Good luck!

What is this bot post?

[NinjaTrader_ChelseaB]

Hello backtester831,

Thank you for catching that. I've removed bimber as I agree this appears to be a bot and has included spam links in other posts.