Chart lag (AGAIN)

[NinjaTrader_ChelseaB]

Hello pjsmith,

Our process is to work through steps to find where the behavior may be coming from.
Attempting to analyze everything in your system without testing is basically impossible.

We start by testing NinjaTrader by itself to see if there is an issue with the NinjaTrader platform we need to report to our development.

If the behavior cannot be reproduced on a single chart or with several charts that are not using any 3rd party indicators, then we start testing the 3rd party indicators.

Sometimes we find specific bar types and indicators with poor design choices (such as creating large numbers of objects in the wrong methods) that can lead to undesired behavior.

From our help guide:
"NinjaTrader 8 provides a development environment allowing low-level access to 3rd party developers to build integrated indicators, drawing tools, automated strategies and more. An Add-On with software bugs can have adverse effects on the entire NinjaTrader application. These add-ons also natively run on your computer, therefore, its important to only install 3rd Party Add-Ons from sources you trust.

The following symptoms post installation could indicate an Add-On is installed causing negative impact:

• Windows become slow or unresponsive to user interaction
• Market data becomes unusually slow to load or update
• Standard features fail to work as designed
• Other scripts fail to work as designed (custom scripts that work on their own may conflict with each other)
• Lost connections from market data providers
• Error messages are generated at various times
• The entire application shuts down abruptly and without warning

If you run into any of the above symptoms post installation of a 3rd Party Add-Ons please try uninstalling the 3rd Party Add-Ons to see if the problem goes away and contact the 3rd party developer for support."
https://ninjatrader.com/support/help...tallingAdd-Ons

There is a major change in 8.0.19.0 that enforces better coding design that may cause some indicators that have made poor design choices to no longer work and will produce errors in the Log (and trace).

From the help guide:
"Attention developers:
•When working with D2DFactory to create Direct X resources, this must be done from the charts UI thread otherwise there will be a performance impact. This is now enforced in 8.0.19.0. If your script does not follow this policy it will not be compatible with 8.0.19.0 and a log error will be thrown. Please contact platformsupport [at] ninjatrader [dot] com if any questions."
https://ninjatrader.com/support/help...8/8_0_19_0.htm

Please let me know if you would like investigate the behavior you are reporting so this may be either discovered to be an issue with the computer hardware and size of workspace, or an issue that needs to be reported to a vendor, or if its a NinjaTrader issue that needs to be reported to our development.

[pjsmith]

Thanks for the responses. So it seems new ver is not significantly different, I guess.

ChelseaB, thanks for the response, but I'm not even going to try that exercise (which I have many times before). As stated, I cannot run .19 and continue to work. .19 charts freeze within a few minutes use for me, so I am unable to trade or avoid even more risk than usual, if I install it. In terms of replicating the lag with an empty workspace, 1 chat, etc. I suspect it works, but that setup is no use to me. (or most others, I suspect). It is NOT how most people use platform and frankly, I don't see it proves anything at all, other than NT cannot cope with mildly complex (if you can call 10 charts complex) setups during volatility. If I cannot run my monitors with my 4 charts per instrument, then the platform is useless to me. I currently have 16 charts open.. Actually very little in way of 'indicators'. Ninja utilization monitor shows nothing significant and top resource users are NT supplied code, like the candle type. Given that, not sure the finger can be pointed at any 'custom' changes my side.

[habibalex]

Lag is definitely showing up for me also. We need a way to stress test our indicators with the ability to test OnMarketDepth as well as OnMarketData through the external connection. I made a program that used on the external connection to feed data to OnMarketData using last, bid, ask updates and I am unable to reproduce the lag after playing back data that caused lag while live. I'm pretty sure the OnMarketDepth level2 data is the source of lag in my indicators but I have no real way to test it.

Also, as far as stability goes. My charts keep freezing on v8.0.19.0. I am using an indicator that has a parallel for loop and that is probably the one causing the freeze. Downgrading to the previous version v8.0.18.1 resolves the issue so I am sticking with that for now. The error was in OnRender and "Attempted to read or write protected memory".

[Lancer]

pjsmith, Using the ChartLagTime script to monitor chart lag, there is still significant lag during fast market times with v8.0.19.0. The lag seems to clear faster than with v8.0.18.1, but I don't see much of a difference beyond that. On Friday, US equities charts lagged at the open as usual, and later in the day the ES chart lagged by up to 10 minutes after one of the news events. ES is always the worst though by far, then NQ. At the same time that ES and NQ are lagging by a minute or more, other lower volume or lower trade-rate markets have no chart lag at all. The subject workspace has 6 markets charted, multiple timeframes tabbed for each, with indicators on all, and no other open workspaces.

PC resources during these fast-market events, showed plenty of unused CPU, GPU, and memory resources, with individual CPU threads around 40% utilization average (high compared to normal market). Overall, CPU (3.5 GHz 6-core Xeon) utilization by NT ranges between 5% and 35%, depending on market activity. Not sure where the bottleneck is, but my PC certainly has more resources available for NT to use. I'd increase PC resources to whatever hardware makes a difference, but NT is not using anywhere near all available resources as-is, and other users report the same with higher-core CPUs than mine and more memory, so there doesn't seem to be any point in throwing money at hardware only in hope that it helps.

Otherwise, in all aspects, v8.0.19.0 has been stable, with no issues, no freezing, and no log errors, with the same results on a second PC with similar hardware and NT configuration.

ChelseaB, my guess is that there is no chart lag using a blank workspace with only one chart and no indicators, but that configuration is not a realistic use model.

[NinjaTrader_ChelseaB]

Hello pjsmith,

Can this be reproduced with 8.0.19.0?

If so, can be reproduced with all workspaces closed in a new blank workspace with a new chart that has no indicators applied?

[pjsmith]

I hate to revisit this, but I am interested to know if anyone during Fridays spike suffered the dreaded chart lag, whilst using the new version of NT released the other day?

For the most part, I have a completely bare bones NT now. Had to strip it and minimise everything to get any reliable performance and unless we get stuff like Friday, the open, or the close, for the most part on stuff like NQ (volatile), it's acceptable (not perfect, but acceptable).

I've been running the chartLag script for a while, nmonitoring latency. During that down move on NQ, it was topping out around 80 seconds lag. I've attached a screen shot showing the lag in excess of 60 seconds. It stayed there for a few minutes, at least, then resumed it's normal 120-200 ms (I'm transatlantic for data).. Of course, the _perfect_ setup occurred, but I'm looking at data over 60 seconds old, so no point even looking at the mouse! There's nothing left I can optimise. I've even removed / enhanced NT bar types to make them faster. The biggest resource use according to ninjascript utilisation manager are things like 'candle type'.

Now, I'm not looking for response from NT - I know they cannot fix this. But, there was comment that the new version just released may have some performance enhancements to resolve this. So, just looking for any feedback from anyone who is using the latest, if they had any issues with lag Friday.

For my part, I tried the new version, but very, very, quickly, found many issues with charts freezing, loads of errors in the log re NT drawing objects, etc. So, I rolled back to the old version and all OK. I tried the update twice with same result. Would not run more than a few minutes during live session before charts froze. 8.0.17.2 64-bit I am on now, seems reasonably stable and certainly does not have the issues the latest one does.

Just clinging to hope the new ver might help with performance, and once the gremlins are sorted with that, I might have a path forward with NT.

Thanks.