Chart lag (AGAIN)

[pjsmith]

Thanks for the hope, Bruce. I tried 19 again on the weekend. it seemed to be running. Within 30 seconds of live data on Monday, charts frooze, so I'm back to 18 again. No reported errors at all in .19. I do have a working solution for .18 now and I was too able to work through yesterday, but I really, really hope they fix up 19 so the rest of us can give it a whirl..

[brucerobinson]

#Alfred thanks - helpful to know

#All- Update. I've been plugging away & think I can now 'sign this off' for my purposes.

I worked away through all market conditions with my (pared down) 6 instrument workspace with no problems.

Today I traded using my 13 instrument workspace through FOMC release trading NQ with TickRefresh at 10FPS and no problems. No chart lag, no DOM lag, no error messages on entries, nothing.

Both workspaces are restored to where they were pre-18.1 which is stock platform plus 2 Ninja add-ons, plus the great #Hurleydood's TickRefresh. Medium time-frame, short time-frame and fast tick chart displayed plus others in tabs running in the background.

CPU and RAM is never troubled. It's never been PC performance. It's always been software coding IMHO. Now, with relatively simple charts for 6 or 13 instruments it performs as one should be able to expect.
The workspaces aren't overly complex, just fit for purpose for day trading the most commonly traded Futures.

The only problem/difference in operation between my 6 & 13 instrument workspaces is that I experience noticeable crosshair lag in 13. (I have different charts in total in the two workspaces because I am only displaying 6 instruments i.e. I haven't just cut out 7 instruments and left blank space where there charts were, but less in total in the 6 c/w 13).

I can work on the crosshairs 'refinement'.

I may go for 2 workspaces and a 'market watch'-type approach i.e. 2 groups of Futures charts and single small charts of the 'other workspace' in each.
I tried to do this with Alerts a lifetime ago but it was an un-mitigated disaster at the time. Again I suspect due to coding - adding an alert is fine, add any meaningful real-life trading situation number and loading time was horrendous and then sluggish performance.

Whatever way, I shouldn't need to change the setup, so hope to concentrate on trading, not platform problems.

Hope this helps.
Kind regards,

[Alfred]

Version 19 at 7:10am (Pacific) today....2 workspaces open...no lag on ES...

[pjsmith]

Today was a perfect example of the volatility that kills NT. I've a working solution for this now, in test for a few days and today was a great one - Today would have killed my NT normally. Now, I traded through it. Have you tried .19? I can't run it (it freezes for me), but elsewhere and in NT's overview, they reckoned a 20-30% possible rendering improvement. Might be worth trying it if you have not. Backup your db first though, because you can't go backwards...

[Lancer]

Chart lag screenshots minutes after PMI news event today.
ES, NQ, GC, and CL lagged (high trade rates). RTY and ZB (lower trade rates) lagged at first on news, but cleared quickly.
As of the time of this post, one hour after news, the ES lag has not yet cleared, now -1,100 sec. ES is always the worst.



NT v8.0.19.0, with 6 markets charted and visible, multiple tabbed timeframes with studies, on 4 monitors
Xeon 3.5 Ghz 6-core, not exceeding 40% utilization spikes by NT during chart lag event, leaving over 50% total CPU available
GPU not exceeding 10% utilization spikes during chart lag event
Lots of unused memory

[Alfred]

Alfred I think simpler works better for NT8. It's when you start messing with tick data and tick charts it all falls apart in volatility. If your method does not need that, advantage you

My main focus is on 2 to 4 seconds charts for patterns, graphics & volume ....along with a few longer term charts....

[pjsmith]

Alfred I think simpler works better for NT8. It's when you start messing with tick data and tick charts it all falls apart in volatility. If your method does not need that, advantage you

[Alfred]

Here is a screen shot from a 10/01 spike....no lag showing.
2 workspaces ...seconds charts and 3 indicators.....simple stuff....
My machine has an i3 and 12 MB of ram.

[pjsmith]

Interesting stuff, Bruce. Thanks for the detailed post. I can't run .19. It's unstable for me, but I have made a breakthrough in lag, in .18 over the last week - And it's not my code that has been at fault. Not trying to be a tease, but I'm not going to post any details until I have carefully tested it, and .19 might even make it irrelevant, but I have found (I think) what causes lag in my setup, and a workaround, which requires changes to a lot of NT's code. I'm been testing it in crude form yesterday, and slightly more refined today, and whilst the periods of volatility were there, and the lag observable, my platform adapted. where's before, it would have likely been stuck in that state until it caught up, which could be some time. I will test .19.x when the next version comes out, and my version until then.

Like you, I've spent 100's of hours on problems and workarounds for this platform. That's a lot of cost...

[brucerobinson]

Good news

I have been using R19 pre-release build because 18.1 trashed my stable 17 setup

My Workspace: stock platform plus a couple of Ninja add-ons. RTH & ETH Sessions r

ange highs and lows

for stock indices, CL, GC, NG and some currency Futures.

Daily, 15m, 5m, 1m etc.

Bare candlestick charts and Volume indicator, Current & Prior Day OHL indicator. That's it. Trading 101 setup,

I've attached a grab.

. Not an 'overly complex' workspace imho. The only non-Stock Platform add-ons are the range shading and Rollover Notification - both Ninja add-ons.

I use #Hurleydood's TickRefresh I removed it and both the non Stock Platform indicators - Barebones stock platform and R18.1 still crashed, lagged etc. where R17 didn't.

I can now report R19 pre-release worked for my setup but with some lag in fast markets, and now R19.0 final build without lag. In the first instance stock platform only, now having added back the two Ninja add-ons and TickRefresh. No errors, no lag, no crash with all functionality in all conditions. A few more days testing yet, but I'm close to saying even extreme volatility has no discernible effect on my setup whatsoever.

My workspace is now only 8 instruments as a result of trying to get R18.1 to work (i.e. I cut it back, but it didn't stop crashing), whereas stable R17 setup had 15 instruments.

Crosshairs remains an 'issue' - in my 8 instrument setup it is not, but in my 15 instrument setup there is perceptible lag in high volatility (#Hurleydood made a nice video demonstrating this). However in multi-monitor workspaces it always has been problematic since NT7. Any movement of Global Crosshairs takes GPU to 100%. This may be by design to maximise use of available resources, I dunno. I can work around it by using Global Crosshairs sparingly.

Quote #ChelseaB "

While I cannot comment about what has changed under the hood of NinjaTrader, I can say that in 8.0.18.1 and previous it was possible to initialize some objects in a thread that is not the ChartUI thread where rendering is done.(and from the Release Notes advisory a consequence of this could be poor performance and errors, crashing)

NinjaTrader has made changes to how rendering threads work and because of this is now enforcing that all objects related to the RenderTarget be created in OnRenderTargetChanged().

The short of it is. Initialize objects in OnRenderTargetChanged(). This would have avoided the performance hit to begin with and will avoid the D2D error." Unquote (itallics text added)

It appears to me that that the stock platform must have been guilty of this in R18 because it produced everything referred to - D2DF, DirectX, citing Paint, Brush, Draw, Object, OnRender etc etc etc & crashes galore.in my Setup - no 3rd party anything involved. Now fixed in R19 and with enforcement.

Releasing 18.1 (I even skip .0 releases so others can uncover the problems) with such issues cost me around 100 hrs and a lot of inconvenience with crashes, reporting them, taking off Ninja's own add-ons etc.. A disgrace.

I watched the presentation over on Futures.io by Brett Barratt on R19. Most was introducing new tools but a couple of things to give hope to those here - he stated the aforementioned changes in R19 had resulted in 20-30% improvements in performance reported by his team, depending on circumstances. I have no reason to doubt his word so that is encouraging to hear.

'Depending on circumstances', there has been questioning of what is fit for purpose for 'real world' setups (in this thread for e.g.) In the Q&A session, un-prompted Brett specifically raised and asked for feedback on multi-monitor/multi-instrument use of NT8 looking forward. Maybe the squeaking gates are being heard

(A minor correction - I do have a recurrent error in the log which is new, I believe, to R19 which I wonder if related to this topic OnRender InitializeIn etc. Indicator OnBarUpdate on bar 23674 (or whatever) calling thread can't access 'cos different thread owns it. I typically get a block of msgs all within a few seconds. They seem benign so not much bothered, but haven't seen before & nothing new (except R19). Just for completeness. Not wishing to hijack #OP's thread for this - I'll put in a ticket if I think it warrants it, but no impact I can see)

Update: same to report today, in extreme volatility things that would always have broken Ninjatrader in the past during high volatility (NT7 or 8) - move a fast chart around to cause re-rendering, change an interval, change days loaded, scroll, re-size axis etc all absolutely verboten with my setups, would have caused to either take forever, timeout, freeze or crash. Deliberately did all this on indices during Open and oil news & can't break it or make it lag.

Hope this helps.

Kind regards,

[brucerobinson]

Hello ChelseaB and thanks, I believe that may be helpful.

I'll follow up with more later

Kind regards,

[NinjaTrader_ChelseaB]

Hello brucerobinson,

While I cannot comment about what has changed under the hood of NinjaTrader, I can say that in 8.0.18.1 and previous it was possible to initialize some objects in a thread that is not the ChartUI thread where rendering is done.
NinjaTrader has made changes to how rendering threads work and because of this our development is now enforcing that all objects related to the RenderTarget be created in OnRenderTargetChanged() (or OnRender()).

The short of it is. Initialize objects in OnRenderTargetChanged(). This would have avoided the performance hit to begin with and will avoid the D2D error.

[brucerobinson]

Hello ChelseaB and thanks,

I believe you're referring to the errors specified by habibalex. I'm not a coder (an increasingly rare species around here) nor looking to write or debug anything (but thanks for the link - I learn something new every day, whether I want to or not),

I was referring to "

When working with D2DFactory to create Direct X resources, this must be done from the charts UI thread otherwise there will be a performance impact" - I've no doubt it is babyspeak to a developer, but it is (almost) impenetrable to me.

Simply put, to me you seem to be saying, correct me if I'm wrong, previous to R19 it was possible not to blahD2DFactoryblahDirectXblah.'..from the chart's UI interface', but not best practice to so so and could result in poor performance.
.
Changes in R19 mean if your were/are doing that, now doing so simply won't work rather than a performance hit and will produce errors. So take heed, you'd better stop your wayward practices and blahD2DFactoryblahDirectXblah.'..from the chart's UI interface from now on.

Correct?

Kind regards,

[NinjaTrader_ChelseaB]

Hello brucerobinson,

The error means that objects that attached to a render target are being created outside of OnRenderTargetChanged().
All objects involved with a render target or the (NinjaTrader.Core.Globals.)D2DFactory (for example when creating PathGeometry) should be initialized in OnRenderTargetChanged() or (OnRender()).

Below is a link to an example.
https://ninjatrader.com/support/foru...606#post789606

[brucerobinson]

#pjsmith
"

I am interested to know if anyone during Fridays spike suffered the dreaded chart lag, whilst using the new version of NT released the other day?"

I'm guessing you, #ChelseaB et al. here will be interested to know (thus giving you a Ninja hOpium fix) if during Friday's spike, using R.19 and similar real-world workspaces as described herein and no 3rd party indicators didn't suffer the dreaded chart lag (& likewise across the pond).

Me.

I'll elaborate & offer some further info/comments/detail FWIW a little later.

If anyone has technical understanding of the D2DFactory reference and could comment (in layman's language - I investigated MSDN etc but above my paygrade), it may help with transparency. Perhaps thinking along the lines of 'what if changes were made in 18.1 which uncovered poor design choices in the stock platform (no 3rd party indicators), throwing Log errors with gay abandon. Hence a major change in 19.0 that enforces better coding design now adopted in the stock platform, the poor design choices in the stock platform having been rectified, but will likely cause problems with others'. Just a hunch (with a lot of supporting information & rationale).

Best,