Fast market and slow NT

[brucerobinson]

#Lancer thanks.

This has the makings of a whole new rabbit hole , and there was me thinking after 2 years of this with R19.1 I may finally be out of the woods.

Your reply - all understood & helpful (get the feeling you've had this one in your crosshairs for some time)

PC clock, yes - I did sync at the outset, just had no idea how quickly it drifted. Was expecting/relying on Win10 timesync on Startup to be good enough for the day. Have a toy for that, now set to sync hourly.

Today - typically sub -100ms. So the drift is put to bed, it's not lag as such, just cumulative clock slip. And the 'normal' lag in the low hundreds ms when sync'd is not a problem (for me).

But earlier today I saw (with your script, may not have without it) what you and others have reported - for no obvious to me reason, a lag building up to several seconds, maybe 20 then decaying. Charts nor platform freezing, so not easy to know that it was not just the market slowing. Once 'out of the way', normal service resumed back down to <~ -100ms and continued since, through the Open without reaching 1s.

And then again, 20s lag. I think it may have coincided with large spikes in volume on both CL & GC.at Inventories news. However, when it did it this (my) am, I didn't notice any market activity coinciding.(hence wondering if my internet connection, but looking less likely)

I don't have any multi-timeframe indicators, not sure what you mean by multi-indicator.

All I have is Current and Prior OHL (multiple instances - one for each Session), Priceline and VOL stock platform, a Ninja add-on that draws Sessions zones, and #Hurleydood's TickRefresh.the only non-Ninja. Simple.

Hmmm. Curiouser and curiouser.

[Lancer]

Bruce, with your 16-core (32 thread) processor running that simple workspace you describe, and assuming your video card GPU % utilization or memory is not somehow restricting, you should have absolutely zero chart lag. That CPU/workspace combo is anything but unoptimized or overly complex. But, the occasional chart lag behavior you describe and are able to see now with ChartLagTime monitoring is the classic lag behavior. My workspace is more complex and my CPU is only a 6-core (12 thread), so I see lag more often and for longer duration. But still, during fast markets when charts can lag, CPU utilization by NT is only 35% max, GPU % is 4% max, all with plenty of memory available, so with all the unused PC resources I don't get it. The lag behavior is almost like there is some kind of processor governing or chart data buffering going on resulting in NT chart data processing at some max limit rate regardless of available unused PC resources that could otherwise be applied to the task. Can't imagine that being the case, but that's how it presents.

By multi-indicator I mean a custom script calling two or more standard NT system indicators, or containing that code. Those and multi-timeframe scripts seem to be the bigger loads among chart scripts.

[brucerobinson]

#Lancer thanks for elaborating.

"The lag behavior is almost like there is some kind of processor governing or chart data buffering going on resulting in NT chart data processing at some max limit rate regardless of available unused PC resources that could otherwise be applied to the task"
In one of the threads (probably one you linked to herein), #pjsmith describes his understanding of Ninja processing every tick sequentially 'through the gate one after another', and PatrickG stepped in to confirm.

#pjsmith's conjecture being that it is how, and the speed at which, Ninja's code is able to process an avalanche of ticks which is the bottleneck in such situations when everything else is over-provisioned (hardware resources, multithreading, speed of connection etc). With everything else addressed wrt overprovisioning on threading, memory, GPU etc it seems the only things remaining are speed at which data transmits to my pc (internet connection), CPU raw clock speed and code speed. My PC benchmarks in the top 2 percentile worldwide for CPU speed, so there is nowhere left to go in that respect.

What I have no insight into, is that my setup can handle, for example, the NY Open, or a Donald China tariffs tweet, when I have multiple Instruments that are sensitive to such events active and they all go ballistic without chart lag, yet at other seemingly more benign events it is seeming to back up on processing throughput, yet availability of resources are not the bottleneck.

I observe that volume is different i.e. CL & GC at news swamp Indices. But it's just an anecdotal observation as yet.

Thanks for your help with this, it has provided some transparency to what is taking place. Not yet decisively why...

I don't have any custom scripts (other than those mentioned) - stock platform only was a strategic decision at the outset of migration from 7 to 8.

I'll keep posting any new findings here.

Kind regards,
Bruce