NT CQG Continuum - do your ChartTime results seem slower today?

[hedgeplay]

I am trying to figure out if this is just me or a system-wide change.

NT CQG Continuum - do your ChartTime results seem slower today?

Thanks!

////////////////////////////////////////////////////

On a strong machine and 400MB internet connection I have been seeing ChartTime results of .35-.5 second delays reported by ChartTime, even while using a fairly loaded up Workspace including Order Flow indicators.

Today even with a bare bones workspace of just the Control Center and one ES candlestick chart and ChartTime as the only indicator added I can get no better results than a 1.2 second delay.

NinjaTrader is set to priority High, is only using 500MB of RAM and only consuming 3-4% of CPU


No change in the 1.2-2 second delay after mutiple reboots and removing all other network devices from the router.
Speed Test and TraceRt show no change from last week with 37ms from the Chicago CQG Level 3 server location, low jitter and 330MB per second download report results.

[NinjaTrader_PatrickG]

Keep in mind that using the add on 'ChartTime' may in and of itself be introducing some kind of delay on top of whatever a root cause of delay may be. Also, depending on how this is programmed, there may be an inherent lag or inaccuracy. Can you attach the indicator for review?

[hedgeplay]

Thanks much for the reply Patrick.

I made no change to the presence or use of that indicator and the reported data delay increased by 120-200%. That being said, Yes, for sure on both points you make.

I believe I downloaded ChartTime from a post in this forum. I have two versions which really may just be the same and they give similar results.

In the image "Chart Lag Measurement" is on the left and ChartTime is on the right. The negative 1.2 seconds in Red implies a 1.2 second delta between the PC time and data being received or processed by the PC.

[hedgeplay]

An update. I manually resynced my MSI Laptop clock time three times with time.windows.com each time am now getting what look like better results.

Finding that the issue would be completely on my side would be great news, because I can then address it.

I noticed that the "Chart Lag Measurement" results in the chart on the left now show a delay of 30-200ms seconds. Given my zero-load latency to GQCs servers of 37ms I struggle to see how my chart data latency could be as low as 30ms so I makes me question the accuracy of that indicator. On fast and slower charts the ChartTime indicator which I have used more often is now showing 650-750ms delay which is higher but much closer to the 330-500ms delay I was seeing before.

So pending validation of slippage and percentage of missed fill expectations testing via sim I believe this issue is resolved.

[NinjaTrader_PatrickG]

The problem with these indicators is not necessarily added processing time, it's just not a very clean test.

The timestamp of the tick comes from the data provider, usually when they send it out. comparing the timestamp of the tick to the timestamp of the PC could have the following discrepancies:

1. PC clock time is not perfectly synchronized with the data feed
2. Latency between the PC and the data server
3. Time it takes NinjaTrader to receive the tick in the connection adapter and pump it to other threads that the script will process, then it comes out in OnMarketData/OnBarUpdate

Here are things to consider in order to mitigate the limitations of the add-ons you're using:

1. Can be tricky if we do not know what server the data provider syncs their clocks/time stamps their ticks with
2. Measuring latency, should be done by pinging the data provider's servers
3. There is no current way to measure this. NinjaTrader would need added functionality to measure time spent as data is received and processed.

Since your latency to CQG's servers is very low, I suspect the inherent inaccuracy caused by #1 and #3 above are contributing.

[hedgeplay]

Before I go... being unaware of a data delay can be the root cause of the loss of thousands on a single trade. As we can see above the the two attached indicators are a great contribution and helpful to highlight the delay but the difference in results calls into question their accuracy.

An official solution from NT would be a much better answer.

Please add a feature request or add my vote to an existing feature request for an official Data Feed Timeliness / Delay solution from NinjaTrader.

Thanks!

[NinjaTrader_PatrickG]

Thank you for the suggestion.

I have submitted this as a feature request to the Development Team. The internal tracking number for your feature request is SFT-3757. Please reference this internal tracking number when contacting Platform Support if you ever have questions regarding this feature request.

When a feature request is implemented, you'll find a description of the new feature in the release notes:

• http://ninjatrader.com/support/helpG...ease_notes.htm

[hedgeplay]

Thank you Much Patrick!

// The problem with these indicators is not necessarily added processing time, it's just not a very clean test.

Agreed. Still I think a good chunk of the problem could be addressed.

// The timestamp of the tick comes from the data provider, usually when they send it out. comparing the timestamp of the tick to the timestamp of the PC could have the following discrepancies:

1. PC clock time is not perfectly synchronized with the data feed
2. Latency between the PC and the data server
3. Time it takes NinjaTrader to receive the tick in the connection adapter and pump it to other threads that the script will process, then it comes out in OnMarketData/OnBarUpdate

///

#1 - Hopefully my PC was just off and #1 was my problem today.

Knowing what delay exists is still a primary need when market traffic picks up and delays start increasing on the charts and in the SuperDOM (just knowing what delay exists is far more important than knowing who's fault it is. The fault /causes and best fixes are the obvious followup questions but those questions are still far less important ( and can wait longer) than just being aware that a delay exists.



// Here are things to consider in order to mitigate the limitations of the add-ons you're using:

1. Can be tricky if we do not know what server the data provider syncs their clocks/time stamps their ticks with
2. Measuring latency, should be done by pinging the data provider's servers
3. There is no current way to measure this. NinjaTrader would need added functionality to measure time spent as data is received and processed.

// Since your latency to CQG's servers is very low, I suspect the inherent inaccuracy caused by #1 and #3 above are contributing.

On #2, Ping configured with large and odd sized packets is good tool to see inherent latency but Ping is often disabled. TraceRT can often still help id zero-load latency.



On #1, seems like need for an official clock to synchronize market data time stamps must have been resolved or the concept of ensuring National Best Bid would not work. So regardless of the clocks on servers sending data would it not be the exchange's time stamps applied to the data?

So would a metric or indicator measuring the Exchange's data time vs the PC's clock as the data became available for that metric to process would be good 'canary in the coal mine' warning tool that data is delayed, regardless of who is at fault (this is the primary need).


Seems like a default simple Ninjatrader Data Timeliness benchmark test would help to clarify to all when the data itself is delayed and wonderfully provide the visibility leads to better products, reliability and responsiveness from Data Vendors.

- A simple official NT Benchmark test like only One workspace is open, this specific NT provided workspace with just the Control Center and a chart with this chart NT provided Chart Template or measure attached to the SuperDOM (if the SuperDOM is the most NT optimized pipeline).

- The delta between this NT provided Data Latency Benchmark and what people are seeing in chart and strategy execution then becomes a loose measure of the delay that is the User's direct responsibility; delay added by that user's PC loading, connection, Ninjascript, number of symbols open, number and nature of indicators and strategies running, etc.

[hedgeplay]

Thanks for submitting the feature request and taking the time to engage in the conversation!

[brucerobinson]

a bit of in-use feedback, I've been using ChartLagTime for a while, it adds nothing to delay/load/overhead.
Polling timeserver frequency is key - 1second interval, for which you need to use a timeserver pool to poll different ones as a single one won't allow, think under attack and block/DOS/reject the request.
Always 'green' despite recent events for 'normal activities' i.e. not loading historical data or something.
Lots of fairly simple charts with lots of data series overly complicated workspace ;-)
Consequently now looking forward to maybe taking chart lag indicators off so I don't have third party ones to blame, but would be nice Platform Feature as a warning of problems in this regard