Make chart refresh rate faster than 250ms possible

[koganam]

Thinking about this, does this mean that OnBarUpdate will be called at most 4 times per second? That's not good for algo/strategy traders for fast-moving instruments.

No. It is not a processing issue: it is a "chart rendering" issue.

The data is processed as it comes in: watching the chart is, effectively, delayed/gated. In other words, watching the chart is not quite WYSIWYG.

[biegea]

Hi everyone,
After years of being a Ninja customer, I've spent hours this week dealing with lag and semi-related issues and troubleshooting. Reading through this thread and others like it make me feel better to not be alone in this issue. Rather than write many paragraphs describing all of my observations and troubleshooting, my powerful computer that is never taxed, yadda yadda... I will try to just come to the scenario/question:

This morning I went into the Open on the MNQ contract in many charts, and I had lag grow from the open up to 6 min over the first 20 or so minutes of open market hours. That lag that didn't fully resolve until 45 minutes after the open.
I have ChartLagTimeV3 on a few charts. On two of those charts, I have TickRefresh (interval set to 100ms -- on 15 sec and 1 min charts) as well. It's hard to prove, but it seems like having TickRefresh on the two charts starts to see much bigger lag buildup once the volume starts flowing than when I just have it on one chart. That is - I believe I don't see the issue when I only have TickRefresh on 1 chart. But when I have it on two, it seems almost certain that I will get up to at least 3-6 sec lag for some period of time -- and possibly a lot more or for longer periods -- when heavy volume is in play (not crazy news volume but just regular day high volume periods, e.g. periodically in first and last half hour and during occasional runs intraday). I could go into much more detail, but I thought I would start here. My questions are:
-Has anyone observed similar issues when using TickRefresh beyond a certain number of charts?
-Is the any combination of settings or other tips around using ChartLag and TickRefresh that is recommended. (Is it extra taxing to run TickRefresh and ChartLag on the same chart? Should 'Calculate' on either/both be 'on each tick' or something else? etc.)

I've been in touch with Ninja and they gave me a list of items to review but they are not in specific recognition my need to improve my refresh rate on at least some charts, so I'm hoping someone out there who has troubleshooted through similar experiences might be able to help.

Thank you in advance!
biegea

[NinjaTrader_PatrickG]

biegea , I was able to locate your ticket in our system (3279679) speaking with Heath. He provided you with a specific set of steps to take based on the symptoms you described and the information contained in your log and trace files. While I encourage discussion among users on similar experiences, there is literally no better expert to be speaking to than our Platform Support Team in order to diagnose and mitigate or resolve the issue.

Based on the information we currently have from you and what we see in your files, you're experiencing what we would call 'chart rendering delay'. Lowering the refresh rate of your chart, which this topic is discussing, would actually exacerbate that symptom.

Please follow the steps Heath recommended and follow up with him via email so that the conversation may continue.

[biegea]

Thanks, Patrick - I am working through those steps and will report back to Heath. But while you will tell me to consider that my use of TickRefresh may be exacerbating my performance problem, I'm reaching out on this thread to others who understand it's not an option to NOT increase my refresh rate. I need to be able to do that AND keep lag at bay. I simply can't trade my market with a combination of slow refresh rates, lag, and a close-trade button that takes a solid second to execute (a separate issue I opened a separate case around, to which the response was 'try replacing your DB file' even though I just did that earlier this month for a different and crash-related issue -- and yes, despite this, I'm going to try that as well).
In any case, I will continue to address all angles from the list Heath provided as well and will continue to work with him.
Thank you,
biegea

[brucerobinson]

Hello #biegea,
I have experienced the same problems you are encountering.

In my case it was not a chart rendering issue, it was a limitation of the platform's capabilities which Support failed to resolve, after wasting quite literally hundreds of hours of my time so I've felt your pain. I identified the issue and found a solution myself. I now experience near zero lag (little more than ping latency)

Experience has led me to disagree that there is 'literally no better expert to be speaking to than our Platform Support Team in order to diagnose and mitigate or resolve the issue.'. It certainly was not the case in resolving my lag problem. I'd say you are wise to seek input from other Users, in addition to Support's.

"My questions are:
-Has anyone observed similar issues when using TickRefresh beyond a certain number of charts?
FWIW I had similar problems on NQ at the Open and other high volatility times causing lag as you describe i.e. sometimes short duration, sometimes 'cumulative' The problem was not chart rendering. I can answer the opposite of your question - I can and do use TickRefresh on multiple charts without issue. I wouldn't disagree with Patrick's response that if lag is is due to chart rendering problems, increasing the refresh rate and the number of charts you do this with will exacerbate the problem

-Is the any combination of settings or other tips around using ChartLag and TickRefresh that is recommended. (Is it extra taxing to run TickRefresh and ChartLag on the same chart? Should 'Calculate' on either/both be 'on each tick' or something else? etc.)"
I don't know the answer to this (understand where you're coming from - are there benefits to them being on different charts in processing terms?, I don't have software development expertise but can perhaps give useful experience) - I use ChartLag and TickRefresh (100ms) on one NQ fast tick chart with update on each tick without problems. My opinion is that I could use both on as many charts as I wish without problems (I have observed on UtilMon and they are 'light'. Although I guess that's not the point with TickRefresh, it's the chart rendering)
Again I can answer the opposite of the question - I can and do use multiple instances of ChartLag and TickRefresh (100ms) each OnEachTick without problems

So I can't tell you anything helpful about why you're hitting the problem, but I can tell you my similar experience was due to reaching limitations of platform performance, and having identified and worked around those I never see any lag of any sort whatsoever (daily for over a year) whilst using multiple instances of chart lag and tickrefresh. I trade multiple instruments with multiple charts in multiple workspaces. NQ was the most problematic of those I trade due to its volatility. The only time I experience lag mow is if due to poor quality internet connection.

I'm not aware of an issue with the Close button but I rarely use it as it is not how I trade (use/move Stops/Targets on the DOM without a problem - just mention in case a viable workaround you could use that I know is reliable). However, I do have experience with 'slow DOM' issues. It is very rare that I alter anything during trading hours but I did have occasion to Edit an ATM Strategy this week which I foolishly did. Slowed to snail's pace, locked up, couldn't close windows via task manager, hot crashed it, opened up all ATM Strategies missing in DOMs, templates gone in some charts/no indicators. Over the years I've learnt not to do that sort of thing.

Hope this helps & gives you hope! I wonder if it might be worth starting a new thread on your setup & problem, along with your post above and email communications with Support to encourage others to chime in. As you'll know there are several experienced Users who lurk and are usually happy to help if they see something they feel they can contribute positively to.

Best,

[hurleydood]

I don't believe its a chart rendering issue.I developed TickRefresh based on this idea below and the observation that charts were very responsive to OnRender() while viewing lagged or delayed data. "The OnRender() method frequently runs once the State has reached State.Realtime in response to market data updates or a user interacting with the chart (e.g., clicking, resizing, rescaling, etc.)" https://ninjatrader.com/support/help...l?onrender.htm

OnRender() is capable of running 100s of times a second without delay. You can test this and your chart's performance with your indicator set by simply dragging the price or time scale which forces a refresh in real-time. This is why I don't believe it's a chart rendering issue and related to how market data is received and processed on every tick. When a large burst of data is received data processing starts to fall behind.

[Rogue_Two]

Hi everyone,
After years of being a Ninja customer, I've spent hours this week dealing with lag and semi-related issues and troubleshooting. Reading through this thread and others like it make me feel better to not be alone in this issue. Rather than write many paragraphs describing all of my observations and troubleshooting, my powerful computer that is never taxed, yadda yadda... I will try to just come to the scenario/question:

This morning I went into the Open on the MNQ contract in many charts, and I had lag grow from the open up to 6 min over the first 20 or so minutes of open market hours. That lag that didn't fully resolve until 45 minutes after the open.
I have ChartLagTimeV3 on a few charts. On two of those charts, I have TickRefresh (interval set to 100ms -- on 15 sec and 1 min charts) as well. It's hard to prove, but it seems like having TickRefresh on the two charts starts to see much bigger lag buildup once the volume starts flowing than when I just have it on one chart. That is - I believe I don't see the issue when I only have TickRefresh on 1 chart. But when I have it on two, it seems almost certain that I will get up to at least 3-6 sec lag for some period of time -- and possibly a lot more or for longer periods -- when heavy volume is in play (not crazy news volume but just regular day high volume periods, e.g. periodically in first and last half hour and during occasional runs intraday). I could go into much more detail, but I thought I would start here. My questions are:
-Has anyone observed similar issues when using TickRefresh beyond a certain number of charts?
-Is the any combination of settings or other tips around using ChartLag and TickRefresh that is recommended. (Is it extra taxing to run TickRefresh and ChartLag on the same chart? Should 'Calculate' on either/both be 'on each tick' or something else? etc.)

I've been in touch with Ninja and they gave me a list of items to review but they are not in specific recognition my need to improve my refresh rate on at least some charts, so I'm hoping someone out there who has troubleshooted through similar experiences might be able to help.

Thank you in advance!
biegea

After much frustration I have concluded TickRefresh is clogging up my Volumetric charts during times of high volume. I wish the issue of a 250ms chart delay in Ninjatrader 8 would be addressed.

[marktrading]

Are you sure about this and that your pc or video card can keep up? There is a crazy amount of data coming in a high volatility and the bottleneck is probably elsewhere than the chart rendering.
It works pretty well for me and in my opinion going over 100ms is pointless as faster than your reaction time anyway..

[JustinaM]

Not much to be added, but the lag is insane and almost makes one think it's on purpose someone benefits from poor execution...why else have the chart show you one thing so when you place the order it's not even close.

[marktrading]

I don't agree and the plugin improves ninjatrader a lot and is free. IT IS NOT 250ms the difference with being profitable or not.

[BGWTEX]

Greetings all,

This thread & the included AddOn, “TickRefresh”, are outstanding, as are the main posters, Bruce Robinson (OP), “hurleydood” (wrote “TickRefresh”) and “pjsmith”, thanks to y'all for sharing.

“TickRefresh” : https://ninjatraderecosystem.com/use...d/tickrefresh/

This thread explains & “TickRefresh” resolves a problem (explained in topic 1) which I wasn't aware of. I've not tried “TickRefresh”, yet, I will shortly (i.e., I've no experience).

Some posters have asked about/for “TickRefresh”, thinking it a FIX to lagging. Some reading this thread may be confused by these posts, and the combination of topics, as I was for a long time. I'd like to try to this clear up.

There are basically 2 topics :

1.) Frequency of Charts Drawing. Up to about Post# 93.
The original topic is the frequency of charts drawing. In NinjaTtrader 8 (NT8), this is hard-coded (can't be adjusted) at 250ms, 4 fps (frames/sec), it is user changeable in NT7. For comparison, for games, the minimum is 60 fps, 120 fps is preferred. Silent films were filmed at 18 fps, a bit "jumpy".

I'm still a bit fuzzy on this part, however, I'll take my best shot. NT8 displays a “picture” 4 times/second. If the price is the same at the ¼ second intervals, price doesn't change on the chart. If the price changes above/below in between these ¼ second intervals, these price changes are NOT on the chart. Increasing the fps catches, at least some of, the price changes and are on the chart.

This is important to some daytraders who use this detailed price action to make decisions.
Increasing fps INCREASES resource usage.
Time to react is a totally separate issue.

2.) Lag. After Post# 93.
NT8 will for some users, mostly daytraders I believe, can fall behind handling incoming data. Sometimes seconds/minutes/hours & even freezing NT.

These topics are separate from each other. If there's a “Lag” problem, increasing the fps makes it worse.

I hope this helps someone, and any mistakes, information left out, anything to add, comments, anything, let's hear it.
Nobody's gonna make me cry.

Adios compas,
Big Tex

[revplant]

I have used charting platforms since the beginning of electronic trading, and I don't recall even the first generation designed in this way.

[morpheus0]

Is there anyway to get onrender to be called _less_ ? As in maybe once or twice every minute ? Similar to bar update? As stated in the documentation some elements maybe static for a while. Yet the drawing capabilities of onRender are far superior than drawing everything on screen.

I tried reducing the amount of times it needed to be called to say 10 times a second and now objects started disappearing on screen. Anyway to keep them static until the next redraw?