Make chart refresh rate faster than 250ms possible

[pjsmith]

I think a big thank you is in order... Is weird, because I have been using ChartControl.InvalidateVisual() for add and remove labels on keypresses and never thought this would force per your code. Thanks again for sharing...

[pjsmith]

Oh man, that is so much better!

[hurleydood]

No problem. Yes it's way better. I would have not switched to NT8 without this.

[Stefan2017]

Have not yet installed it, but already a very big "THANK YOU VERY MUCH!!" to you from my side!

Great you are sharing your indicator with us!

[pjsmith]

I have been using since my last post, set to 100ms/on price change, on my main entry chart only. Difference, especially with my price line on is stunning. Much, Much better for my purposes. Ninjatrader staff should be ashamed! Seriously. Once they try this (if any of them trade), they might realise what we've all been harping on about for the last few years...

[explorer101]

thanks for sharing this simple trick - was using in other parts of code for some other aspects - but this is a simple all in one short indicator workaround which affects the whole chart i presume
Will test it out ; these instances really show the contribution and collaboration of ideas when ninja falls short and its like crickets in here with zero ninjatrader input.
So a question even though posts below state that the InvalidateVisual() is queued and another onrender even setting this to 100ms is an improvement on the current 250ms?
Any confirmation appreciated
thanks

[hurleydood]

explorer101. I would say it makes a visual difference. See this video 100ms vs default https://youtu.be/0sb8OANbcyo

[brucerobinson]

Hurleydood (& anyone else ....)
Some clarifications please ahead of trying this, if you would be so kind....

If I understand correctly from your posts and others, TickRefresh works per chart so I can apply it as and only where desired to have a chart refresh faster than 250ms.

The common use case I imagine (at least in the context of this thread and per PJSmith's post) - I can put it on my fast/order entry charts only, set to (say) 100ms to get 0.1sec refresh on those charts only? (hence no unnecessary hit to PC performance by increasing refresh on slower charts where doing so adds nothing)

As an approach to deploying this across my trading setup, I might, for example, use it on an NQ or FDAX 15 tick or 5s chart set to 100 or even 50ms to assist during times of high market activity & volatility, but not use it at all on Bonds, or even ES where 250ms is adequate (in my experience and for my trading).

Depending upon what I find as a consequence of using it (any significant hit to performance as I increase the number of charts I use it on), I *might* use it to slow refresh in order to save resources e.g. slow the refresh to 1000ms on Minute charts?

Simply put, it seems to give me back what we had in NT7 - the ability to custom set chart refresh speed per chart, rather than the one-size-fits-most blunt instrument 250ms solution imposed on us in NT8. However we should use this cautiously and intelligently to avoid the problems the blunt instrument approach sought to protect users from.

Does anyone have any comment on the health-warnings NT give in Help on using this code in this way in the context of the above i.e. regarding potential to hit 'deadlocks'? (I don't speak much conehead, so it would be helpful for me to know what 'hitting a deadlock' will look like in practice). My setup is very stable now, with some occasional hiccups outstanding/being investigated but not with much progress. Hence I am reluctant to introduce anything new that might deflect from finally putting those to bed. I'm hoping that a 'common sense' approach of adding one instance of the indicator set to 100ms to a 'mid level' example (say YM 25tick chart) and watching CPU and GPU load at market open/news is the way to go, then if all well NQ, FDAX, RTY etc can follow, incrementally. And keeping an un-adulterated back-up of my setup such that I can simply revert.

What I am asking is what to look out for if problems occur, to as far as possible identify that it is this that has caused them, not something else - chart freeze, sluggishness until processing 'catches up' I guess? Anything else to confirm or eliminate, e.g. from Log and Trace file, or Log tab in Control Center

Many thanks for contributing and sharing. If it works well, two years to re-instate NT7 functionality in order for chart traders to properly benefit from NT8 CPU multi-threading/offloading to GPU, rather than go backwards

Kind regards,

[pjsmith]

brucerobinson Yes, you understand correctly. The change only applies to the specific chart you applied the indicator to. If NT were to re-introduce the feature that was in NT7, we could of course set our long term charts to a much longer interval than 250ms, and save resources there, too.

As you say, I think it needs to be applied with some restraint to prevent issues. I applied it to my entry charts only and as it is primarily price action I wanted it for (not volume indicators, etc.), I set it to on-price change. I think, using on-price change is the perfect efficiency option, because, unlike NT7 where it will blindly invalidate per the interval, we only update here, if price has actually moved.So, I'm getting an update every 100ms, but only if price has changed. I changed it to 50ms now, just to see, but I noticed no difference in performance of my platform at all.

The difference in chart behaviour is stunning though and this single change has prompted me to reconsider migrating to something else.

In terms of if it will make your platform unstable. I've been running the same method used in this indicator for a couple years in my own order entry logic, without realising! I never put 2+2 together, else I would have coded this long ago! I have never had issue with this call. Obviously, it's being used far more frequently here, but I was running it and trading live since my post, all day the remaining days of the week for the full eu and us sessions, and I had no issues at all, just lovely printing bars, one at a time in fast moves, rather than a 'where did those 5 bars come from' type update.

I would guess the only issue is going to come with the number of visible drawing objects on the chart at the time the chart is refreshed. My charts have a lot of algo type marks on them, so maybe worse than some, but still, I noticed no issues. Entry charts are likely going to be covering small time frames, lessening this issue anyway.

I note the author has already removed some of the redundant code and made it more efficient. I'd already done some tinkering in that respect too. In it's latest form, it looks good to me. If I notice any issues, I will post them here.

[brucerobinson]

Thanks pjsmith for comprehensive reply.

"If NT were to re-introduce the feature that was in NT7, we could of course set our long term charts to a much longer interval than 250ms, and save resources there, too."
To be clear - if we found the need, to offset/free up resources consumed by usińg 100ms, we could use this indicator also for the purpose you describe (no need to answer if this is so - basically the availability of this allows us to tinker any and all charts faster and slower as NT7 before to optimise to individual requirements).

"On price change" - yes indeed, me too, a no-brainer in resource efficiency terms

Platform stability experience with it, drawing objects etc - thanks for the feedback, it is re-assuring in order to move forward

I'm bordering on being excited to give it a go!

I'll post my progress, warts and all if any.
Best,

[pjsmith]

For info, I'm not sure we can use this indicator to save resources on charts that do not need refresh very often. As far as I understand it, it does not overwrite the inbuilt 250ms timer, so that will run regardless, if there are chart updates to be made. I;m not expecting to have to go to these lengths anyway, to be honest.

[brucerobinson]

Ah, I see what you mean, longer refresh in the indicator won't over-ride the hard-coded 250ms. Good point well made. Thanks for taking the time.

Likewise, I'm hoping & expecting we can simply tinker with this only with the charts that will benefit, without bringing things to a standstill or anything close.

If it did, i'd probably take that as sufficient reason to cease and desist at this point for now.

Thanks

[hurleydood]

If I understand correctly from your posts and others, TickRefresh works per chart so I can apply it as and only where desired to have a chart refresh faster than 250ms.

Yes it works per chart and it works with tabs. Inactive tabs or minimized windows will suspend the refreshes for those charts. This indicator is only designed to have a faster refresh rate and eliminate the 250ms delay.

The common use case I imagine (at least in the context of this thread and per PJSmith's post) - I can put it on my fast/order entry charts only, set to (say) 100ms to get 0.1sec refresh on those charts only? (hence no unnecessary hit to PC performance by increasing refresh on slower charts where doing so adds nothing)

You can put it on all charts. You will have to add the indicator to each chart you want faster updates. The indicator does not use a timer to determine but it measures the time between incoming ticks and if that time is greater than the refresh time interval it will refresh.

As an approach to deploying this across my trading setup, I might, for example, use it on an NQ or FDAX 15 tick or 5s chart set to 100 or even 50ms to assist during times of high market activity & volatility, but not use it at all on Bonds, or even ES where 250ms is adequate (in my experience and for my trading).

Depending upon what I find as a consequence of using it (any significant hit to performance as I increase the number of charts I use it on), I *might* use it to slow refresh in order to save resources e.g. slow the refresh to 1000ms on Minute charts?

The indicators goal is only to produce a higher frame rate and not actually save resources. I do not recommend this for refresh intervals > 100ms. Also if there's no new ticks or price changes over a 250ms time period then NT8 will automatically check with its own timed 250ms interval if the chart needs to be updated.

Simply put, it seems to give me back what we had in NT7 - the ability to custom set chart refresh speed per chart, rather than the one-size-fits-most blunt instrument 250ms solution imposed on us in NT8. However we should use this cautiously and intelligently to avoid the problems the blunt instrument approach sought to protect users from.

My approach is pretty intelligent. It can update on each tick or on each price change. And it will only refresh if resources are available. With a refresh time interval set it will do a refresh if the difference between the ticks is greater than the refresh interval. If the refresh time interval is 0, on each tick or on price change it will try to refresh if it's not busy already refreshing.

Does anyone have any comment on the health-warnings NT give in Help on using this code in this way in the context of the above i.e. regarding potential to hit 'deadlocks'? (I don't speak much conehead, so it would be helpful for me to know what 'hitting a deadlock' will look like in practice). My setup is very stable now, with some occasional hiccups outstanding/being investigated but not with much progress. Hence I am reluctant to introduce anything new that might deflect from finally putting those to bed. I'm hoping that a 'common sense' approach of adding one instance of the indicator set to 100ms to a 'mid level' example (say YM 25tick chart) and watching CPU and GPU load at market open/news is the way to go, then if all well NQ, FDAX, RTY etc can follow, incrementally. And keeping an un-adulterated back-up of my setup such that I can simply revert.

What I am asking is what to look out for if problems occur, to as far as possible identify that it is this that has caused them, not something else - chart freeze, sluggishness until processing 'catches up' I guess? Anything else to confirm or eliminate, e.g. from Log and Trace file, or Log tab in Control Center

The NinjaTrader staff does not recommend this indicator due to potential deadlocks. I've seen a deadlock in an earlier version. For example, if it takes 5ms to render a chart and you receive more than 200 ticks in one second the chart will start to show a delay until it "catches up" as it's trying to render 200 frames. It was only noticeable in the initial versions at the 4PM ES close with calculation on each tick and refresh interval of 0. New improved code will not force a refresh if the chart is already busy refreshing. Here is a video testing the yesterdays ES close 3/29/2019. NT8 was able to keep up with 125k contracts at 4PM without delay.

https://youtu.be/09UuFqYqcJU

It also works great in Market Replay mode if you want to test out your set of indicators. Check out this video with Market Replay at maximum speed.

https://youtu.be/O2bThPJBguU

I ran some analytics and a new version will be out next week with improvements.

[brucerobinson]

I didn't see Hurleydood's reply until now. Despite being Subscribed sometimes I don't seem to get Notifications.

Firstly a big thanks for taking the time to give the comprehensive answers to my wonderings - whilst I don't know much about what goes on under the hood, it is comprehensible and all makes sense, seems very well thought through/out and hence re-assuring.

By way of my contribution - to report back, I've been using the indicator on all my 'fast' (typically 15-50 tick) charts on faster instruments (usual suspects, FDAX, US Indices, CL etc) running 13hrs most days, for a week. So circa 600 'instrument hours', news events, Closes etc)

No discernible performance hit, at any time (I have a CPU LCD permanent read-out).

No discernable deadlocks in the manner you helpfully expanded upon for me, e.g. slowdown/catchup behaviour.

I have noticed a few 'Error in realtime market data handling....' type errors in the log. I mention this in no way 'casting aspersions' that the indicator is responsible - I don't usually look for log errors unless I am experiencing a problem or trying something new.. It is not to say that similar aren't there otherwise, nor that it is a problem, as such. It may be 'normal behaviour'/to be expected with or without this indicator applied. I only mention because you/others may, or may not, find it useful to know and/or know with certainty such errors are nothing to do with 'what this does and how it does it', (so don't be offended! it's simply above my pay-grade to know).

I'll keep a watching brief and report again with more ticks under the belt


Current status/impressions - bloody marvellous, not from a geeky perspective that it can be done, but from a trading perspective seeing price on a chart next to the DOM moving in glorious synchronised harmony. Heavenly.

Kind regards,

[hurleydood]

Thanks for your input Bruce. I released a new version last week, please give it a test. I haven't seen the error you mentioned myself but I'll keep a look out. Can you check your NT8 Log directory to see if the error has been seen prior to the indicator been used?

Yes it's great to see the DOM synchronized in harmony with the chart. I trade primarily from the DOM but I can easily trade from the chart as well now. You can actually see realtime price action on the chart now.