Please make the chart refresh rate settable

[NinjaTrader_Brett]

Thanks Bruce. I read your feedback, its appreciated. I'll review it internally.

[brucerobinson]

For those interested in the likely potential to run multiple charts, maintaining higher refresh speeds, multiple instruments etc, - on the Future.io site in response to Ray Deux's Ask Me Anything on NT8, a User reports running 4-5 charts each (which must all be at 250ms) on 9-11 instruments simultaneously with Jigsaw tools enabled also, which is encouraging (and in which case why limit to 250ms fastest? Suggests most with fewer charts/instruments wouldn't see a problem)

[Trader_55]

Trader55,

Would you have a quick moment to write me at platformsupport at ninjatrader dot com ATTN: Brett. I'd like to get your workspace and the name of the connection technology you use. If you don't use any third parties as you mention then I'd like to run your workspace on some test machines here. Since I'd like to see what you mean by high CPU / unusable on that setup and furthermore if/as I confirm that setup does have performance impact I'd like to run tests to figure out what that exactly is. Since it may be related to refresh rate, it may not be.

Some background info if its relevant for you:

- We believe in simplicity over complexity.
- We believe that the majority of users should not have to play with such 'complex' settings...and in principle it should just "work".
- We get countless support tickets in NinjaTrader 7 from users experiencing bad performance in NinjaTrader 7 only to see they've adjusted the refresh rate down on charts. Users looking for 'faster charts' instinctively reduce the number not realizing they actually increase CPU load and hence actually make performance worse. (We even put a modal dialog warning when setting it lower but found users just ignore the dialog)
- Hence there is a strong feeling that the exposing of the property in NinjaTrader 7 overall made us worse off then it helped us. Hopefully you can see the 'conundrum'.

I'm just sharing in case it was helpful for you to see the other side of the coin. Though the fact still remains you report bad performance in your workspace you use on a daily basis. This is what I want to research and look into first. We then could decide if the feature of settable chart refresh is the cause or not.

Thanks Brett. There are some problems with your response here. Please go back and read my original post... I asked you to allow us to be able to make the chart refresh rate SLOWER, not faster. So basically, all you did above is explain to me why you don't want the refresh rate faster than 250 ms. Great, thanks. Can you now explain to me exactly why we can't change it to 1000 ms, or 10000 ms? How can that possibly be a bad thing? You won't be getting any more support requests if you do this, you will be getting LESS. You should make the refresh rate changeable, leave the minimum at 250 ms if you want, and make the default 1000 ms for daily charts, and 2000 ms for weekly and monthly charts, as I suggested. That will actually give people a better impression of your software "out of the box", as I stated in another post.

As for people who set the refresh rate too fast in NT7 even after being warned, you can always make the warning big flashing yellow letters or something. Something like this:

WARNING: SETTING THE REFRESH RATE FASTER THAN "X" MAY RESULT IN PERFORMANCE ISSUES IF YOUR SYSTEM IS NOT FAST ENOUGH TO HANDLE IT. ARE YOU __SURE__ YOU WANT TO PROCEED?

I run NT7 and I don't see this kind of very obvious popup whenever I adjust the refresh rate, it just lets me do it with no popup. So I hardly think you have done everything you can do to warn people in the code.

Anyways, if you get support requests even after a very obvious popup like this, all that tells me is that some people on this planet are too stupid to tie their own shoelaces without a friggin' instruction manual. What can I say. While I can sympathize with you not wanting to handle the support requests from these idiots, limiting all the rest of your customers just because some people are stupid morons and you don't feel like devoting customer service resources to support your product is not acceptable from a customer service point of view, at least to me. You can have a pre-written longer message post on this forum laid out explaining to these idiots what the problem is, and just forward it to them whenever one of these numbskulls contacts you. It literally takes two seconds to do that. I doubt you should get more than a few of these problems per day at most (if that), and each one shouldn't take too long to just direct people to a thread discussing it. I am honestly amazed at how stupid some people are, if you are really getting a very high volume of questions about this, but in any case you have not done everything you can in the code to warn people, and I know because I am using NT 7 every day and it's not THAT obvious because there is no popup.

In any case, back to my main point: I wanted a SLOWER refresh rate, so nothing that you said makes any sense in the context of why you are locking every chart in at 250ms. If you want to keep the minimum there, fine. I don't care personally, and I'm not going to argue the point. But I need a slower refresh rate on many charts, and there is absolutely no reason whatsoever why you cannot do this immediately.

As for my workspace, there is no point. I literally just open up a few dozen charts with nothing at all on them, and it's still using more CPU than NT7. I am switching over to run an extra copy of NT7 in a virtual machine to deal with this. This is not my workspace which is an issue, this is an issue with NT8 being programmed inefficiently,and has been the case across multiple workspaces I have tried since the Beta program began. Open up a number of bare charts in NT7 and NT8 and just compare the CPU usage if you want, but in any case for the moment I am moving on to running an extra copy of NT7 until you guys can make the program more efficient. Thanks!

[Trader_55]

2) Chart refresh we want to update 'slower'. This suggestion is coming in since performance is not in acceptable range with a setup and we're looking for a solution to improve performance. I want to understand why performance is bad in the first place before moving forward which is the point I was trying to get too in my post.

Thanks.

I just saw this post and wanted to make a point. Has it occurred to you that people may want to run a lot of charts in some setups? Sometimes the reason it is "slow" is just because there are more charts open, and there's nothing you can do about that. CPU resources are finite, not infinite. Each PC may also have other things running on it. Your post is implicitly assuming that 250 ms refresh rate is "just fine" and that each person must be having some individual problems just because of their setup. It's NOT fine. And we are not all novices here, I know that it's not a problem with my setup because it is super-simple and I have nothing on my charts. Whatever the performance of NT8 is ("good" or "bad"), the point is that 250 ms refresh rate means we can have less charts open than if we had a slower refresh rate - period. How is that a smart thing to build into your product? Refreshing weekly and monthly charts at 250 ms refresh rate is just plain wasteful of CPU resources.

[NinjaTrader_Brett]

- It never is expected for CPU to be the same as NinjaTrader 7 with NinjaTrader 8. The core of NinjaTrader 8 is fundamentally different. Specifically we're multi threaded on the UI and to take advantage of all CPU cores on a PC and there is some inherent overhead to passing data between threads which increases base CPU consumption but unlocks the CPU potential of all cores. (Going multi-threaded is not free)

- Please see the attached screenshots, I went ahead and did a quick in house test of the following scenarios on an older 4 year old machine.

(50) 1 Minute ES 09-17 Charts, Not Connected, Idle PC:

https://www.screencast.com/t/A2aQYhWoCnu

(50) 1 Minute ES 09-17 Charts, Connected, Right at ES market open.

https://www.screencast.com/t/BblJnTskEf

- We have more complicated internal test setups but in this basic base test you should be able to see the refresh rate on 50 base charts makes minimal impact to the CPU. Hence I'm growing concerned that we actually have a performance problem with a specific feature you're utilizing as part of rendering that needs to be understood and optimized thats impacting you. Which is the sole basis for my workspace request.

- We used and will continue to use evidence based practices to decide on performance tuning of the product and features and functionality related to performance. I simply need to collect the actual data to look into this further.

Note to those of you that might have an interest in 'faster' chart update speed. Your feedback is not lost in all of this. This post is specific to Trader 55's request and post for the option of slower chart update speeds.

[brucerobinson]

Hello Brett,
CPU Utilization is 10% unconnected and 20% connected? I must be misreading something?
If 250ms refresh speed is in fact 0 CPU utilization impact in your test, expect 100ms likewise.
ES 1 minute is popular, but it is a slow instrument. Suggest could you take a look at NQ on 25 tick at the open today.
Kind regards
Bruce

[jcrza]

Frankly, you're wasting your time. They don't care that NT8 performs like crap compared to every other charting program. I'm not paying for it anymore next renewal. They always have excuses but no real fixes.

[hurleydood]

Why is there even any CPU utilization at all when disconnected? Can't you refresh the screen based on receiving data or changes in price. Must the the entire window require a fresh? I have 20 charts open on NT7 and its using 0.15% CPU when disconnected and during an active market 5-10% CPU and the chart update interval is set to 0.1 seconds. NT7 is a single CPU app and runs really well. I get NT8 uses the GPU and multiple cores of a CPU but in doing that the performance should actually be better and even reduce CPU load. But NT8 struggles just to drag a cross-hair across the screen. My system has a hard time running NT8 but my PC can run 3D multiplayer video games at 100 fps. It doesn't make sense. Java based applications like Thinkorswim or Etrade Pro run way better. Even web based trading platforms like TradingView or Tradovate run great. Like jcrza said. "You're wasting your time. They don't care that NT8 performs like crap compared to every other charting program." NT8 is broken by design. Nothing can be done to fix it. The customer service response will always be send me your log files.

[sledge]

Why is there even any CPU utilization at all when disconnected? \.

Do you have crosshairs on by chance?

[NinjaTrader_Brett]

In my screenshot or on your own setup? If my screenshot then this is just the background operation of the PC without NinjaTrader even running, as I would close NinjaTrader down it would have the same base CPU utilization. NinjaTrader 8 does not in general spend any CPU cycles repainting charts if disconnected same as NinjaTrader 7. Unless you have active re-painting ongoing like crosshairs as sledge and your post mention.

If your own setup we would be happy to setup a support session with you to identify the cause. Let us know at platformsupport at ninjatrader dot com.

[hurleydood]

Do you have crosshairs on by chance?

On NT8, never. Too high latency between mouse movement and to actual cursor location. I've suggested they draw the hardware cursor.

My comment was referring to support having a bunch of NT8 windows open while disconnected and having still having 10% CPU usage for no apparent reason.

[brucerobinson]

Cursor X-hairs, other findings and a Challenge...

Hello Brett,
You haven't responded directly to issues I raised earlier, but I infer you are saying your 'base test' 50 charts ES 1 min, with no indicators, DOMs etc etc consumes incremental 10% of CPU which you consider 'minimal'.

Info.....
My Setup - similar to #hurleydood & OP. 6 instruments incl. fast/volatile FDAX, NQ, CL & GC, 30+ charts Daily to 15 tick, each with 'customised' Refresh 0.1s & up, 12 DOMs, footprint charts & indicators. Indicators set to Calc on bar close except Priceline, 'Baseline' CPU usage (similar spec to your 4yr old bench test, good graphics card for 4k monitor) is single figures when 'normal' tick data coming in (e.g. US closed, FDAX open but not at the Open), mid to high teens 'normal upper end' & 70+ at US news e.g. NFP. On one core, no GPU assist. Never freezes. Under duress charts lag DOMs, but 'come back' after a brief pause is worst case scenario.

Crosshairs - (per #Sledge)
With NT7 as above, opened NT8 also & connected, one 'bare chart' open with DAX. Absolutely nothing else running. With Mouse pointer selected, no problem. Select Crosshairs (not Global, only one chart) no problem on a relatively small chart (<A4). Maximise full screen (42" 4k monitor) shows terrible lag, unusable. Switch to Mouse pointer, no lag. Switch to NT7 with my setup (i.e. 30+ charts, 6 instruments etc), connect & maximise one chart, select Crosshairs, lag is perceptible but usable. What that seems to demonstrate irrefutably is that Crosshairs is 'better implemented' in 7 than 8. Is it 'good enough' in 8 such that on 'normal sized charts' it won't be a problem? I don't know. Does it give me confidence it will be? No. Would I have more confidence that it won't be a problem in 8 with 30 charts open if it weren't a problem in NT8 with one chart open as it isn't in NT7 with 30 charts open? Yes. What I do recall is that this Crosshairs problem did exist in 7 several years ago. I may be mistaken it may have only applied to Global Crosshairs.

My setup works as a result of many hours of trial & error not only with Properties such as Chart Refresh rate but also thinking & trying to determine how to create an efficient setup. For example, if I have a fast tick on DAX, a monthly Pivot indicator is not an option because it requires to load into memory too much lookback data, so I put a monthly pivot on a 200 tick chart, not on a 25. But I had to figure this out myself. I noticed it took a very long time for the chart to load data. Then I thought 'why?', then I thought, 'ah, maybe because I added a monthly pivot which requires a greater lookback than a daily pivot so it needs to load more data, & on a 25 tick that will be a lot of data, maybe I should try the monthly on my 200tick instead, ah, bingo, not a problem'. Similar intelligent tweaking per OP on chart refresh. If I increase Refresh & causes a problem, what can I do to overcome this? Do I need 250ms on a Daily? Do you think the solution would be to limit the amount of lookback data the User can load depending on the tick or minutes? Or would it be better to make value-added advice readily available to users so it doesn't require a Sherlock Holmes-like forensic ability of your Users to figure out?

Re: your own and, for that matter Trader_55's comments on the 'dumb' things users do that Support subsequently has to deal with, hence your move toward 'out of the box' just works, some comments - Support's 'support' in this regard is weak, & this smacks of typical IT 'it's an end-user problem' attitude. Your third bullet portrays this - 'we even put a modal warning...' but found users instinctively.... What you say is if you increase this you'll increase the load on your CPU. What you don't do is give constructive guidance as to how to address the issue as a whole, which if you did you might not have the problem come back at you. Users aren't usually 'dumb', they are looking to achieve something and lack information. What has Ninja invested in addressing this? In my case I invested a huge amount to get NT7 to meet my needs. I received almost no assistance from Support in this regard. Support's response typically being take all your indicators off and rebuild from scratch, 'it's your fault your system is freezing, you've increased your chart refresh rate and there's a message telling you it will increase CPU load'. Yes, but I did so in order to get the setup to refresh at the speed I need in order to trade. Where is the 'value added advice', such as 'OK, increasing Refresh has now had a consequence of reaching the limitations of your system, you could consider taking these actions.... reduce refresh on slower charts, etc etc etc'. Where is the Resource you have invested in to Help Users to help themselves? (don't point me to the Performance Tips - pitiful after so many years)

So, I want NT8 to run my NT7 setup. What I see is with 50 bare charts, no indicators and a slow instrument incremental 10% CPU usage. So what do you think it will be with 6 instruments with 'core setups' including DAX (fastest is15 tick), NQ, CL, GC, 30+ charts, nearly 300 indicators (I'm the opposite of an indicator junkie, the vast majority are enhancements to NT7's platform functionality rather than trading indicators and some may be redundant with NT8, lightening the load), 12 DOMs, and additional trading tools such as Economic news, footprints? When DAX is slow and US closed (maybe similar data coming in as your 1 min ES?) NT7 is using less than 10% CPU on a single core, compared with your 10% of 4 cores.

So, given this information, intuitively, what do I/you think the likelihood is that NT8 will cope with my setup? As #hurleywood points out, his NT7 appears similar to mine and has similar cpu usage way below your 'base test'. This, & the Crosshairs, doesn't give me motivation to spend hours recreating my Setup in NT8 because the signs - insofar as they are there & to my limited understanding - point to likely outcome of failure, not success. This may be incorrect, maybe your CPU will stay at 10% with my setup. May be you have confidence in that. But you are not providing anything to give confidence, quite the opposite. What would I rather see to give me confidence - 'better CPU usage than NT7, or worse?'. With your 10% on a seemingly more benign setup, I am seeing worse. #Hurley is seeing worse in a side-by-side comparison of 7 and 8 on his/her Setup. S/He is seeing worse compared with running 3D games (probably a 'He' then) - other, real-world applications renowned for being CPU and GPU demanding.

With this history and context, that you quantify 10% incremental CPU usage as 'minimal' I find rather unfathomable. Whilst I haven't done a side-by-side comparison of your 'base test' on NT7 vs NT8, I think it is safe to say that my setup is more demanding with 6 instruments, 30 charts, 300 indicators, 12 DOMS yet uses less on a single core under what are probably similar (incoming data) conditions. There are lots of caveats you may throw in - your test was at the Open, etc etc. End of the day my NT setup seems more demanding than the example you've shown and doesn't seem to exhibit the same CPU usage, despite 8 being multithreaded & using GPU.

You will note that #Hurleywood achieves 0.1s refresh in NT7 (so do I on 15 & 25tick DAX, but only by throttling down refresh on other charts, intelligent tweaking and deduction with no added-value input from Support). He can't be doing this on NT8 because it won't let him so he has max 250ms refresh on NT8 vs 100ms in his NT7 setup). Yet you believe its his 'workspace'.

Perhaps you'd accept a challenge. Because of the above & responses (others having very very similar experiences with similar setups to my own) I don't have the confidence to invest further time at the moment to pursue 8. I tried 6 months ago & paused. Same situation now. So that's two times given up (for now at least). If you have confidence, by all means take my NT7 Workspace & replicate my NT7 setup in NT8 to show me and others it works acceptably in 8, preferably better than in 7. It would be a good exercise & if successful helpful to you and others. Risky for you maybe, but at the least learn from real world users' setups. I'm not saying NT8 won't be able to do it because I don't have sufficient knowledge. But what I am saying is that the issues raised here are absolutely the issues preventing me spending more time - you have made refresh too slow for my needs & non-user adjustable, this will use resources unnecessarily per #Trader_55 which technique I had to use to achieve success in NT7, NT8 uses more CPU per your 10% & has worse performance on Crosshairs, so what's the likelihood of success of my setup working with all 30 charts using 250ms yet my fastest ones won't be fast enough?, can you 'afford' to take this approach when a far less demanding setup than mine uses 10% cpu on 4 cores & mine on NT7 just a few % on one core? Crosshairs problem may not be a problem on a chart that isn't maximised in NT8 on 42" 4k screen. But the fact is it is a problem on a maximised chart, whereas it isn't on same NT7 maximised chart, and this same problem used to happen on NT7. So, does that make me think it is worth investing my time to find out whether or not if I enable crosshairs with 30 charts open but not maximised in NT8 it will NOT be a problem? No it doesn't. It's your product, you demonstrate to me why it WON'T in light of this, preferably by removing this behaviour as per NT7.

I am not of the opinion that Ninja doesn't 'care' as it is unlikely in your business interests (albeit it may be NT8 is suitable to capture the more mainstream social media market who may be less technically demanding), nor that 8 is broken and can't be fixed.

Let me know if you're prepared to take it on and I'll send my workspace.xml.

Kind regards

[NinjaTrader_Brett]

1. We're aware of crosshair performance and are looking into research and development on an alternative implementation on crosshair, we need to push crosshair to the GPU same as we did with charts. This requires tweaks to our core rendering routines which is something we're working on internally. I don't have any feedback yet on that but stay tuned.

2. Our support desk is and always has been available as I mentioned in one of my first post on this thread to look into any performance issues. A performance issue needs to be analyzed individually as it could be caused by anything. We will not assume its chart refresh rendering related since at this time we do not feel it to be related at all. If it is related I would want to seek out that information and get tests setup to understand the case sooner then later. So far I have no actual real world case to test that is causing issue due to chart interval.

If you have such case, please reach out to platformsupport at ninjatrader dot com and reference this forum post. We will then work with you to get workspaces, trace, logs and dive into your case head first. With this post I'm locking this thread since its of benefit to no one to continue to discuss this academically and in the abstract. As the next step would be reaching out to us directly so I could get facts and scenarios which I could add to our test scenarios and really dive into details.