Please improve performance using Range bars

[JS999]

In NT 6.5 there is a serious performance problem when you have any range bar charts open. I have been doing some testing using Mirus futures with the Zen-Fire data feed and monitoring the speed/performance of the quotes coming in... and I have found that having even one or two range bar charts open slows down the system in a noticeable way. If you have multiple charts open it starts to be a serious issue in heavy volume trading times, as quotes will lag.

Please address this concern in NT7, and make sure that you do some system performance stress testing with Range bars and other non-time-based charts under heavy load... please optimize that code if you haven't already.

(Just for reference I have one of the most powerful machines you can buy with gobs of RAM, and hardly any indicators at all on the charts, with all indicators set to CalculateOnBarClose = true. Also, my DB has been reset/deleted/etc. so it is nothing to do with my specific setup, only your implementation of the afore-mentioned code.)

Thanks!

[NinjaTrader_Jason]

Hello JS999,

Can you please double click the chart and check what value is used for 'Display update interval (sec)'.

Did you perhaps had workspaces opened in the background?

Does the issue perist using two range charts without any indicators?

[JS999]

Hello JS999,

Can you please double click the chart and check what value is used for 'Display update interval (sec)'.

Tried with 0 (real-time) and .2 seconds as an update interval. Anything greater than that is unacceptable for real-time trading, so if the interval needs to be greater than .2 seconds then you need to fix it.

Did you perhaps had workspaces opened in the background?

No other workspaces exist on my PC.

Does the issue perist using two range charts without any indicators?

The range charts themselves had no indicators on them, and closing them is what speeded up performance. Try it during heavy volume times. Set up one PC with no charts, and another PC with a few range charts, and compare performance. See what you get... then try closing down the charts and see what happens.

[NinjaTrader_Jason]

It is as expected the setup that contains charts will request more resources than the setup without charts.

Is there a difference if you use volume charts instead of range charts?

[JS999]

It is as expected the setup that contains charts will request more resources than the setup without charts.

Is there a difference if you use volume charts instead of range charts?

The huge difference comes from constant-range charts, and to a lesser extent constant-volume charts (although not as much, I think). Minute charts are fine, you can have lots of them open without affecting performance. Even having one day of range data slows things down, and multiple charts with multiple days makes it seriously untradeable.

[NinjaTrader_Jason]

In regards to performance in NT7:

NT6.5 supports multi-core/multi-processor hardware. NT internally supports multiple threads of executions which would be run by the Windows OS on different cores/processor. However, NT6.5 is not optimized to make full usage of a multi-core/ multi-processor hardware.

NT7 will come with optimized multi core/multi-processor support. This would allow you to make full usage of a multi-core/ multi-processor hardware.

However, due to the nature of the Window OS, NT7 would not be able to fully use all available multi-cores/multi-processors at any given time.

[JS999]

I'm sorry, but I really didn't understand what you meant in this post...

In regards to performance in NT7:

NT6.5 supports multi-core/multi-processor hardware. NT internally supports multiple threads of executions which would be run by the Windows OS on different cores/processor. However, NT6.5 is not optimized to make full usage of a multi-core/ multi-processor hardware.

You said "NT internally supports multiple threads of executions which would be run by the Windows OS on different cores/processor"... no it doesn't. When I run NT 6.5 it runs on one core, while the other 3 sit idle. That is not "supporting multiple threads of execution" properly, that is running on only one core. Saying that it "supports" multiple threads because it runs on a system that has multiple cores is not accurate - all windows program run on a system with multiple cores. To support multiple threads properly it would have to make use of all my processor cores, which it does not.

In regards to performance in NT7:
NT7 will come with optimized multi core/multi-processor support. This would allow you to make full usage of a multi-core/ multi-processor hardware.

Ok....

However, due to the nature of the Window OS, NT7 would not be able to fully use all available multi-cores/multi-processors at any given time.

Huh? Didn't you just say the exact opposite one sentence ago? Either NT 7 can make use of multiple cores, or it can't. Which is it? Are you saying it can only use 2 out of 4 available cores, or something like that?

[NinjaTrader_Dierk]

>> When I run NT 6.5 it runs on one core, while the other 3 sit idle
Not sure why you think that is a valid prove of your perception.

Anyway, there is no need to argue, NT6.5 is multi-threaded (I know, since I coded it). However, multi-threading could be used in more places which we're doing with NT7.

[JS999]

>> When I run NT 6.5 it runs on one core, while the other 3 sit idle
Not sure why you think that is a valid prove of your perception.

Anyway, there is no need to argue, NT6.5 is multi-threaded (I know, since I coded it). However, multi-threading could be used in more places which we're doing with NT7.

Let's redefine our terms here: when I say multi-threaded, what I mean is the ability to utilize multiple cores, as some rendering programs do, for example.... it clearly doesn't do that. So is NT 7 going to allow for full use of a quad core processor? You said that it would take advantange of multi-core processors, but not fully - what exactly do you mean by that?

[NinjaTrader_Dierk]

We will provide details the next few days.

However, windows has limitations which prevent you from using e.g. a quad core processor "fully" like: all UI activity needed to be in the same (main) thread. Please consult the Microsoft documents for details.

[JS999]

We will provide details the next few days.

However, windows has limitations which prevent you from using e.g. a quad core processor "fully" like: all UI activity needed to be in the same (main) thread. Please consult the Microsoft documents for details.

That's fine... UI can all be in a single thread, but there are plenty of other things going on that use up processor cycles, and from what I can tell NT 6.5 doesn't leverage multi-core processors to speed them up. For example... right now when I see a burst of activity on the time + sales screen, all the charts hang for a second or two (sometimes even longer) before updating after the burst is done. It is obvious that there are some calculations/saving of data or whatever going on before the UI is updated, and this is the type of thing that needs to be changed.... hopefully NT 7 will do that.

[mrpowerballad]

For example... right now when I see a burst of activity on the time + sales screen, all the charts hang for a second or two (sometimes even longer) before updating after the burst is done.

Just want to chime in to say that I also use range bars usually with 2 or 3 instruments/charts, multiple indicators on each chart set to Close=False, update interval set to 0, and I haven't had any notcible problems with lag in the year or so I've been using Ninja. My computer is very standard -- 4 gb ram, 2 core E8400 chip. I also use Zen-Fire/Mirus (via Comcast interent).

Just out of curiosity, are you measuring bar lag soley by time and sales? Not to point out the obvious, but are you certain it's the case that the bursts of activity on the T&S you're associating with lag are actually pushing through new levels? If not, then the range bars may not have a reason to update. You often see huge bursts of acitivity on the ES tape, only to then notice that a particular price level was never actually broken.

However, if a minute bar or the DOM is pushing through new levels and the range isn't doing anything, then that's obviously a problem. Have you compared range bars to minute bars or DOM side-by-side?

Just curious in case this is something I should keep a closer eye on. Thnx.

[JS999]

Just want to chime in to say that I also use range bars usually with 2 or 3 instruments/charts, multiple indicators on each chart set to Close=False, update interval set to 0, and I haven't had any notcible problems with lag in the year or so I've been using Ninja. My computer is very standard -- 4 gb ram, 2 core E8400 chip. I also use Zen-Fire/Mirus (via Comcast interent).

How far back are your range bars going (in terms of days)? And what instruments? That will help me to do some more testing...

Just out of curiosity, are you measuring bar lag soley by time and sales? Not to point out the obvious, but are you certain it's the case that the bursts of activity on the T&S you're associating with lag are actually pushing through new levels? If not, then the range bars may not have a reason to update. You often see huge bursts of acitivity on the ES tape, only to then notice that a particular price level was never actually broken.

Yes, they are pushing through new levels, and in a dramatic way. And it's not just the range bar charts that don't update, it's all the charts. Time + sales scrolls through at an extremely rapid rate, the chart is frozen, then all of a sudden the chart jumps 10 ticks higher.

However, if a minute bar or the DOM is pushing through new levels and the range isn't doing anything, then that's obviously a problem. Have you compared range bars to minute bars or DOM side-by-side?

I have multiple connections to Mirus, some running in virtual machines. What I have done is set up a connection through a VM and another normal one. I note that the normal one is typically faster in terms of getting the data (by a minuscule amount). However, when I add a range chart or two, all of a sudden the normal T & S slows down be slower than the one in the VM.

This has been happening across multiple OSes that I have tried it on, and more than one internet connection.

[Radical]

Can you elaborate upon how you're running NT in VMs, and how that increases performance? I know this is off topic, but I've seen you mention this in other threads, and I'm just curious since I have an i7 system sitting next to me waiting to be built, and I would like to take full advantage of what it can do if possible.

[JS999]

Can you elaborate upon how you're running NT in VMs, and how that increases performance? I know this is off topic, but I've seen you mention this in other threads, and I'm just curious since I have an i7 system sitting next to me waiting to be built, and I would like to take full advantage of what it can do if possible.

All that it means is that I am running multiple copies of NT, each with its own connection to my broker. This means that I can assign the entire VM to one of the cores, so in effect, if I am running 3 of these things then I get to use 3 of the 4 cores of my i7 920 system. Each copy runs a number of charts, so the load is split up amongst the cores. I make sure that the copy that runs on the core that handles actual trades only has a couple of small charts open, with the other copies running longer terms charts or charts with more indicators. One VM runs a long-term range chart which would kill my the T&S coming in if I ran it on the main copy, for example.