Less new partners, less new features, more *real* multithreading

[darkspeed]

It looks like your multi threading has threads that dont actually run independently of each other. Love your product! despise the speed at which it runs..

[NinjaTrader_RJ]

Hello darkspeed,

Thank you for your post.

Are you reporting a possible bug? Can you please clarify how you came about this information so I can send it to my development team if necessary?

Thank you!

[darkspeed]

Well without seeing code it is a best guess.

I use a lot of multi threaded applications - most of which will evenly load the available processors to near the available capacity, minus headroom for the system process.

When you have a multi thread application that loads a lot less than the available processing power ( im seeing 8 - 25 % with a lot of the processors parking ) usually the problem is threads queuing up to write or read a common data - rather than running completely independent until the end of a process. When a large number of threads are waiting their turn in line to read / write everything slows down.

edit : i just went back and tried running ninjatrader.exe as high priority with no other applications running, but did not help

[NinjaTrader_Jason]

Hello darkspeed,

Unfortunately this is as expected since processes related to charts, indicators and the main GUI are limited to one thread. Multi threads are used when running backtests or optimizations in the Strategy Analyzer and order processing.

[darkspeed]

Hello darkspeed,

Unfortunately this is as expected since processes related to charts, indicators and the main GUI are limited to one thread. Multi threads are used when running backtests or optimizations in the Strategy Analyzer and order processing.

I am describing back testin and optimizing.

[NinjaTrader_Brett]

Hello,

As long as you have no enumerations or bools in your properties for a strategy and you select optimize it will split its work load on to multiple cores. Due to many variables you can see cores max out at 30% CPU usage for example. You are seeing this on your side and the thread that manages all other threads would be the same the the NT GUI is on which would account for why you get Not Responding from time to time during backtest. This is expected with current design of NinjaTrader.

We have already made a lot of enhancement to optimization speed in our next major release of the software. However as such when you click the optimize button NinjaTrader will run as fast as it can with your CPU configuration. There are not really any settings you can change to change performance at this time.

I will let development know of your request for these speed features and for the best utilization possible of multiple cores.

-Brett

[darkspeed]

Hello,

As long as you have no enumerations or bools in your properties for a strategy and you select optimize it will split its work load on to multiple cores. Due to many variables you can see cores max out at 30% CPU usage for example. You are seeing this on your side and the thread that manages all other threads would be the same the the NT GUI is on which would account for why you get Not Responding from time to time during backtest. This is expected with current design of NinjaTrader.

We have already made a lot of enhancement to optimization speed in our next major release of the software. However as such when you click the optimize button NinjaTrader will run as fast as it can with your CPU configuration. There are not really any settings you can change to change performance at this time.

I will let development know of your request for these speed features and for the best utilization possible of multiple cores.

-Brett

Thanks Brett, I guess I will eagerly wait for NT8

In the current code is there any advantage to replacing all bool true/ false with int 1/0 ?

[NinjaTrader_Brett]

Specifically any properties that are bool would be good to replace it with int 0/1. As bools have a limitation with multi-core and will disable multi core support.

-Brett

[darkspeed]

Thanks, i do this by default but did not know if it actually had an effect on nt7

anything else i should be looking for like this?

[NinjaTrader_Brett]

Only items are bools and enum (Enumerations). Steering clear from them and you are as optimized as possible.

-Brett

[darkspeed]

Thanks!

Im also throwing in a vote for being able to queue backtests / opto's to run one after the other as a batch..

[NinjaTrader_Brett]

vote received.

Thanks.

[Prov1322]

We have already made a lot of enhancement to optimization speed in our next major release of the software.

-Brett

Brett...this is related to your reply, but slightly "off-topic".

Will the next "major" release be a "code-breaker" like v7 was for v6.5? (In other words, some indicators written for 6.5 would not work in 7...so will the same be true for the next major release?)

Thanks.

[NinjaTrader_Brett]

Once we get closer to release we will start to release this information. I do not have a release date at this time.

-Brett

[darkspeed]

Just wanted to include this in case it helps with the next release of the software.

I removed half the memory for this test - so it is run on 32gb ram.

The optimizer will run at 25% processor utilization, uses and releases memory, but it always seems to release less memory until it hits a steady 90% memory full.

At that point the processor utilization maxes out at 8% for the remaining duration of the optimization. When it hits the 8% it will start to park cores and the ninjatrader.exe shows as " Not responding" for the full duration of the optimization.

I am going to retry this with 64gb ram and also edit out core parking to see if it makes a difference.