Not a True Multi-Thread application

[lmatiukas]

Correct me if I'm wrong...but....is this a real multi-thread app?

A fresh install crashes on my quad core 4GHZ+ 12GB DD3 Ram at 1600.
I ran Ccleaner, uninstalled old NT8, deleted all files, got rid of extra registry connections, and so forth.
I can't go much faster on the processor, so I will upgrade to 32GB Ram DD3 at 1800+ just in case, but I don't see much faster there for sale, and again I think my system should be sufficient for a fresh install.

It looks like Strategy Analyzer is connected along with Control Panel, not sure what else is running together. Any plans of separating this two, so Strat.Analyzer won't crash the system.

Any plans on making sure Strat. Analyzer gets its own core if possible?

Historical Data also is connected to Strategy Analyzer and same issue. If I'm downloading unrelated historical stuff, S.A. won't work. Or is there some magical Network connection limiter?

Thanks, curious about this!

[lmatiukas]

A bit of a run-on and misguided post that was ^^...

Really I meant to ask, how to ensure that a fresh install on a 4GHz+ intel i7 quad core processor with 8MB Cache and 12 GB DDR 3 Ram won't crash running a simple strategy?

I'm sure if I get this on a clean and fast PC, you can recreate the issue. Run a strategy on SP500 for the past year, and try to click anywhere on Strategy Analyzer multiple times, same thing can be done for Market Analyzer. When they're faced with dealing with a large set of numbers, crashes are imminent =(. Beta thing I guess

[NinjaTrader_PatrickG]

A bit of a run-on and misguided post that was ^^...

Really I meant to ask, how to ensure that a fresh install on a 4GHz+ intel i7 quad core processor with 8MB Cache and 12 GB DDR 3 Ram won't crash running a simple strategy?

I'm sure if I get this on a clean and fast PC, you can recreate the issue. Run a strategy on SP500 for the past year, and try to click anywhere on Strategy Analyzer multiple times, same thing can be done for Market Analyzer. When they're faced with dealing with a large set of numbers, crashes are imminent =(. Beta thing I guess

Can you provide further details about your NinjaTrader 8 Beta and your Strategy Analyzer settings? I would like to try to replicate what you are experiencing using your exact settings.

Who are you connected to?
What are your exact Strategy Analyzer settings? Here is a screenshot example of the information I need:

http://screencast.com/t/oK2qxXlJrCn

[lmatiukas]

Here's a snipet of the settings - well it's not a built in strategy, but it's still a "simple" strategy for sure.

I've attached the latest crash, as it was different from the usual. I'll see if there is anything on the log the next time crashes from just doing too much.

I guess I do suspect that on larger data sets it would need an unlimited amount of GB of ram, but it's happened just running a fault free strategy on NASDAQ100; I only have indicators that were built in, or converted to NT8 from NT7 by NT staff and posted on this website.

And then I only have that one strategy of mine - because really why have more, if one simple one does all it should.

[lmatiukas]

Patrick, I don't really mean to be negative, and this could really be a beta thing, because things seem to change from one reinstall to another. For example, outside of 100+ or so failed stock tickers for historical data, I was able to run the strategy for the whole NYSE in about 1.6 hours. How does the system act when running a large data set? Where are things stored during the duration of optimizer or strategy when it's running for hours? Also, any chance to use multi threading WITHIN the analyzer? Having 4 cores, means it's only going to access 1 core per task, so my CPU does go up to 20%+ while running strategies. Would love to see 2 cores getting accessed, if maybe different parts of optimization could be spread out. Just curious what your thoughts are on that.

What I have done to not have it crash maybe is to download CCleaner and clear any registry errors, as well as anything else that thing found. Additionally, I've ran defrag on all drives. Going forward I'm also replacing by 12 GB DDR 1333 MHZ ram with 16-32GB 2400MHz ram. It doesn't seem I can go much higher on the processor... But i'll be curious to see if it will crash at all after that, and if running the strat on NYSE take less than the current 1 hour and 40 minutes.

[sledge]

Having 4 cores, means it's only going to access 1 core per task, so my CPU does go up to 20%+ while running strategies. Would love to see 2 cores getting accessed, if maybe different parts of optimization could be spread out. Just curious what your thoughts are on that.

What I have done to not have it crash maybe is to download CCleaner and clear any registry errors, as well as anything else that thing found. Additionally, I've ran defrag on all drives. Going forward I'm also replacing by 12 GB DDR 1333 MHZ ram with 16-32GB 2400MHz ram. It doesn't seem I can go much higher on the processor... But i'll be curious to see if it will crash at all after that, and if running the strat on NYSE take less than the current 1 hour and 40 minutes.

4GHz+ intel i7 quad core processor with 8MB Cache and 12 GB DDR 3 Ram

Which i7 exactly? It's probably hyperthreaded, so you would have 8 logical processors.

Please open up Windows Task Manager while running an optimization and see if you are running out of memory.

http://www.screencast.com/t/7pXHgfY82

Are you running tests back to back? Is NT not releasing memory? Does restarting NT help?

[lmatiukas]

Sledge, I think you've "NAILED" it (get it). It does seem like NT8B12 isn't releasing memory. Running starts back to back is what seems to cause it to slow down more. Shutting NT down and restarting surely helps. I also did just order a bunch of 2.6kMHz Ram for a total of 32GB from 12 GB that I have now. I'm sure that will help. I will post once it's installed on the differences.

Thanks for looking out. Opening performance screen likes yours I do see 8 logical processors. How have you been faring with your set up? Any lag issues?

[sledge]

How have you been faring with your set up? Any lag issues?

I haven't used strategy analyzer.

I have only used market replay. The only time I have issues is with too many charts/tabs...NT collapses on itself and slows down big time. This started in B10. I used to be able to run 2.5 years of replay data in about 4 hours without any consumption of memory. I haven't tried in B12 that big setup again, but I don't believe it's been resolved.

I'm not sure what is in your code, but hopefully you are releasing anything that you are creating.

[NinjaTrader_PatrickG]

Patrick, I don't really mean to be negative, and this could really be a beta thing, because things seem to change from one reinstall to another. For example, outside of 100+ or so failed stock tickers for historical data, I was able to run the strategy for the whole NYSE in about 1.6 hours. How does the system act when running a large data set? Where are things stored during the duration of optimizer or strategy when it's running for hours? Also, any chance to use multi threading WITHIN the analyzer? Having 4 cores, means it's only going to access 1 core per task, so my CPU does go up to 20%+ while running strategies. Would love to see 2 cores getting accessed, if maybe different parts of optimization could be spread out. Just curious what your thoughts are on that.

What I have done to not have it crash maybe is to download CCleaner and clear any registry errors, as well as anything else that thing found. Additionally, I've ran defrag on all drives. Going forward I'm also replacing by 12 GB DDR 1333 MHZ ram with 16-32GB 2400MHz ram. It doesn't seem I can go much higher on the processor... But i'll be curious to see if it will crash at all after that, and if running the strat on NYSE take less than the current 1 hour and 40 minutes.

I will follow up with Product Management regarding your questions about NT8's expected behavior, however, I would first like to see if this can be replicated on my end with your strategy or on your end with a built-in strategy.

Can you run a test using a built-in strategy such as the Sample MA Crossover? Can you send me your strategy? You can send it to PlatformSupport[AT]NinjaTrader[DOT]com Ticket # 1544277 ATTN Patrick G

[lmatiukas]

well one thing got confirmed. I avoided the crash with 16GB of faster ram... Using my strategy that is pretty simple, but includes 6 indicators, NT8 ran up more than 10GB of Ram tab and this time around was just really slow, as compared to previous effects 12GB. Now I fully get why it crashed, but still would be curious to see if it's possible to flush Ram after finishing a strategy analyzer/optimization run. Will have 32 GB in 2 days, will re-asses again.

To answer your point, I'll check what build up I get using normal strategies; but It does seem that me checking for 10-20 different conditions on every bar update, builds this thing up... Just wish it could be flushed out.

[lmatiukas]

Will still get to it. But what I mean by multi-threading. How about EVERY analyzer and optimizer and EVERY screen runs on its own. Currently if I do 2 or 3 strat analyzers simultaneously, they seem to be all sharing the same core - you're so close to perfection with real multi-threading; maybe for each iteration of open window or task, assign it to the next least used core, and cycle them in a nice rotation.

I'm being selfish here I know; but I'm also thinking $.5 million account, and upgrade to a small supercomputer with 1 or 2 x 8-core processors with DDR4 Ram for 64-128GB; and let this Ninja system trade my script. So you can understand my "b****g" I hope, and again I apologize for it, I only mean to share my two cents, and surely loving the progress.

[NinjaTrader_PatrickG]

well one thing got confirmed. I avoided the crash with 16GB of faster ram... Using my strategy that is pretty simple, but includes 6 indicators, NT8 ran up more than 10GB of Ram tab and this time around was just really slow, as compared to previous effects 12GB. Now I fully get why it crashed, but still would be curious to see if it's possible to flush Ram after finishing a strategy analyzer/optimization run. Will have 32 GB in 2 days, will re-asses again.

To answer your point, I'll check what build up I get using normal strategies; but It does seem that me checking for 10-20 different conditions on every bar update, builds this thing up... Just wish it could be flushed out.

Is your strategy using any drawing tools? There is a known memory leak which is being worked on actively by the Development Team with a fix being rolled out in the near future. If you are using drawing objects, try commenting out any drawing objects and running a test again.

Here is a great link on understanding the logic behind the optimizer and how your settings can affect performance:

http://ninjatrader.com/support/helpG...ionPerformance

[lmatiukas]

hey there,

yes I draw 7 indicators. I'm up to 22GB now, and it maxed out, but almost made it! In a similar strategy optimizer. Keep in mind, this time it DID NOT crash, so clearly it's a memory hog and given enough juice it will continue operating.

It seems to just build up and build up, the longer the analyzer/optimizer is running, or if they're run back to back even. Tomorrow I'll go upt to 32 GB and even faster DD3 Ram and give it a shot again, running the same optimizer.

[NinjaTrader_PatrickG]

hey there,

yes I draw 7 indicators. I'm up to 22GB now, and it maxed out, but almost made it! In a similar strategy optimizer. Keep in mind, this time it DID NOT crash, so clearly it's a memory hog and given enough juice it will continue operating.

It seems to just build up and build up, the longer the analyzer/optimizer is running, or if they're run back to back even. Tomorrow I'll go upt to 32 GB and even faster DD3 Ram and give it a shot again, running the same optimizer.

Do your indicators call Draw methods which are available from the chart's Drawing menu, such as Draw.Line(), Draw.AndrewPitchfork, etc.?

If so, please comment these out to test.

Your last comment sounds like I would expect: It would build up as much memory as the system has. Once the framework realizes you're almost out of memory,it just buffers out what it needs through garbage collection. So yes, it will just keep operating. This is expected.

The main question though is, should it use that much memory? If using draw objects, it probably will use unnecessary memory resources. If not, I would recommend further optimizing your code using some of the recommendations here:

http://ninjatrader.com/support/helpG...niteration.htm