Choose CPU for backtesting and optimization

[honest-trader]

Thanks Jim will review them.

[NinjaTrader_Jim]

Hello tradestar and honest-trader,

tradestar, we apologize for the delayed response as the ticket did not reopen in our support queues.

Question:
Is the single thread theory on data processing true? Should I invest in intel vs amd?
In my local area I can find i9 10900K at a better price than 9900ks. Should I go for 10900k?
Do I need a powerful GPU, I was thinking of getting those heat sink type. I don't what it is call. but they are very affordable at less than $100. Question is will they power my monitors and TV?
Is ram speed important for my use cases?
My friends using the same indicators report NT8 + indicators on i5 kept crashing. What do you think I should get in overall setup to make things smooth as butter?

Optimizations will use all cores available to run simultaneous optimization iterations. It is often best to have more cores for simultaneous iterations, but you will also need to have the memory in order for each optimization iteration to be able to allocate data and calculate trades for their specific backtest. Outside of this, bottle necks may appear if individual threads are overloaded and this is where a faster CPU or a CPU that performs more instructions per clock cycle would do better. A mix of a power CPU with multiple cores and enough RAM to allow the optimization to carry without stalls would be ideal.

A fast powerful GPU does not matter as much because the optimization is performed on the CPU.

We would not see crashes (where the platform shuts itself down) but would see stalls and stutters when memory is fully utilized and needs to be decommited with garbage collection before new resources can be committed and the optimization can resume.

At the end of this post, I have included some information to help understand where resources are spent in an optimization.

honest-trader, You will need to unlock the code and set IsInstantiatedOnEachOptimizationIteration to false in State.SetDefaults. After doing so, you must make sure that all class level variables are reset in State.DataLoaded. Some properties may be reset in State.Configure, but Satate.DataLoaded would be applicable as well and would be suitable for resetting variables that depend on the data loaded.

Please see the second example in the Help Guide article you have linked and take note of how the variables that are used in State.DataLoaded are reset and re-instantiated there.

Understanding Optimization Memory Utilization

I have created a video demonstrating how the SampleMACrossover strategy, a simple strategy that utilizes IsInStantiatedOnEachOptimizationIteration=false for efficiency, can still quickly utilize memory resources.

Demo — https://drive.google.com/file/d/15pz...w?usp=drivesdk

We should consider the following for memory consumption:

Data * Strategy resources * Number of optimization iterations * Number of trades * Keep best # of results.

As we can see there are a number of factors that are involved and memory utilization can climb very quickly depending on a few of these factors. Once memory gets maxed out, we can experience short freezes where memory is decommitted, stored to disk, and then new resources are committed before the backtest is resumed.

We can easily control the number of iterations involved and we can also consider writing our strategies to use IsInstantiatedOnEachOptimizationIteration = false; (Which requires that we reset class level variables in State.DataLoaded.)

IsInstantiatedOnEachOptimizationIteration — https://ninjatrader.com/support/help...niteration.htm

Optimization Tips — https://ninjatrader.com/support/help...ionPerformance

Walk Forward Optimization — https://ninjatrader.com/support/help...ss_metrics.htm

Genetic Optimization — https://ninjatrader.com/support/help..._algorithm.htm

We look forward to assisting.

[honest-trader]

Hi could anyone help with this please? how to change the code for reducing optimization times?

[honest-trader]

Hello Diezguinho,

Thanks for your post.

I will leave this thread open for other community members to share their insight, but I would not spend too much time focusing on the graphics metrics of a popular benchmark since CPU, memory and using a Solid State Drive will have the biggest impact on backtesting/optimizations.

Optimizations will benefit most with as many cores that can be utilized. Memory will have an important factor, and strategies should be written so they use IsInstantiatedOnEachOptimizationIteration=false and they clear and instantiate their objects in State.DataLoaded so they can as performant as possible in an optimization backtest.

More information on using IsInstantiatedOnEachOptimizationIteration = false can be found here - https://ninjatrader.com/support/help...niteration.htm

I don't have details on memory impact, but I did share some results I had using a Ryzen 1920X that might be able to help you make a decision.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1034949-limits-of-optimizer-multi-threading

NinjaScripts process realtime data on an instrument thread while historical OnBarUpdate processing is done in a thread pool. Once the script is running with realtime data, its data processing methods will be processing on one core. (Scripts on separate instruments will be processed on a separate instrument thread.) Optimizations, like we see in my linked post, will utilize all cores.

We look forward to assisting.

Im trying to use multi factor optimization using 3 factors but the time estimated is crazy, ive bee reading this https://ninjatrader.com/support/help...niteration.htm as per previous suggestions by NT support team but im clueless where i need to do this? Should i unlock the strategy code and paste this bit of code mentioned in this link?

[tradestar]

I am new to ninjatrade NT8 and know nothing about computers. I only had been using macbook pro laptop for the last 6yrs. I need to build a desktop and wonder where my money should be place on. Logically the processor, mother board, Ram and SSD. I do not play games but am powering a 55 inch 4k tv and 2 other 4k monitor.

Need some hardware expert recommendations.

I was advised to get Intel i9 9900ks.Reason being the data feed coming in is using single thread and intel are the top in terms of raw power from single thread. Also I am using heavy algorithmic indicators.

Question:
Is the single thread theory on data processing true? Should I invest in intel vs amd?
In my local area I can find i9 10900K at a better price than 9900ks. Should I go for 10900k?
Do I need a powerful GPU, I was thinking of getting those heat sink type. I don't what it is call. but they are very affordable at less than $100. Question is will they power my monitors and TV?
Is ram speed important for my use cases?
My friends using the same indicators report NT8 + indicators on i5 kept crashing. What do you think I should get in overall setup to make things smooth as butter?

[NinjaTrader_Jim]

Hello Diezguinho,

Thanks for your post.

I will leave this thread open for other community members to share their insight, but I would not spend too much time focusing on the graphics metrics of a popular benchmark since CPU, memory and using a Solid State Drive will have the biggest impact on backtesting/optimizations.

Optimizations will benefit most with as many cores that can be utilized. Memory will have an important factor, and strategies should be written so they use IsInstantiatedOnEachOptimizationIteration=false and they clear and instantiate their objects in State.DataLoaded so they can as performant as possible in an optimization backtest.

More information on using IsInstantiatedOnEachOptimizationIteration = false can be found here - https://ninjatrader.com/support/help...niteration.htm

I don't have details on memory impact, but I did share some results I had using a Ryzen 1920X that might be able to help you make a decision.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/1034949-limits-of-optimizer-multi-threading

NinjaScripts process realtime data on an instrument thread while historical OnBarUpdate processing is done in a thread pool. Once the script is running with realtime data, its data processing methods will be processing on one core. (Scripts on separate instruments will be processed on a separate instrument thread.) Optimizations, like we see in my linked post, will utilize all cores.

We look forward to assisting.

[Diezguinho]

Hi,

I am choosing a processor to get the best performance in optimizations and backtestings, and secondly for live trading.
Could you tell me which benchmark I can guide me with?
Specifically I am deciding between Ryzen 5 3600 and Ryzen 7 2700X.
For example, in "Passmark" 3600 is better and in "Cinebench", 2700X.

Another question, are backtesting / live trading single-core and optimization is multi-core? Is this true that I suppose?
All the information you can give me will be appreciated.

Regards!