Computer upgrade decisions

[Temjinck]

Hello,

I am in the process of upgrading my computer for future strategy development on NJ. I understand the incoming NJ7 is quite efficient in its memory usage, but I still would like to make sure my new computer would be optimized to run backtesting on NJ as quickly as possible.

An important question first.

What is the usual bottleneck for backtesting on massive minute/tick data? CPU? Ram? Hard drive latency and/or reading/writing speed?

I know that NJ7 supports multi-core CPU and I would be getting a high end quad CPU, so that's done.

As for ram, would I get benefit from a 6 GB memory over a 4 GB?

More importantly, would I get a significant boost in backtesting speed if the program (and database) is on a Solid State Drive?

I'm still deciding between getting the Intel 80GB X25-M G2 SDD or G.Skill Falcon II 128 GB SDD.

The main difference is in that the Intel drive is far capable in random small bits writing/reading speed, but has inferior sequential writing speed. Whereas, the G.Skill has much faster sequential writing speed but average random small bits writing/reading speed.

My assumption is that importing historic data into NJ would considered as sequential writing. That is of a low priority to me as it is usually a one off job.

As for backtesting, I would assume it involves a lot more reading than writing, and would be more on small bit data than large size files. That means the Intel drive would be faster in that regards.

Of course, that also assumes the HDD is the real bottleneck when backtesting even on high end CPUs.

Any advices would be great. Thanks!!

[NinjaTrader_Jason]

Hello Temjinck,

Unfortunately we do not have any recommendations how much memory and what CPU to use. The general rule applies the faster, the better. Minimum requirements can be found at the link below.

http://www.ninjatrader.com/webnew/support_installationguide.htm

I am not aware of an usual bottleneck when backtesting strategies.

In addition, I am not familiar with a Solid State Drive either. We never tested such a drive and not sure if this would be faster when backtesting.

[Temjinck]

Thanks Jason.

What about when testing under NJ 7? I am aware that it is far more memory efficient and it seem memory capacity is no longer the bottleneck. When your development team perform those tests, are their CPU utilization maxed up way before their entire memory are used?

Does anyone else have an opinion on what is the appropriate computer setup for optimum back testing speed? Or have used SSD in their backtesting machine?

Thanks

[NinjaTrader_Jason]

Please see the link below for more information regarding NinjaTrader 7 and tests that have been performed.

http://www.ninjatrader.com/webnew/NT7/NinjaTrader7.html

Case 1
Many tick based charts (tick, volume and seconds) on high volume futures markets. On application start up/workspace recovery:
Version 6.5 - Blew up after five minutes with an out of memory exception and 1.2 GB RAM peak usage
Version 7 - Application loaded after 75 seconds with peak RAM usage never exceeding 100 MB

Case 2
Backtesting of a multi-series strategy on a basket of instruments
Version 6.5 - Peak RAM usage of 1.3 GB and crashed with an out of memory exception at around 6 minutes
Version 7 - Peak RAM usage never exceeding 430 MB with a completion time of approximately 80 seconds

Case 3
Backtesting of our SampleMACrossOver strategy on 50 tick ES series on approximately three months of historical data
Version 6.5 - Peak RAM usage of 1.2 GB, 550 MB retained in memory after backtest completion in about 4 minutes
Version 7 - Peak RAM usage never exceeded 110 MB and backtest completion in about 40 seconds

Case 4
Backtesting of our SampleMACrossOver strategy on 50 tick ES series on approximately 1 year of historical data
Version 6.5 - Failed with out of memory exception
Version 7 - Peak RAM usage of 140 MB with a completion time of less than 3 minutes
These performance tests were performed on a standard Dell Inspiron laptop.