Backtesting at tick level

[ctrlbrk]

Hello,

I have a strategy I want to run against tick level data. 400 ticks is fine.
I am using one year as the range, and the period is Tick / 400.

When I try to backtest the strategy, Ninja crashes with the "Microsoft Windows has detected a problem", or Ninja stops responding with it just failing to do anything (zero cpu usage for extended period of time like 1 hour) and it just sits at Loading Data.

In each case I've got taskmgr loaded and noticed Ninja reaches approx 1.3GB in size when it crashes.

Not once have I been able to successfully get this to work. If I take it from one year down to a few days it works most of the time, but backtesting a few days is not backtesting at all...

What is the fix for this?

Thank you.

P.S. I tried it using the built-in strategy "MACrossOver" just to make sure it wasn't my strategy, and had the same results. My strategy normally only produces about 100 buy signals a year when using minute data, whereas MACrossOver produced over 7000, so I was hoping if anything my strategy would be less resource intensive.

[NinjaTrader_Jason]

Hi ctrlbrk,

What happens if you reduce the historical data by half? I suggest to modify the amount of historical data to test how much historical data you can use without the crash occurring.

It seems a backtest with one year of 400 tick data is requesting too much resources.

[ctrlbrk]

It does not work (half data, or six months).

I have never been able to get it to work on even an entire calendar month. Somewhere around 3 weeks seems to be the max. I would have to manually change date periods about 17 times to backtest just 1 year one single time this way, and I would most certainly have no hair left about half way through because I would pull it all out.

On occasion when I go beyond 1 calendar month, it doesn't produce an error but it does produce invalid results, like it says it entered 1 trade over a six month time frame. My strategy enters trades based on 2m data, the period I test against is usually 1m data to refine exists or add to a position. So going from 1m period to 400 tick period should only change exits and not drastically change entries, so this seems to be NT having a problem processing the data.

[ctrlbrk]

Was looking at log just now, this error was there from the same timeframe I was running the above test.

This is when it came back with just 1 trade over a six month period.

.

[NinjaTrader_Jason]

The error indicates NinjaTrader at that moment in time requested too much resources and subsequently an 'out of memory' exception was thrown.

What happens if you use the same backtest parameters and backtest the 'SampleMACrossover' strategy? I suspect the strategy you backtest request a lot of resources.

[moneyman]

backtesting

I have experienced the exact same thing. It runs forever, eventually crashes, and creates some multi gigabyte file on my hard drive. this is testing on a 2250 volume chart. When i try and optimize with just a few weeks of date, it eventually finishes, but is completely inaccurate. Any help?

[ctrlbrk]

Pray for a solid NT7 beta soon.

[moneyman]

Anyone help here?

Jason, do you know of anyone who can accomplish this backtesting, or whether ninja can do this at all? I really need this accomplished somehow. You would think using market replay could do it for shorter time frames, but i can't even get it to backtest accurately with a week or two of data...

[NinjaTrader_JoshP]

moneyman,

Market Replay is not backtesting. If you want to add a tick time frame you need to do Add(PeriodType.Tick, 100) for instance. Tick data is very resource intensive. Please try running on less days back and ensure you are running on an as strong as possible machine.

[moneyman]

But can it be done at all?

I understand it is very resource intensive, but if it can't do 2 weeks, then it basically can't do it at all. I have a quad core with 4 gig ram, less than a year old. Should have plenty of horse power. My question is, can the current version of ninja backtest tick data based strategies accurately? No matter what i try, I either get an error like others on this forum, or I get wildly inaccurate results.

[NinjaTrader_JoshP]

moneyman,

Please use less days back.

[moneyman]

less days

OK, I tried 2 days, and it hung. Are you saying it can be done? Really, if it can't do it at least 2 weeks, it can't do it. That's OK, i understand it is an area Ninja has to improve, but it would nice to hear for sure that it can't, so I can quit trying...

[NinjaTrader_Jason]

This is hard to tell, since it depends on the strategy as well. It seems the strategy you are using uses a lot of resources. If the backtest causes an out-of-memory exception using 2 days of historical data, the same would occur using two weeks.

[moneyman]

tick data

OK, but can ninja accurately backtest tick data based strategies at all? Just need a yes or no here.

[moneyman]

follow up

and, when you answer that last one, the 2 days i tried to optimize, that is with only 2 parameters, with a strategy that only takes 2-5 trades per day. No way can it be using too many resources to test that. It should have tried about 20 combinations max, by my calculations...