big variations in backtesting with order type

[PolarBear]

Hi All,

I've recently experienced some weird behaviour with backtesting in relation to the type of orders I'm executing. I understand the fundamental difference between them, but my results are confusing, so I wanted to know if someone can shed some light.

I've been testing a high frequency trading strat on AUD/USD with 1 minute data.
I observe my strategy does really well in LONG trades through the period 2006-2011
shorts have a downward equity slope but overall there's a huge upward equity curve.
This was using a EnterShort / EnterLong entries

If I alter the code so I'm using EnterShortLimit and EnterLongLimit (setting the entry price to the Close of the 1 minute dataseries - immediate execution at the price I want),
suddenly the short trades do very very well, and the long trades aren't so good.

It's obviously to do with the limitations of the fill algorithm or something.
Some other info which could be relevant, is I'm testing with a small profit target of around 10-20 pips

Can someone please advise of the limitations in relation to backtesting as above, and/or with smallish profit targets.

Brokers these days are offering very tight spreads (<1-2 pips) so I wouldn't have thought the above would be a problem, HOWEVER, the backtesting algorithm something is definitiely messing something up.

thanks

[NinjaTrader_Bertrand]

Hi PolarBear, which broker or datafeed provider are you using here?

If you work on FX data, normally the bidprice is using for displaying the last so this could influence results in backtesting when you're not executing with including the bid / ask spread of course.

With the default fill algorithm, Limit orders would not fill on a touch only, price has to trade through to ensure a conservative limit order fill estimate.

http://www.ninjatrader.com/support/helpGuides/nt7/backtest_a_strategy.htm

If you're using non market orders on the managed approach you could also run into ignored orders, as resting limit orders are still working and non cancelled for example before attempting to reverse, for this please run with TraceOrders = true in the Limit orders model to see more detailed debug order output in the NT output window.

http://www.ninjatrader.com/support/forum/showthread.php?t=3627

[PolarBear]

Hi Bertrand,

Thanks for the info. I'm using the default fill, so the price action must go through the price to fill the order which is good.
Also, I am using high quality CQG data for a 5 year period (no gaps in data). It's 1 minute data, last price.
I understand it would be better to have bid/ask or tick data, but I believe what I have should be good enough to provide accurate results provided i understand what's happening on each trade.

I observe one potential problem in that if a trade enters and closes on the same 1 minute bar this can lead to false results, as you don't know if the price action actually touched your entry before exit price.
To combat this, I have create a script that I run post backtest, and remove all the trades that open and close on the same 1 minute bar.
Is there a better way to do this in NT7???

My results seems to still be favouring short trades for some reason.
I have a high number of trades, and my output (before or after removing the trades that open/close on the same 1 minute bar) is VERY successful.
I don't get excited yet, because i know most likely it's some product of an error in the backtesting.

To recap, I have quite a small stop (10 pips), however I've shown if i change that to 20 (to account for bad slippage) and run the backtest again it's still profitable.

Can you think of anything in the backtesting methodology / fill process that would create inflated results in the backtest as i may be observing??

thanks

[NinjaTrader_Bertrand]

What methods are you using here for your exits PolarBear?

You could wait a bar in backtesting before submitting them, thus the exit could not fill on the entry bar already.

Did any ignored orders show up when testing with TraceOrders = true as I suggested?

I would not know what else could contribute here, but I would be glad to give it a run on our end here if that would help you.

For regular FX spot backtesting not incorporating proper spread / slippage is the most common issue for overly optimistic results, as the bid / ask spread is not factored in.