Tick Replay vs High Fill Resolution of 1 Tick give different results

[AlgoJason]

A simple strategy of entering a long position with a trailing stop gives very different results depending on the settings of Tick Replay and Fill Resolution.

Figures:

Top: Using the standard fill resolution, the stop is set at 944.25 and gets hit.

Middle: Using the high resolution set to 1 Tick, the stop is set at a different value of 942 and does not get hit.

Bottom: using Tick Replay the stop gets set to 941.75 and does get hit.

Obviously they give different PnL.

My question to NT staff is: What is the most accurate method of these three? Is it to use Tick Replay?

Also, why does using Tick Replay give a different result to using the Fill Resolution of 1 Tick? I thought they would be the same.

Edit: The strategy has Calculate.OnEachTick. If changed to Calculate.OnBarClose the differences still exist.

[AlgoJason]

Using Calculate.OnBarClose

I have repeated the experiments with Calculate.OnBarClose and output the OHLC, Fill and Stop price.

The fill price is the same in each case. Only the stop price differs.

Should we use Tick Replay if we want the most accurate simulation of stops?

[NinjaTrader_JoshG]

Hello AlgoJason,

Thanks for your post.

The most accurate method of backtesting out of these three would be to use the "middle" option. Using an order resolution of "high" set to 1 tick would give you the most data points possible to calculate your order fills. This is also because Tick Replay was not designed to provide accuracy in order fills while backtesting and should not be used to simulate how orders would fill on live data. Tick replay is really only meant for building bars and not for executions. The High fill resolution setting applies to the historical executions and allows for them to use a smaller granularity for fills rather than just the historical bar of their series.

Tick Replay allows for OnMarketData() and OnBarUpdate() events to be played back in the exact sequence as if they were live. If you are not using OnMarketData() in your strategy than you are not fully utilizing Tick replay in the way that it was designed. Tick Replay only replays the Last market data event, and only stores the best inside bid/ask price at the time of the last trade event.

I am including a couple of help guide links below that I believe you will find helpful. The page for Developing tick replay has a graphic that shows the differences in what is happening on tick replay between Calculate.OnBarClose versus Calculate.OnEachTick . As you will see, those settings have a varying degree of impact on how your strategies OnBarUpdate() event is called.

Understanding order fill resolution
https://ninjatrader.com/support/help...FillResolution

Developing for Tick Replay
https://ninjatrader.com/support/help...ick_replay.htm

Please let me know if you have any further questions.

[AlgoJason]

Thanks for clearing that up. Looks like I have suffered from a serious case of RTFM!

It says pretty clearly:
"Tick Replay was NOT designed to provide accuracy in backtesting concerning order fills and execution and should NOT be used to expect the exact sequence of executions as running a strategy on live data. For greater order-fill resolution and accuracy in strategy backtesting, you can use the High Fill Resolution in the Strategy Analyzer. Furthermore you cannot combine both Tick Replay and High Order Fill resolution."


Unfortunately I cannot use the High Fill Resolution because I have a Multi-Series Strategy.