Backtesting results

[Matts]

I've been testing a strategy on both the market analyzer and market replay using the same data. The strategy analyzer is giving different results to the tick replay, most likely because it's running on Renko bars. E.g. a trade is generated about 25 minutes later in the analyzer compared to the tick replay. Is it possible to get the strategy analyzer to calculate in the same way as the market analyzer, so it will generate the same executions?

With the tick replay comparing the execution time and price to historical data, it appears to have executed 1 tick higher than the historical price.Is there a way to have the trade execute at the last price instead of one tick above or below? I attached some screen shots to make it clearer.

[NinjaTrader_JessicaP]

Hello Matts, and thank you for your question. To make sure we are on the same page, I would just like to ensure we are comparing these two things :

• NT8 control center -> New -> Strategy Analyzer
• NT8 control center -> Connections -> Playback Connection

If this is the case, the differences between Historical, Playback, and Live trading are detailed here,

http://ninjatrader.com/support/helpGuides/nt8/en-us/discrepancies_real-time_vs_bac.htm

If this is a single time frame strategy, I would like to recommend using high fill resolution with a one-tick data series to place trades on. You can learn more about fill resolution here,

http://ninjatrader.com/support/helpGuides/nt8/en-us/?understanding_historical_fill_.htm

Please let us know if there are any other ways we can help.

[Matts]

Yes, you are correct. I was using the playback connection with historical tick data, and strategy analyzer. Will using High resolution fills on Renko bars enable more realistic fills with backtesting? Or is the only way to realistically test Renko strategies by using playback connection?

The backtest results I'm getting are very different between the two. E.g. The backtest generates a profit factor of 2.4, but when run on the playback connection, it's more like 0.8.

Edit: I tried using High resolution and it seems to be working better. I added extra ticks (1 renko bar) to the stop on the playback so that it doesn't get hit within the renko bar distance buffer. Not sure if this is needed. Ideally what I'd like to do would be to get as close to the backtest results with the playback connection.

[NinjaTrader_JessicaP]

I am glad your results are closer between the two. High fill resolution creates a different data series that trades are placed on, and so the bar type you are using primarily will be irrelevant. The extra data series is only used for communication between your strategy and the virtual trade desk.

There will always be slight differences between the strategy analyzer and the playback connection, since the playback connection takes volume data into account when deciding when and how to fill trades. However, high fill resolution with a 1-tick series for the trade desk will enable your strategy to take advantage of nearly every opportunity available.

[Matts]

Although High Fill resolution is the solution to a more realistic test, I was consistently getting profitable results from the standard fills. Ideally what I'd like to do is somehow program a strategy so that it will use the standard fill algorithm in market replay, or as close to it as possible.

[NinjaTrader_PatrickH]

Hello Matts,

Thank you for your response.

There is no means to force any other fill type in Playback.

Please let me know if you have any questions.

[Matts]

It makes sense that it wouldn't be possible to change the fill type, but I would've thought it'd be possible to create a stop strategy that emulates the way the standard fills are working. If Renko bars are being used for the primary series I'm assuming that the entries would be calculated correctly for both standard and high fill resolution, which is OnBarClose. The stops are most likely causing most of the difference between the fill types as they would cause more entries if the trades are getting stopped out, and more stops would be hit more often.

Can you provide any further details about how a stop is executed on a Renko bar using standard fill resolution?
Does it calculate at the close of the Renko bar, and then use the last price at the level of the stop?

In some cases I was using trailing stops, so maybe it could be tracked by shifting stops by increments of the bar size instead of tick by tick. Also maybe stop/limit orders could prevent the trade getting stopped out too early.

[NinjaTrader_JessicaP]

I am providing an excerpt from the help guide.

Historical Fill Algorithm

... The Standard order fill resolution uses an algorithm to break each historical bar into three virtual bars to mimic the movement of price within each bar's timeframe. The virtual bars are created based on the proximity of the Open price to the High and Low prices. This provides more realistic intra-bar fills compared to traditional backtesting algorithms which only use static OHLC values.




The Standard setting creates virtual bars according to the following logic:

When the Open price of the bar is closer to the High price then the Low price:

1. Open price to the High price
2. High price to the Low price
3. Low price to the Close price

...


Slippage

Slippage can be added to your order fills to help mimic real market conditions. The value is expressed in "ticks", the minimum value of fluctuation for an instrument, and is only applied to market and stop-market orders. NinjaTrader will add the slippage to each order however you cannot have more slippage then the high/low price of the next bar.
Order Fill Resolution

...Order fill resolution of "Standard (Fastest)" is the default setting and will use the existing bar type and interval that you are running the backtest on to fill your orders. This means that the historical fill algorithm will use the same Open, High, Low, Close, Time values that are available to the strategy for running the order fill simulation.

This means that if you are using Renko bars and Standard fill resolution, your orders will be filled based on virtual (OHL -> HLC) Renko bars with the same brick size, plus or minus a value up to a specified slippage value.

[Matts]

I did read this, but it didn't make a lot of sense in relation to renko bars because:
open and high, and low and close are the same value for red bars.
open and high, and low and close are the bar size apart for green bars.
high and low are the bar size apart for all bars
Intrabar fills don't matter that much, as the biggest problem for the strategy is the stops that are being hit outside the high and low of the renko bar.

In standard fill prices outside the renko bar won't be hit, but with high resolution they will. What I want to create is a trailing stop that will keep it's distance during the formation of the renko bar, but if hit by the renko bar, would then execute on the close of that bar. It should be possible to do this using the Renko bars themselves instead of a stop, but as it's more than 2 bars, it'll need all the combinations of green and red bars to execute the trades properly.

[NinjaTrader_JessicaP]

If you would like total control over how fills occur, rather than relying on virtual bars or a high fill resolution series, I would like to recommend creating a multi-timeframe strategy, and ensuring all your trades are placed on the second timeframe. You can then configure this second time frame, as well as the logic that interacts with this timeframe, in your code. Provided this second time frame is very granular (for instance, a 1 tick timeframe) your strategy should then perform very similarly in every kind of backtesting and in live trading. You will want to use an overload that allows you to specify a bars in progress index when placing trades, e.g.

EnterLong(int barsInProgressIndex, int quantity, string signalName)