Standard backtest vs. Market replay data & playback

[cupir2]

I created a strategy which when tested in the Strategy Analyzer appears profitable. See image 01, blue circle shows the dates of backtest, green circles shows the expected profitability. Image 02 shows the results produced by using Market Replay Playback. The blue circle shows the dates, the red circles show the lack of profitability.

Both tests ran the same strategy with exactly the same settings.

Image 03 shows the trades produced by each method. The green circles highlight the similarity of the trades taken by both strategies. The red circles highlight differences in profitability and entry/exit times. It is interesting to note how the entry/exit times are offset by 30 minutes but otherwise track together for the highlighted trades.

What can account for the differences seen in theses results?

Thanks
Ken

[NinjaTrader_AlanP]

Hello,

Thank you for your note.

The reason for this is your barstype, heiken ashi. If you compare that bar type to a 1 minute candle, you’ll see that that bars type is not an accurate reflection of price action, but rather modified for visual purposes. For example you’ll notice certain times when the open of heiken ashi bar type is always below the close of the previous bar on an up move, giving you instant profits in a backtest. Running on replay is going to give you the most accurate returns.

To resolve this in the backtest I suggest you add a 1 tick series,

https://ninjatrader.com/support/forum/forum/ninjascript-educational-resources/reference-samples/6852-strategy-backtesting-ninjascript-strategies-with-an-intrabar-granularity?t=6652

Please let us know if you need further assistance.

[cupir2]

Hi Alan,

Thanks for the reply and particularly the url. I am using the Strategy Builder to create my strategies. Is this suggestion different from setting the strategies Calculate mode to On Each Tick?

Looking at my options when defining EnterLong, I don't appear able to specify an alternate bar series. So while I can create an additional bar series with ticks, I don't see how to reference that when setting up the EnterLong action in my strategy.

Also in the example downloaded from the url, for the secondary series, the bar period type was defined as 'tick'. The comments refer to this as if it where a 1minute bar series. Is that correct?

[KINGKODA]

Hello cupir2,

First, you'll need to unlock the code and manually edit the EnterLong order to make it look like this:

EnterLong(1, 1, "ordername");

The sequence is explained as follows.

EnterLong(int barsInProgressIndex, int quantity, string signalName)

We choose 1 for barsInProgressIndex to execute off the secondary data series.

Second, the secondary series is set at 1 tick to get the most accurate execution. 1tick is faster than 1min. It seems to me that the 1 min reference was only used to explain the sequential order of logic under OnBarUpdate.

[cupir2]

I have been using Calculate set to "On Bar Close" within the strategies Default Properties? Can I still use that or must I now use "On Each Tick"?
Thanks

[KINGKODA]

When the Strategy or Indicator are set to Calculate On Bar Close, it means that the logic or conditions will be run once on the close of each bar. When it's set On Each Tick, then it runs the conditions once every tick.

This is not the reason for the disparity between playback and strategy analyzer. The reason for this disparity is the way Historical Processing works; explained here = https://ninjatrader.com/support/help...on_builder.htm

When you link your order to execution off a 1 tick series, then Historical Processing becomes more accurate. But I would still recommend you always double check your testing on Playback since it is the most accurate.

[NinjaTrader_AlanP]

Hello cupir2,

Kingkoda explains it well, thank you.

Yes, this would be different than running on each tick. This you would be adding an additional data tick series so that you submit orders to that series rather than the custom bar series.
This would be slightly more advanced than the strategy builder and would require unlocking the code and adding the additional data series, but if you were to unlock your code and compare the text vs the sample I provided, you should be able to figure it out.

If you would like someone to reach out with a list of third parties I can have someone pass on a list.

Please let us know if you need further assistance.

[cupir2]

Hi,

Thanks to both of you, this has been very helpful. While I am not a programmer this makes enough sense that I can easily make those changes. I would be interested in a list third parties that could provide coding help and possibly tutoring.

Thanks again.

[NinjaTrader_JaredM]

Hello Cupir,

This is Jared from the NinjaTrader Business Development team responding on behalf of my colleague Alan.

You can search our extensive library of NinjaScript consultants through the link below. Simply enter a consultant name or search by using our filter categories. Once you have identified your consultants of choice, please visit each consultant's site for more information or contact them directly to learn more:

https://ninjatraderecosystem.com/Vendors/Consultants#81

You can locate the contact information for the consultants on their direct websites for any additional questions you may have. Since these consultants are third-party services for NinjaTrader, all pricing and support information will need to be obtained through the consultant.

Let me know if I may be of further assistance

This NinjaTrader Ecosystem website is for educational and informational purposes only and should not be considered a solicitation to buy or sell a futures contract or make any other type of investment decision. The companies and services listed on this website are not to be considered a recommendation and it is the reader's responsibility to evaluate any product, service, or company. NinjaTrader Ecosystem, LLC is not responsible for the accuracy or content of any product, service or company linked to on this website.