Backtesting not working properly.

[PedroBM]

Hello,
I am backtesting some strategies using 1 tick high resolution historical fill processing.
For whatever the reason, it doesn´t work properly. Sometimes, stop loss doesnt work or orders that should be stopped, ends in profit.

When I replay the strategy, it works perfectly and the result doesn´t match the backtest result.

I am attaching an image showing some examples. The first trade is stopped properly, but the second one that should be stopped, ends in profit.

Regards

[NinjaTrader_Emily]

Hello PedroBM,

Thank you for your post.

It is typically expected to observe some discrepancies between running a strategy in real-time/playback and in a backtest. My colleague Chelsea has created a very helpful forum post regarding discrepancies as well as order fill resolution here:

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/94098-isfirsttickofbar-vs-onbarclose-for-backtest-live#post773377

A different option that allows for intrabar fills in your strategy that could be more similar to the results you see in real-time or playback would be to add a 1-tick data series to the strategy itself and submit orders to that 1-tick data series. We have a reference sample demonstrating this concept here:

https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

When running strategies in playback, what is the Calculate setting for your strategy? Are you calculating On Bar Close, On Each Tick, or On Price Change? This could also affect the results you are seeing in the backtest. If you run the strategy On Bar Close in playback, then you would want to use the standard order fill resolution in a backtest. If calculating On Each Tick or On Price Change, then using the high order fill resolution or the added 1-tick data series in your strategy logic would yield the most similar results possible.

Please let us know if we may be of further assistance.

[PedroBM]

Dear Emily,

Thank you very much for your post. It is quite clarifying and helpful.

I have been doing some modifications on my algo.

I have added Tick data with AddDataSeries, calculating OnEachTick.
Also modified the orders so they use tick dataset for execution, but the result still is quite disappointing.

Reviewing the results, if the bars where the entry order is executed and also hit the stop; this stop is not considered and later on when the price hit the stop some bars later is executed.
It looks like that it is not considering the ticks data for order execution.

I am attaching an image for clarification.

Regards

Pedro

[NinjaTrader_Emily]

Hello PedroBM,

Thank you for your reply.

You previously mentioned that when you replay the strategy, the strategy works as expected and the results are not matching those of the backtest. I suggest comparing the properties of the strategy when running in replay/playback to those from the backtest to make sure there aren't any settings that are different between the two. It is also important to check that you are testing the same range of time with the same range of data. If you run the replay/playback on historical vs. playback mode, do you notice a difference there? For more information about the two playback modes:

https://ninjatrader.com/support/helpGuides/nt8/index.html?playback.htm#UnderstandingHowThePlaybackWorks

Another useful tool for understanding what is happening in playback vs. backtest would be to add debugging prints to your script. This will help you to see how conditions are evaluated while the strategy is running and could help to narrow down what the discrepancy is that is causing different results between the two environments. For more information about adding prints, please see this page:

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working#post791121

I look forward to hearing the results.