Market Replay vs Back Test results

[NinjaTrader_Bertrand]

It'a on our feedback list in product management to enhance this area for our next major update, I've added your vote and input in here as well.

Can you please clarify your second question?

[cfp462]

Even though I cannot back test it, I would still like to be able to set up a PF or Kagi system that trades on the reversals. So lets say after the first 2 Os down in a reversal I would want to buy. Or in a Kagi system I would want to sell when it switches to a thin line

[NinjaTrader_Bertrand]

You could access the price data of the 'boxes' generated of course as well in those chart types, give for example this script a try for reversals on your Kagi chart :

...where flip is a variable of type bool.

if (CurrentBar > 1)
{
	if (Close[0] < Low[1] && flip)
		{
			DrawArrowDown(CurrentBar.ToString(), 0, High[0] + TickSize, Color.Red);
			flip = false;
		}
				
	if (Close[0] > High[1] && flip == false)
		{
			DrawArrowUp(CurrentBar.ToString(), 0, Low[0] - TickSize, Color.Blue);
			flip = true;
		}
}

[cfp462]

Bertrand, thank you for that. It almost works like it is supposed to.

However, it drawing the arrows after the next bar is put in, not when the line first switches. I tried using BarsInProgress to get it to try and calculate intrabar, but that didn't seem to work.

How can I get it to draw the arrows when the line first switches; as apposed to after the next bar is drawn (like it is doing now)?

Once again thank you so much for all your help

[NinjaTrader_Matthew]

Are you running this on CalculateOnBarClose set to True or False?

[cfp462]

That was exactly the problem, thank you guys for all your help.

[NinjaTrader_Matthew]

Glad to hear it was a simple fix. Let us know if you need anything else.

[Majorshi]

How can I backtest strategy on replay data?

I have a live automatic strategy running on IB account, However, there was a weird order entered last night. I try to figure it out by running backtest on replay data. I couldn't, how? thx.

[NinjaTrader_Bertrand]

Majorshi, it would be best if you would contact us via Help > Mail to Support so we could review your logs / trace for the order in question.

[Malletto]

Hello, I’m having the same issue. Can someone from ninja help me with this. My automated strategy does incredible on the strategy analyzer but the results are different live. Id really appreciate the help

[Malletto]

Hello, I’m having the same issue. Can someone from ninja help me with this. My automated strategy does incredible on the strategy analyzer but the results are different live. Id really appreciate the help

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

Please review the help guide document on the differences on real-time vs backtest (historical).

https://ninjatrader.com/support/helpGuides/nt8/discrepancies_real-time_vs_bac.htm

A strategy running real-time (live brokerage account, live market simulation, Playback connection etc...) will produce different results than the performance results generated during a backtest.

When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

Below is a link to the help guide on Calculate.
https://ninjatrader.com/support/help.../calculate.htm

To improve the accuracy of a backtest, you may use Tick Replay along with an added 1-tick series to have logic processed intra-bar and have orders filled intrabar.

Tick Replay would be used to have the logic process OnEachTick or OnPriceChange with historical data, but this does not allow for intra-bar order fills. You would need to add a single tick data series and submit orders to that single tick data series for a strategy that uses Tick Replay.

High Order Fill Resolution allows for intra-bar order fills with historical processing, but is not compatible with Tick Replay.

Please reference the SampleIntrabarBacktest example and the following Help Guide links for more information.

SampleIntrabarBacktest 'Backtesting NinjaScript Strategies with an intrabar granularity' - https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

TickReplay — https://ninjatrader.com/support/help...ick_replay.htm

Developing for Tick Replay -
https://ninjatrader.com/support/helpGuides/nt8/developing_for__tick_replay.htm?zoom_highlightsub= developing+for+tick+replay

Additional information may be found in this NinjaTrader Forum post —
https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance?t=102504

[Malletto]

Thank you for the response. What if my order full is in standard? Also, are there issues with on the kagi chart or with point and figure also?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

From the "Differences on real-time vs backtest (historical)" help guide page linked in post # 27: "This difference may be more easily seen on certain Bars types (e.g. Point and Figure or Renko) than others due to their inherent nature in bar formation."

This would include Kagi Bars Types as well since they are an exotic Bars Type.

Adding intrabar granularity to a script could help improve the accuracy of a backtest.

Review this forum thread for additional information about intrabar granularity: https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance?t=102504

You could also consider running the strategy on the Playback Connection with Market Replay data to more accurately reflect the strategy running on realtime data for Kagi or Point and Figure Bars Types.

See this help guide page about using the Playback: https://ninjatrader.com/support/help...connection.htm

"What if my order full is in standard?"

Are you referring to Order Fill Resolution Standard?

Order fill resolution of "Standard (Fastest)" is the default setting and will use the existing bar type and interval that you are running the backtest on to fill your orders. This means that the historical fill algorithm will use the same Open, High, Low, Close, Time values that are available to the strategy for running the order fill simulation.

The above information still applies regarding Kagi and Point and Figure Bars Types when using Standard Order Fill Resolution.

Please review this help guide page "Understanding historical fill processing": https://ninjatrader.com/support/help...ical_fill_.htm

[Malletto]

So the results that i see through backtest, are they better then there supposed to be? Or worse?