Historic Fill Processing in Strategy testing

[mtamaku]

Hello: What is the use or purpose of the Historic Fill Processing parameter in NT8 strategy ? In what case it can be used and does it get used at all in testing real time strategy ?
What is the best way to test the strategy so my test results are exact same or as close as real time trading. ? Thanks,

[NinjaTrader_Kate]

Hello mtamaku,

Thank you for your note.

Are you referring to the Order fill Resolution setting? Our Historical Fill Algorithm will run on existing data that you are backtesting and simulate historical orders using the method described in the link below in "Understanding the Historical Fill Algorithm". You can optionally choose to bring in a secondary data series to be used to get more granular fill on orders and is explained in the section "Understanding Order Fill Resolution of the following link as well:

https://ninjatrader.com/support/helpGuides/nt8/understanding_historical_fill_.htm

You should expect that a strategy running real-time (live brokerage account, live market simulation, Market Replay etc...) will produce different results than the performance results generated during a backtest. This difference may be more easily seen on certain Bars types (e.g. Point and Figure) than others due to their inherent nature in bar formation.

During a backtest you can select conservative or liberal fill algorithms which will produce different results. Fills are determined based on 4 data points, OHLC of a bar since that is the only information that is known during a backtest and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick'.

Also, here is a link to the differences on real-time vs backtest (historical).

http://ninjatrader.com/support/helpGuides/nt8/en-us/discrepancies_real-time_vs_bac.htm

Adding intra-bar granularity can help with this.

Intra-bar granularity adds a second data series such as a 1 tick series so that the strategy has finer granularity in the historical data in between the OHLC of the primary series. This allows for more accurate trades by supplying the correct price at the correct time for the order to fill with.

In NinjaTrader 8, there have been two new enhancements so that programmers do not have to manually add this secondary series and code the script to for high accuracy fills (Order Fill Resolution, which we touched on in the above link) and for intra-bar actions (TickReplay).

Here is a link to our forums that goes into depth on using Order Fill Resolution and Tick Replay to ensure your backtests are as close to real time results as possible:

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance?t=102504

High Fill Order Resolution and TickReplay cannot be used together. If it is necessary to have both, it is still possible to add intra-bar granularity to a script in the code itself for order fill accuracy and use TickReplay to update indicators with Calculate set to OnPriceChange or OnEachTick historically.

However, testing using Market Replay would give the most accurate-to-real-time results.

Please let us know if we may be of further assistance to you.

[mtamaku]

Thanks for Info. do you suggest test strategy using real time data in SIM account or using the Market Reply using last few days of replay data would be same as real live ?

[NinjaTrader_Kate]

Hello mtamaku,

Thank you for your reply.

Both of those would be about equally close to running a strategy on a live account. The main difference with both would be that orders would be filled based on the simulator's calculations, whereas with a live account orders would be filled based on market dynamics.

Please let us know if we may be of further assistance to you.

[mtamaku]

Hello: If I use Renko bars to test my strategy using another indicator let's say MACD in real time using SIM account, would the result be same in Live account similar to SIM account ?
As per you explained and as per my test, historical fills are always based on Open Close of the the bar and always positive but in real time it should fill as per the condition met by the indicator logic. Correct ?
If I want to rule out all different results based on historical fills, would it be always better to do the real time test in SIM to see exact behavior of the indicator logic ?
Thanks,

[NinjaTrader_Kate]

Hello mtamaku,

Thank you for your reply.

It would be advisable to test a strategy that uses bars that redraw the open, such as renko bars, on real time data, that's correct. You could also use Market Replay data with the playback for essentially the same effect, as that plays data back just like real time. The results may not 100% match live results, but the only difference would be that the live results would be filled based on market dynamics and the simulator results would be filled based on the Simulator's calculations as to what price the order should be filled at. In most cases the difference would be negligible.

Please let us know if we may be of further assistance to you.

[mtamaku]

Hi:
Do you mean Simulator results when running test using Market Reply, correct ?
Would the result be same as far as order fill if I use the real time data on SIM account and on LIVE account ? Thx

[NinjaTrader_Kate]

Hello mtamaku,

Thank you for your reply.

When I say Simulator, I mean the simulation engine that fills orders for the Sim accounts. This functions the same on both live data and Market Replay when using a Sim account.

Please let us know if we may be of further assistance to you.

[mtamaku]

Hello: If my logic is to trade based on comparing Open and close of the current bar to previous bar. should it get the same result in historical ( either using Standard or High fill) compared to real time ? Also if I use the tick charts for back testing, all ticks are available to place the trades so would historical fill use those ticks vs open,close,high,low for historical fill. ? What I read in the post this issue is with certain bar types like Renko in particular. Does it apply to Line break bar if I use open/close logic to place the order ?

Thanks,

[NinjaTrader_Kate]

Hello mtamaku,

Thank you for your reply.

If you're running the strategy On Bar Close, yes, it would get approximately the same results in historical as on real time data. If you are running the strategy On Price Change or On Each Tick, no, as on Historical data all the strategy will know is the OHLC of the bar.

If you are using tick based charts for testing, this will still be the case as the OHLC of the bar is all that is known for historical bars - it's not playing those ticks back in real time. The line break bars also redraw and it would be recommended that for accurate testing that you use Market Replay data with the Playback connection.

Please let us know if we may be of further assistance to you.

[mtamaku]

I am using ObBarClose , so if I do the test in realtime using SIM account, would order fill will be similar to historical fills based on open and close of the bar using the logic. ?
Also market replay and realtime would get me same result for the time or data I pick ? lets say last 30 days, it would use the same data in market replay that was used from realtime.

[NinjaTrader_Kate]

Hello mtamaku,

Thank you for your reply.

The Strategy Analyzer and the Playback connection use a little different of an algorithm, so you might see small variations in the prices of fills, but they should return roughly the same if you're running everything on On Bar Update. You'll see some differences on live due to orders being filled by market dynamics rather than the simulator algorithm.

Please let us know if we may be of further assistance

[mtamaku]

Hi:
For realtime testing in SIM account, I have a logic that use open/close of bars, and when I use the Historic fill using standard fill I get positive result but when I use High fill option, it is in negative.
My confusion is if it use OHCL of the bar for historical and in real time using OHCL logic where it do not care about tick or intra bar or tick by tick data, then why I do not get two results same for two different option in historical fill.

From testing perceptive, How do I convince myself that if my result is positive in real time using Standard fill, that it will produce similar result in real time in live account ? i know there is no future guarantee of any trading system but I want to be sure that my strategy is working or not.
Thanks

[NinjaTrader_PaulH]

Hello mtamaku,

Thanks for your reply.

Regarding the Historical fill options, on a chart connected to real-time data, this would only apply to historical trades on that chart. When you apply your strategy to a chart that has x days of data loaded, the strategy will perform historical trades over those X days and these trades would be made using the historical fill option selected. Actual real-time trades would be filled according to the market dynamics and the historical fill option has no impact on actual real-time trades.

You are asking some great questions and our advice here, to help convenience yourself, is to examine each and every trade performed on a chart and see that it does or does not meet your entry/exit criteria. If it does not then you would need to dig further to understand the contributing factors and to that end, NinjaTrader_Kate has provided some good references to review in post #2. I would also suggest using Playback with market replay data especially if you are using exotic bar types line Renko, Heiken-Ashi, etc.

[mtamaku]

Hi Paul: Thanks,
So if I do the test using realtime data in SIM account and if I leave the Historic fill at Standard(default) and do not care about historic result,, and if my result for next 5 days or a month positive, this is how it would fill order if I go live with this strategy.
I understand that SIM engine is not same as real market fill but it would fill as close as possible to SIM. correct ?