Historic order fill issue

[momchi]

\\

Hi, I'd appreciate a bit of guidance here - what to do about an issue where I see my orders in Historic filled horrendously unrealistically (and mismatching my Realtime performance).
This makes Optimization and Back-testing very difficult (as I am trying to match these with what happens in Realtime)...

I did read the "Historical Backfill Logic", and it makes sense to some degree, but then it doesn't - if a strategy runs on anything but OnBarClose. I run on 10k tick candles. Take a look at the attached image - does it make sense to have X-number of minutes to (to skip to next candle, or even one after next?) in order to fill A MARKET order?!... Slippage is set to 0, BTW.

In the screenshot below - I am trying to do 2 pt stop size (clearly impossible in Historic!). My orders are Unmanaged, market orders.

Please, advise me how I should address this? Should I write my own methods for order fill... clearly not optimal solution!

Thank you,
would appreciate some practical help...
P.S. the issue is worse when the tick frequency slows down, OR, the next candle completes during the Futures Trading Halt (4:15pm - 4:30pm EDT)

[NinjaTrader_ChelseaB]

Hello momchi,

May I confirm this strategy adds a 1 tick series to use for intra-bar granularity to increase historical order fill accuracy?

Below is a link to a forum post that details this.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/94098-isfirsttickofbar-vs-onbarclose-for-backtest-live?p=773377#post773377

When Calculate is OnBarClose, the conditions are evaluated when the bar closes. If the conditions trigger an order, that order would be submitted at the open of the next bar. A Market order would fill at the open of the next bar.

What is Calculate set to in the parameters window for this instance of the script (not in State.SetDefaults), and is TickReplay enabled?

[momchi]

Hi Chelsea, very much appreciate your reply -
your link to the post discussing NT8 features for TickReplay and Order Fill Resolution is invaluable, need to pin in to the top of topics list!

Need to read it more thoroughly, but it seems that adding intra-bar granularity data-series can help me as I could at least rely on the order to be filled/processed at the close of the intra-bar series rather than the main series, if I understood the advice correctly...?

I am so happy to have a customer service like you who has that level of understanding of the platform -
thank you, Chelsea!

If I don't come back to this post, would mean the above link/post has helped resolve my issues!
Best wishes,
Momchil

[NinjaTrader_Jim]

Hello momchi,

Yes, submitting orders to a single tick data series can help to have orders filled according to the tick by tick movement of the instrument rather than having the order fills estimated from the OHLC values of the next bar in the data series.

If the strategy requires historical Calculate.OnEachTick or Calculate.OnPriceChange, Tick Replay can be combined with submitting orders to a single tick data series.

We look forward to assisting.

[momchi]

Hi Jim & Chelsea,
appreciate the guidance and help so far very much. However, I might need a bit more guidance -
I tried both approaches and (think) I learned a lot about the differences, but still having difficulty...

Jim,
do you mean that the tick series that issues the orders have to be the one to close these? (My case is with stop-loss orders). I submit these when logic in main series (larger time-frame) triggers... then tried to Close these by submitting from the finer tick series... but it still uses the (Open?) price from main series (Close of last bar?) and not the one from the sub-series...

See screenshot #1 (log file) - it shows Close[0] price at time of order submission, for the particular (short) order price was submitted at 1977.5, stop triggered at 1979.5


Does the framework care which series issued the order and which closes it? (BTW, I do use OrderAction.BuyToCover for the Short Stop orders, but only Sell for the Longs, as there is no other action)

From screenshot #2 (executions) - one can see that the price of the Buy (to cover / stop) is same as the one at Sale (1977.5 !?).
This is not right, it seems that it does not Fill at the Close[0] of the finer series (1979.5), but at the (OHLC?) of the main series still...



Am I missing something? How can we control this to match the sub-series Close[0] price / fill ?
All the help / hints will be appreciated very much!

P.S. Again, I am using un-managed orders. TickReplay = on, if it is off, nothing is changed. Fill resolution is Standard (cannot turn on, due to addDataSeries complaint). Calculate is OnEachTick.

[NinjaTrader_ChelseaB]

Hello momchi,

The 1 tick series is only used as the BarsInProgressIndex for the order submission methods. The logic is still performed on the primary series of the chart, BarsInProgress 0.
No, it does not matter which series is updating OnBarUpdate to place an order to any series.

This allows the orders to fill using that 1 tick series for fill accurate fill prices.

TickReplay would be used at the same time, as this would allow the Calculate setting to update for OnEachTick and OnPriceChange historically.