Incorrect Fill Price even when using tickreplay and having bid/ask.

[Trader212310]

Hi NT8,

You NEED to add a setting to your Strategy Analyzer that will let it:

1. Fill orders at whatever the price was at the moment the order signal was generated. Not at the opening price of the following bar or following tick.

2. Same as item 1, but must fill on Bid/Ask data if the user has Bid/Ask data. Especially Bid/Ask data that happened in between Last ticks and on Last ticks (if the user has this data).

3. Your Strategy Analyzer's Chart needs to have the same features as your regular chart (such as buttons in the GUI that can be used to display BID/ASK data, multiple instruments and so on.)

PERIOD!

I'm sorry to be harsh, but not having these features is just downright irresponsible and reckless. I can't image how much money your users have lost due to the misleading backtests results the lack of these features has caused.

I've been in the industry a while and am actually in the process of writing a letter to the CFTC suggesting they require all commercial trading/backtesting software have the above mentioned features 1 and 2 to prevent retail traders from misunderstanding backtest results.

SFT-132 does seem like a good idea.

[NinjaTrader_Ray]

Hi NT8,

You NEED to add a setting to your Strategy Analyzer that will let it:

1. Fill orders at whatever the price was at the moment the order signal was generated. Not at the opening price of the following bar or following tick.

2. Same as item 1, but must fill on Bid/Ask data if the user has Bid/Ask data. Especially Bid/Ask data that happened in between Last ticks and on Last ticks (if the user has this data).

Thank you for your feedback.

1. Only market orders can be filled immediately and they are filled at the time the order is generated. Signals are generated on the close of a bar thus the next tick (which is the open of the next bar) is where the order is filled at. Limit and stop orders are "working" on the next bar and evaluated against this bars data to determine if it indeed has been filled and at what price.

2. Thanks for the suggestion.

[Trader212310]

Thanks for the response Ray.

You said

Signals are generated on the close of a bar thus the next tick (which is the open of the next bar) is where the order is filled at

The price of the next tick you mentioned can be extremely different then what the price was at the close of the bar that generated the signal, especially with thinly traded symbols.

Why not have it fill at whatever the price is at the the nanosecond after the bar closes? Or better yet just at whatever the (ideally Bid or Ask) price was at the moment the signal happened?

You've built such a great platform, I fail to understand why you undermine it with inaccurate backtesting
Ninjatrader is superior to many of the $5k-$10k a month platforms I've tested in my career, but its fatal flaw is backtesting/optimization accuracy, which is such a shame because it seems like such a simple thing for you to fix.

[DaveE]

Hello rspine,

I dont see anyone answering your question in post#3

1) Can you point me or explain exactly how the fill price is calculated. With formulas and snippets etc.

Here is what I deduced about market order fills after some testing of the fill engine (IsHistorical):
Assuming you have OrderFillResolution.Standard and Slippage=0:
a) If you are in the context of a Ask data series then a market buy order will get filled at GetOpen(CurrentBar+1). This is a correct simulation of what would likely have happened in Realtime, ie the broker would probably have filled at the opening Ask price of the bar-being-built which is when OnBarUpdate gets executed.

b) If you are in the context of a Bid data series then a market sell order will get filled at GetOpen(CurrentBar+1). This is a correct simulation of what would likely have happened in Realtime, ie the broker would probably have filled at the opening Bid price of the bar-being-built which is when OnBarUpdate gets executed.

c) However if you are in the context of a Ask data series then a market sell order gets filled at the same price as a buy. This is obviously inaccurate and I suppose that the idea of the Slippage property was to set this to some 'average' spread and then you might get a slightly more realistic sell order.
Similar problem applies to executing market buy on a Bid series.

So for instance optimizing SampleMACrossOver with default Slippage=0 will give an illusion of higher profits (or lower losses) that you would have got Realtime, simply because the Bid Ask spread has not been taken into account.
With the fill engine as at RC2: The only way I can see to get accurate inclusion of the historical Bid Ask spread (which might vary wildly) into backtests, is by ensuring that all Buy orders get executed on a Ask data series and all Sell orders get executed on a Bid data series. But then your code is going to look a lot more complicated than SampleMACrossOver! Has anyone got a better idea?

[NinjaTrader_Ray]

Why not have it fill at whatever the price is at the the nanosecond after the bar closes?

That is exactly what is happening. The very next tick after the bar closes is the tick that opens the next bar.

Using bid/ask data is another discussion and an item we have already put on our suggestion list.

[DaveE]

Why not have it fill at whatever the price is at the the nanosecond after the bar closes?

This would not make sense if the bar was logically closed due to a session break.
(Ray's answer seems to assume there is no session break).
For a simple example, and avoiding the bid ask spread issue, lets say you are trading on a 30 minute Ask data series, in ET time zone using Trading Hours=Nymex Energy RTH (sessions 9am to 2:30pm):
In Realtime you might think of the bar ending at 2:30pm as closed if you are a nanosecond after 2:30pm on a Friday, however NT does not actually close it until it gets the first Ask tick after new session opens at 9:00am the next Monday. Only at that point does OnBarUpdate() fire.
Its possible that NT receives some Ask ticks during the defined session break, and these might get stored in Historical Data, but it would not make sense to fire OnBarUpdate() for these since the Broker would presumably reject them (no order connection).
So in Historical it is correct for the Fill Engine to wait until the end of a session break (if there is one). That means it is correct for market buy orders on the Ask data series to fill at GetOpen(CurrentBar+1). If instead it tried to fill at some rogue Ask tick that arrived during the session break it would not be simulating what happens in live.

Incidentally:

I think the fact that GetCurrentAsk(), when used in Historical, returns Close[0] is partly to blame for the confusion, since it helps to perpetuate the myth that OnBarUpdate() is called at the start of a session break. Because it is actually called at the end of a session break it would make more sense for GetCurrentAsk() to return GetOpen(CurrentBar+1) see here

[Trader212310]

Hi Ray

I don't think your

That is exactly what is happening.

quote in post 20 is correct.

Lets assume I got an entry signal at the 2pm close of a one hour bar and the closing price of this 2pm one hour bar was 100. Lets assume (its a thinly traded symbol) and the next tick does not come until 2:18 pm and that tick's price in 106.

The logic you are saying would cause the strategy analyzer fill the order at price 106.
I would need it to fill the order at the 100 price.


P.S.

FYI:
Regarding my post 18 quote below

Why not have it fill at whatever the price is at the nanosecond after the bar closes?

In the scenario I've described in this post (post 22) 100 (not 106) is what the price was the nanosecond after the 2pm one hour bar closed.

[koganam]

The logic you are saying would cause the strategy analyzer fill the order at price 106.

If you use a market order.

I would need it to fill the order at the 100 price.

Which it will if you use a Limit Order and the new bar has a low lower than 100. In other words, the entry bar will have to cut the entry price, which is what is most likely to happen even a live market.

[NinjaTrader_Ray]

The logic you are saying would cause the strategy analyzer fill the order at price 106.
I would need it to fill the order at the 100 price.

Exactly what I am saying and what is expected. On the tick of 100 which is the last seen price for the bar, your strategy now will process your trade logic and issue an order (market order for the sake of this discussion), the next incoming tick is 106 which is the open of the next bar which is now where your order is filled.

[Trader212310]

Hi Ray,

Would the fill price be 100?

[NinjaTrader_Ray]

Hi Ray,

Would the fill price be 100?

No, the fill price would be 106. On the close of the bar when you issue your market order, the price of 100 is now in the past. 106 is the next price after the market order is issued hence its filled at that price.

[Trader212310]

Hi Ray,

You need to make the fill price be 100 in this situation. Not 106.

Again, your platform is fantastic and I don't mean to sound harsh, but not doing this is just

downright irresponsible and reckless

as I mentioned in post #16.

Thanks.

[NinjaTrader_Ray]

Hi Ray,

You need to make the fill price be 100 in this situation. Not 106.

Again, your platform is fantastic and I don't mean to sound harsh, but not doing this is just as I mentioned in post #16.

Thanks.

I am sorry but I disagree 100%. To try and illustrate my point.

1. Bar closes with a price of 100
2. Your strategy now executes logic and it decides to enter a market order
3. The next price comes in is where your market order is filled

It is chronologically impossible to fill the order in step 2 with a price seen in step 1 from the past. I don't think there is any backtesting tool that would do what you want.

[DaveE]

...
Lets assume I got an entry signal at the 2pm close of a one hour bar and the closing price of this 2pm one hour bar was 100. Lets assume (its a thinly traded symbol) and the next tick does not come until 2:18 pm and that tick's price in 106.

The logic you are saying would cause the strategy analyzer fill the order at price 106.
I would need it to fill the order at the 100 price.
...

Trader212310, When you say

I got an entry signal at the 2pm close of a one hour bar

You seem to be assuming that in OnBarUpdate(), where I assume your 'signals' are generated, is called at the logical time of close of the 2pm bar, ie at the same time as is shown in Time[0], but it is not.
Assuming there is no session break starting at 2pm (if there was see my post #21), then this is what would happen in Realtime (assume 60 minute Ask data series so the prices in your example are Ask):
At 2:18pm OnBarUpdate() is called. CurrentBar points to the bar with Close[0] = 100 and Time[0] gives the logical Close time for that bar (2pm). But if you Print(DateTime.Now) you will see the actual time is 2:18pm. If you Print(GetCurrentAsk()) this will give 106.
If you Print(Bars.GetOpen(CurrentBar+1) this also gives you 106. This is correct since its the price that a Market Buy will likely get filled at (if broker has enough quantity). Lets say in this OnBarUpdate() your code indicates a signal to Buy at Market and issues EnterLong(), then it will most likely get filled at 106 (not 100). Note that your signal is not being triggered at 2pm but at 2:18pm, so your requirement 1 in post #17

Fill orders at whatever the price was at the moment the order signal was generated

is happening Realtime (and also correctly simulated in Historical see below).

Ideally A Historical Fill Engine (used in both Strategy Analyzer and for processing live strategies while they are in State.Historical) should be simulating what would happen Realtime as closely as possible:
Here is what happens with the same example running in State.Historical:
When OnBarUpdate() is run for Time[0]=2pm: If you Print(GetCurrentAsk()) you get 100. This seems to support Trader212310's argument, but I think it is a mistake (see here).
If you If you Print(Bars.GetOpen(CurrentBar+1)) you get 106, which is the correct value simulating what would have been a Market Buy fill price in Realtime.
If your code does exactly the same thing regardless of Realtime or Historical then your 'signal' should do the same thing as in Realtime (it might not if the logic includes GetCurrentAsk). Lets say your code indicates a signal to Buy at Market and issues EnterLong(), then the Fill Engine will fill it at 106 correctly simulating what would have happened Realtime.
You need to think of your 'signal' code in OnBarUpdate() in Historical as working at the the open of a new bar since that is what happens Realtime.
This means that if your 'signal' is working off Close[0] then your signal is out of date. You could try driving your signal off Bars.GetOpen(CurrentBar+1) instead which is the latest price at OnBarUpdate().
Even NT development sometimes make the mistake of thinking that Close[0] is the latest price. See for example the incorrect rule for Historical Stop price

[Trader212310]

Hi Ray,

Apologies for not being more clear on what I would be using the functionality in post #27 for.

I understand there are ways to custom code an NT8 strategy so that it "sends" orders to 1 tick Bid bars and 1 tick Ask bars accordingly (and fills them at the according Bid or Ask Price) when used in the strategy analyzer
With that in mind, lets assume the following:


1. I have imported all Bid ticks and Ask ticks (that happened both on and in between trades) into NT8.
2. The numbers 100 and 106 mentioned in my previous post were Ask prices, not Last prices
3. I have programed my strategy to send Long entry orders to the 1 tick Ask bars.

The functionality requested in post #27 would be so that my Long entry would fill at 100 instead of 106.

Hope this clarifies why I am looking for this feature.