Canceling an Order after a level change. (Another issue with Simulation Engine)

[iantg]

I found what I believe to be an issue with the Simulation Engine regarding how orders are able to cancel after a level change. Here is what I am observing:

1. I submit an order to go long and my limit price = the current level's bid price which is 2370. My goal is to only get filled on the current level and not get a fill on the level below which will start at bid = 2369.75.

2. I observe a level change where the new bid price is 2369.75, so the level moved down.

3. I didn't get filled on the level that I wanted which was bid =2370 and now that we are on one level below (bid = 2369.75) if I get filled here, I will immediately be down 1 tick. So I submit a request to cancel my order.

4. The cancel is confirmed and I do not get filled at this new level.

End

So there are a couple of things that I observe in this series of events that I would characterize as wrong. If there is a level change down, then I would expect to have been filled on the original level prior to the level change. I would expect that because in order for the level to decrease from 2370 to 2369.75, every bid limit order resting would have to be filled and there would have to be 0 contracts at this level. Then and only then would the level drop. So If I had a resting order at the original level of 2370 it should get filled prior to the volume going to 0 and the next level coming into play.

I also seriously doubt that the mechanics of an exchange would allow for a change in the bid / ask price (AKA a level change) prior to filling every order at the prior level (for the side that would have had to clear 100% to create the level change in the first place).

So it seems that the assumption is, that a level change (lowering of the bid price or raising of the ask price) can occur prior to every order being filled at the previous level. And if one is quick enough they can submit a request to cancel their order prior to it getting filled.

I am not 100% sure about the exchange mechanics but I am 90% sure that this underlying assumption made by NinjaTrader in their Simulation engine is not the way the market will behave. I mean it would be great, if there is a lag (After a level change but prior to a fill occurring that I didn't want), but I don't thing there is one, and further I don't think there is any possibility of being fast enough to place a cancel order and have it go through prior to the level change.

I have some code I wrote that will capture these events and highlight the issue, and I will be glad to share if needed. But can someone from NT support confirm that they are aware of this behavior and speak to the questions that I have raised. I don't think the scenarios I have outlined are possible, but if they are, please explain what I am missing.

Thanks,

Ian

[NinjaTrader_RyanS]

Hello,
Thanks for the note.

In the situation described, it sounds like the simulation engine is operating as it is intended. While the Sim engine is a 'best guess' for order fills and execution, it uses multiple data points to calculate order fills.
In your scenario you had a limit buy order at 2370, which means you want to buy at 2370 or any lower price available. Then the bid ask spread went to 2369.75, 2370 which meant the ask was at your limit price. However, its likely there were not enough sell orders at that level to fill your order. This is where the simulation engine uses depth of market, approximate position in queue etc. to determine whether or not you would be filled in a live market scenario.
For example, if your approximate position in the queue was 100, and there was a depth of market at 2370 of 50 orders, you would likely not be filled.

However, I'll reiterate that the sim engine is a simulation of what would occur in live markets. While it has been edited to be as accurate as possible, nothing can truly recreate a live market scenario.

If trading live, the market will determine whether or not your orders are filled, so there is nothing in NinjaTrader itself that can be done to improve/change this.

Please let me know if you have any additional questions.

[iantg]

Ryan,

Thank you for looking into this for me. Here is where I need some clarification.

1. My limit order is at 2370, and I submit it on the level where the bid price = 2370. So I am now in the queue on this level. Let's say at the time I submit the order the bid volume is at 200, so I am going to be number 201 in the queue.

2. Now at some point the price level decreases by one unit, so now the bid is at 2369.75.

Here is the issue / question: I was in the queue at the previous level at # 201 when I submitted my order.

Primary Question: My understanding is that In order for a new lower level to come into existence the entire book of bids from the previous level has to clear 100%. Can you confirm this? The same would apply the opposite way. In order for a new higher level to come into existence, the entire book of resting asks would have to clear at the previous level. Only when the applicable bid volume goes to 0 can a level change go down, and only when the applicable ask volume goes to 0 can a level change go up. Are there any exceptions here that I am not aware of, or can you confirm that I have this part right?


So if I submit a long order at the bid at 2370 and I confirm that my order is working at this level, and I am in line in the queue. Then the only way possible for a new lower level to be created at 2369.75 is if my order is first filled because this lower level only exists by a function of the resting bid volume at 2370 going to 0.

If this is true, then how then can the level change down, not fill me, and then allow me to cancel before I get filled at the new lower level?

I completely understand that If my limit price is for 2370 this implies 2370 or better which 2369.75 would be inclusive of this, but as I am submitting my order while on the current level = 2370 how is it possible for the level to change and not first fill me at 2370? Could you explain the mechanics of the exchange / in connection with the assumptions NT is making just so I understand this part?

Ian

[NinjaTrader_RyanS]

Hello,

Thanks for the response.

In this case, the previous bid/ask would have been 2370/2370.25 when the buy order was placed at 2370.
For the level to move, it means all the buy orders at 2370.25 were filled, thus moving the new ask to 2370, and the new bid to 2369.75.

If the new ask volume is lower than your approximate position in queue, your order would likely not be filled. So if your position in the queue at 2370 was 201, and the new ask volume is 73, your order would not be filled immediately.

To isolate it to your question, the bid ask moves down when all of the buys at the ask are filled. This means that your order at the bid will now be at the ask, and will be dependent on the ask size at that price, and your position in the queue.

Please let me know if I may clarify further.

[iantg]

Hello,


For the level to move, it means all the buy orders at 2370.25 were filled, thus moving the new ask to 2370, and the new bid to 2369.75.


Please let me know if I may clarify further.

Ryan,

I think the above is the point where we are seeing things differently. For market orders only, the long positions are served by the ask side, and short positions are served by the bid side. But for limit orders it is the opposite. It I place a limit order to buy at 2370 then I am joining the bid volume queue and waiting on the bid price of 2370, I am not concerned with the ask price or ask volume in this scenario. If the bid volume goes to 0, then the price level will move down. Alternatively if the ask volume goes to 0 the price will move up. This works counter intuitively to how we typically think about the market (From the perspective of market orders).

But keep me honest here, this is my understanding of the mechanics of it. For example: https://money.stackexchange.com/ques...-ask-size-help

But in terms of the scenario outlined from my original question, what I am seeing is that I join the bid queue at 2370 and don't get filled, but then all the bid volume goes down to 0 , then at the new level of 2369.75 I would theoretically get filled immediately by virtue of the fact that every resting bid order in the queue at 2370 (including mine) would get filled prior to the level breaking and moving down. But I am still able to cancel ahead of the fill that would occur as a function of the price moving down. (This last part is the issue.)

Now I know this sound confusing and or impossible. I can provide use cases fairly easy to illustrate this and privately share some of my code that shows this behavior. But let me provide my theory on what I think is occurring under the hood of the simulation engine.

1. Due to well intentioned and conservative settings, a fill will typically need additional time, signals and reference points prior to changing states from working to filled. In the scenario I am describing above, the fill technically occurs at the bid = 2370 level, but since I am in the very back of the queue the safe assumption to make is that only after the level changes to 2369.75 should I get my fill confirmed. So we need the level to change down as a first step.
2. Once the level changes down, there is likely a wait period of several milliseconds of some deliberately placed delay to simulate the latency of the exchange.
3. I deliberately wrote a code that only executes cancellations at the precise moment that a bid / ask level changes. I run this code the OnMarketData event handler so it moves as fast as possible. I believe that cancellation commands are given high priority in NT, so as a result my cancellation request is being executed ahead of the more conservative assumptions for fills. So this is why I am able to cancel ahead of time. If I removed my cancellation from my script in every case I would get filled a few milliseconds later anyway.

This is what I am observing. I think, that unless I am completely missing something, this behavior is likely a bug that I found due to the fact that canceling occurs faster than filling. So if filling and canceling are occurring on conditions that are both true, canceling will happen ahead of filling.

What I am hoping to see is only cancellations go through that would occur in the real market, but instead I am seeing cancellations go through that I don't believe would ever occur in the real market. My goal in doing this study it to see under what conditions I can cancel orders, and this is a case of one I believe that is not working properly.

Let me know if this sounds about right, or there is something else in play here. But if I am right about this, then I would like to report this as a bug and have it addressed in the next release.

Thanks,

Ian

[NinjaTrader_RyanS]

Hello Ian,

Thanks for the response.

Testing fills down to a millisecond level pushes the limitations of the sim engine programming. The sim engine is designed to replicate live market fills, however it is unable to do so to the accuracy and precision you are testing with your backtest.

If using this strategy on a live exchange, it would be near impossible to determine exact fills to the level of accuracy you are requiring.The sim engine might be considered an unreliable representation of success/failure when scalping inside the bid ask spread compared to what would realistically happen in live trading.

There are simply too many factors in a live market to reliably simulate this type of strategy, which would most likely require incredibly accurate data. For this reason, our development team isn't considering this a 'bug', but instead an intended limitation of simulation order execution.

Please let me know if I may be of further assistance.

[iantg]

Ryan,

Thanks for looking into this for me. I will follow your recommendation and just consider this a limitation and not a bug, as I have likely found something that exceeds the limits of the simulation engine. I think my working theory is likely correct in terms of how the events are occurring and why I am able to sneak a cancel in after a fill should have occurred.

Thanks,

Ian