OnMarketData : Last price between Bid and Ask?

[braincell]

I'm developing my own indicator looking at the OnMarketData. I know it's not supported but it would be good if someone who used OnMarketData extensively could shed some light on this issue.

I am getting some prices with DataType = Last which are above Bid and below Ask. How is this possible?

What is DataType.Last supposed to be?
Is it the last price at which a trade occured or can it in fact be the lowering of the Ask and the raising of the Bid? Or is it just that the Bid/Ask DataTypes do not get updated fast enough?

I am using eSignal data feed and it's supposed to be pretty accurate. I compared it to Time and Sales and the outputs are exactly the same. So can someone tell me why Trades are happening between Bid and Ask?

Here is a part of my output for example:

OnMarketData ************************************* Time = 17:46:55:732
DataType = Bid
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.68
CurrentBid = 566.66
New Bid = 566.68
Volume = 3
******************************************************
OnMarketData ************************************* Time = 17:46:56:644
DataType = Ask
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.68
CurrentAsk = 567.01
New Ask = 567.01
Volume = 1
******************************************************
OnMarketData ************************************* Time = 17:46:58:336
DataType = Bid
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.91
CurrentBid = 566.68
New Bid = 566.91
Volume = 1
******************************************************
OnMarketData ************************************* Time = 17:46:58:336
DataType = DailyVolume
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.91
Volume = 1481980
******************************************************
OnMarketData ************************************* Time = 17:46:58:337
DataType = Last
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.91
CurrentLast = 566.79
New Last = 566.96
Volume = 100
Bid = 566.91 ___ Ask = 567.01
%%% %%% %%% ERROR : Price is between bid and ask!

******************************************************

[NinjaTrader_Austin]

braincell, OnMarketData() is supported. The last data type is for actual trades taking place. It reports the price and volume at which the trade took place. These trades are supposed to occur on the bid or ask. It could be something going on with the price feed, or it could just be a trade reported late.

[braincell]

Ah ok. Well the Time and Sales window shows the same data, ie Trades happening between bid and ask, so do you think the most likely reason is eSignal doesn't update Bid/Ask quickly enough or could it be something else? Maybe I should ask that on their forums.

[NinjaTrader_Austin]

braincell, that could be the case. I would ask them for clarification. I've never seen this on a Zen-Fire feed, for example.

[RJay]

I'm developing my own indicator looking at the OnMarketData. I know it's not supported but it would be good if someone who used OnMarketData extensively could shed some light on this issue.

I am getting some prices with DataType = Last which are above Bid and below Ask. How is this possible?

What is DataType.Last supposed to be?
Is it the last price at which a trade occured or can it in fact be the lowering of the Ask and the raising of the Bid? Or is it just that the Bid/Ask DataTypes do not get updated fast enough?

I am using eSignal data feed and it's supposed to be pretty accurate. I compared it to Time and Sales and the outputs are exactly the same. So can someone tell me why Trades are happening between Bid and Ask?

Here is a part of my output for example:

OnMarketData ************************************* Time = 17:46:55:732
DataType = Bid
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.68
CurrentBid = 566.66
New Bid = 566.68
Volume = 3
******************************************************
OnMarketData ************************************* Time = 17:46:56:644
DataType = Ask
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.68
CurrentAsk = 567.01
New Ask = 567.01
Volume = 1
******************************************************
OnMarketData ************************************* Time = 17:46:58:336
DataType = Bid
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.91
CurrentBid = 566.68
New Bid = 566.91
Volume = 1
******************************************************
OnMarketData ************************************* Time = 17:46:58:336
DataType = DailyVolume
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.91
Volume = 1481980
******************************************************
OnMarketData ************************************* Time = 17:46:58:337
DataType = Last
e.MarketData.Ask = TheAsk = 567.01
e.MarketData.Bid = TheBid = 566.91
CurrentLast = 566.79
New Last = 566.96
Volume = 100
Bid = 566.91 ___ Ask = 567.01
%%% %%% %%% ERROR : Price is between bid and ask!

******************************************************

Hi braincell,

Been there, done that.

Go here and read post #1 for analysis of the problem and how to eliminate it.

http://www.bigmiketrading.com/ninjatrader-programming/6025-ninjatrader-providing-bad-price-volume-data-onmarketdata.html

RJay

[NinjaTrader_Austin]

RJay, braincell never posted his code. Depending on what the code is, he might be printing the right values from his data feed.

As Ray mentioned in your thread on BMT, it is advised to "store local variables and update them as OnMarketData() is fired based off the data type that was changed."

Actually, I'll post Ray's entire response here:

"You are correct that if you are to use OnMarketData() and you wish to do value comparisons you should store local values in your indicator/strategy. This is expected and has always been the case and is by design. We have advised developers of this in the past but in further research, I see this critical point should have been clearly documented and it will be going forward.

Here are a few points of clarification:

-OnMarketData() is guaranteed to be called in the correct sequence for all level I market data events received from the underlying data source (no data is ever dropped)
-e.MarketDataType determines what data type (bid, ask, last etc…) changed and triggered the call to OnMarketData()
-You should store local variables and update them as OnMarketData() is fired based off the data type that was changed
-e.MarketData can be ignored (if doing comparisons) since values can and likely will be ahead of the data type/value that triggered OnMarketData()
-This is because the underlying MarketData object can be updated on different threads and reflects the most recent values received (you of course will get a OnMarketData() call for these events subsequently)"

[braincell]

Hi RJay! During my research I googled that thread of yours and another one on the NT forums as well, so I'm already aware of that. I also tried both methods of getting Bid/Ask for comparison and they seemed to be in sync.

What I find is that eSignal has the exact same data in Time&Sales as my Output. I posted on the eSignal forums asking why some items in the Time & Sales are posted as a Trade which is at between Bid/Ask and I am awaiting for their reply. I suspect it's a problem with their feed. Do you think that's unlikely? Could it perhaps be dropped packets due to a slightly slower connection speed I have?

@Austing, thanks for the summary. I did google before asking though, so I was aware of that information before.

[NinjaTrader_Austin]

braincell, all L1 updates should be timestamped, so you can use that to compare times to see if something came in out of order. I highly doubt packets were just dropped.

[braincell]

You mean e.Time ? Hmm ok I did not check that, that's a good idea. I'll compare it to system time and see on monday when I get live data again.

[RJay]

Hi RJay! During my research I googled that thread of yours and another one on the NT forums as well, so I'm already aware of that. I also tried both methods of getting Bid/Ask for comparison and they seemed to be in sync.

What I find is that eSignal has the exact same data in Time&Sales as my Output. I posted on the eSignal forums asking why some items in the Time & Sales are posted as a Trade which is at between Bid/Ask and I am awaiting for their reply. I suspect it's a problem with their feed. Do you think that's unlikely? Could it perhaps be dropped packets due to a slightly slower connection speed I have?

@Austing, thanks for the summary. I did google before asking though, so I was aware of that information before.

Hi braincell,

That make perfect sense to me. If data is the same in both places, I would look to the source.

I have built many indicators that utilize OnMarketData or OnMarketDepth.

Good for you to be pursuing understanding and creating your own indicators.

Bravo!!!

RJay

[braincell]

Glad somebody agrees! Your threads were very helpful, I've been reading some others as well for quite some time.

By now I have really great indicators for correlation pressures. You know how they talk about "Day traders are evil!" and yet they want to decrease market fragmentation? Well, I'm glad to do it for them, especially if that makes me some moneys. See all the stocks going down, and one is lagging behind, jump on the short, voila, it's sure to follow the others and then some. I've created a unified indicator for this which is like 10 in 1 with normalized value outputs, and some parameters for tweaking during the day. Only problem is I may need more computers to run more than 60 stocks with it. I found out how to grab data from NT and put it into my own application, so I'll probably continue coding indicators in Visual Studio with my own database and distributed computing. Fun stuff.

[JoshDance]

So can someone tell me why Trades are happening between Bid and Ask?

I don't know any details about what you're doing, but the last traded price is unrelated to the current bid and offer. The last traded price can be below the current bid, above the current offer, equal to the current bid or offer, or in between if there is a 1+ tick spread. The two are unrelated. Once a transaction takes place (the last traded price), offers and bids can be pulled and added. Where they currently are is not a function of the prior transaction. If this is not clear, then I can give examples on your chart to look at or explain differently.

Either way, this does not mean that your code is correct of course, but just be aware.

[braincell]

Yes, I am aware of that. It's just that the trades are happening between bid/ask but the bid/ask don't move until a few seconds later. I'm pretty sure my code is correct, I can post it, it's nothing fancy.

I'm not sure what you mean they can be in between if there is a 1+ tick spread though. Are you saying if Bid = 10 Ask = 15 that a trade can happen at 12 regardless? Shouldn't the bid/ask be updated before the trade, meaning there's actually an offer at 12? How can anyone buy/sell at 12 if there are no bids/offers?

For example, this was in my Times and Sales window of eSignal.

You can see this:

Bid: 566.56
Ask: 566.83
Trade: 566.80 and 566.82

Date          Time           Exchange    Type     Price     Size    Sequence ID    Condition    
12/08/2011    06:06:58 PM    NASD        Trade    566.83    100     786021                      
12/08/2011    06:06:58 PM    PSE         Trade    566.83    100     786019                      
12/08/2011    06:06:58 PM    PSE         Trade    566.82    100     786018                      
12/08/2011    06:06:57 PM    NMS         Ask      566.83    2       22226653                    
12/08/2011    06:06:53 PM    BSE         Ask      566.83    1       22222870                    
12/08/2011    06:06:51 PM    NASD        Trade    566.80    500     785832                      
12/08/2011    06:06:51 PM    BSE         Ask      566.83    2       22221350                    
12/08/2011    06:06:50 PM    NMS         Bid      566.56    3       22220950

The bid/ask were not pulled up/down after the trades. You can see the ask at 566.83, then a trade at 566.80 and then again an ask at 566.83. If there was a trade there, I would expect to first get an Ask update, unless the trade happened so quick (as soon as the ask appeared) that the ask was not updated. Are you saying this is possible?

Right now I posted at eSignal forums and they're saying that it is strange, and they will look into it.

[JoshDance]

You can see the ask at 566.83, then a trade at 566.80 and then again an ask at 566.83. If there was a trade there, I would expect to first get an Ask update, unless the trade happened so quick (as soon as the ask appeared) that the ask was not updated. Are you saying this is possible?

I have no idea if it's possible. But I do know that all the time I get trades reported as "between the bid and ask" on the time and sales window, and this is due to either a poor algorithmic check of the bid/offer related to the transaction time (namely, checking only the current state AFTER the transaction and not before), OR a data synchronization issue at the data provider level or the exchange level.

If you are indeed checking all events which occur, and this is what you're getting, then it would seem that either (1) the feed is not correctly matching the exchange data, or (2) the time resolution of the events is not granular enough to capture the events which occur in the correct order.

I observe, particularly in very illiquid market conditions, a spread of, say, 5 ticks, and a transaction goes off in the middle of the two, and the bid/offer never visibly change. A computer program is obviously instantaneously buying any offer which appears below the current best offer, and instantly it's gone, and through some bottleneck, the change never even appears on the DOM. We're talking sub millisecond transaction times probably, and whether this is the cause of your problem or not, I'm not sure, but it can certainly lead to this type of behavior. Whether you should see that programatically in your feed though, I have no idea.

[braincell]

I think so too, that it's a comobination of the two: a computer buys it real quick and the feed updates aren't granular enough. Therefore, to count the trades at bid or at ask, I just average the bid/ask price and treat those above the average as at ask, and those below as at bid. This works well enough for my indicator.