Big problem with synchronisation between L1 and L2 order book data streams

[ScoobyStoo]

Hi guys,

I think you have a big problem with the synchronisation of the order book data in your L1 and L2 data streams.

Please run the attached indicator to see the problem. For each transaction it prints to the output window the following information:

Transaction Price
L1 Ask Price
L2 Ask Price
L1 Bid Price
L2 Bid Price

You will see lots of line such as the one below in the output:

1.6136,1.6137,1.6137,1.6136,1.6135

Notice how the final 2 numbers are out of synch. This means that the L1 and L2 bid prices did not agree at the time of the transaction. This is a significant problem if you are trying to classify transactions. In this case using the L1 data would result in the transaction being classified 'at bid', but using the L2 data it would be classified as 'between ask & bid'.

I'm using a Zen-Fire feed and have spent quite a lot of time looking at the R | API documentation from Rithmic. The R | API generates only a single market data message for an update to the order book so the synch problem must exist within the NinjaTrader core. The OnMarketData() and OnMarketDepth() method calls relating to a change in the best ask or bid quote should be happening almost simultaneously. Obviously they are not. I'm thinking that possibly the problem lies with the L1 and L2 data being processed on separate threads?

Could you please look into this as a matter of urgency. It's a fairly critical problem for me as I don't know which data stream to trust when classifying transactions.

Thanks

[NinjaTrader_Bertrand]

ScoobyStoo, thanks for the post and sample code, will let it run on my end to check.

[ScoobyStoo]

Thanks Bertrand.

I have always assumed that, with regards to order book data, the L1 and L2 data streams duplicated each other (after all, there's only one order book at the exchange). As a result I have just been using the L1 data stream for transactional data and relying on the L2 data stream for all order book data. I fear that this may have been a big mistake...

[NinjaTrader_Dierk]

NT presents you the events as they come in from the broker (here Zen, RAPI). If you experience a particular sequence of events, then that sequence is imposed by the broker API.

OnMarketData and OnMarketDepth are triggered by different (!) RAPI events.

[ScoobyStoo]

NT presents you the events as they come in from the broker (here Zen, RAPI). If you experience a particular sequence of events, then that sequence is imposed by the broker API.

OnMarketData and OnMarketDepth are triggered by different (!) RAPI events.

Hi Dierk

What are the different RAPI events? To my knowledge the RAPI raises only one (either AskInfo or BidInfo) for every update to the order book.

Thanks

[NinjaTrader_Dierk]

Incorrect. Please consult the RAPI support for details on their interface.

[ScoobyStoo]

Incorrect. Please consult the RAPI support for details on their interface.

Suggest you read this post from Jonathan at Rithmic discussing this issue.

http://www.elitetrader.com/vb/showthread.php?s=&threadid=177461&perpage=6&pagenumber=2

Specifically...

"...when we began developing a feed handler for globex data (think cme) the frequency of best bid/ask data messages from the exchange did not match the frequency of the changes in the best bid/ask prices as seen in the order book. so we decided to ignore the best bid/ask messages published by the exchange and published our own best bid/ask messages based upon the changes to the order book. after a while, globex changed its data dsitribution platform and, i can confirm this later, abandoned publishing best bid/ask altogether. they only publish market depth updates so any best bid/ask prices must be derived from the order book."

This strongly suggests that, since they are generating both the L1 and L2 best ask/bid messages synchronously on the Rithmic infrastructure, either there's a problem with NT's processing of the data or the Rithmic data feeds are out of synch.

Can you confirm which please?

[NinjaTrader_Dierk]

Unfortunately I'm unable to disclose or discuss details of our NT internal code. We coded the adapter according the guidelines provided by Rithmic.

Personally I would not try to "cross reference" L1 and L2 data. I would not expect that both are in sync (just guessing). However, I believe L1 data by itself should give you the right sequence of best bid/ask and trades.

[ScoobyStoo]

Unfortunately I'm unable to disclose or discuss details of our NT internal code. We coded the adapter according the guidelines provided by Rithmic.

Personally I would not try to "cross reference" L1 and L2 data. I would not expect that both are in sync (just guessing). However, I believe L1 data by itself should give you the right sequence of best bid/ask and trades.

Thanks Dierk.

Looks like I'll have to do some recoding to start capturing L1 best ask/bid data.