Difference between OnMarket & OnBarUpdate

[photog53]

I ran two separate pieces of code, both on 1 Tick charts. One uses multiple DataSeries for Bid & Ask...the other uses OnMarketData (with TickReplay check).

I compared the output results and found that almost everything matches fine....except for a few items. ...(see attached screen shot)....

I a few instances MarketData code (...using TickReplay) shows "atBid" or "atAsk", while the OnBarUpdate multi-data series code shows "betweenBidAsk".

Is this difference caused by multiple Bids (or Asks) coming in on the same millisecond??

If so, I am assuming Tick Replay has the correct sequence....which then leads to my question. What can I do to get the correct sequence when using multiple DataSeries for (Bid/Ask) in OnBarUpdate??

Thanks.....

[NinjaTrader_DaveI]

Hello,

Thank you for reporting this. So that I understand your screenshot more clearly, can you tell me which column represents which approach (Left/Right column = Tick Replay or multi-series?) Also, in the right column, are there any instances of BetweenBidAsk, or does that never occur for that one?

[photog53]

Sorry for not being clear...
Left side is multi data series....(Last/Ask/Bid) Right Side is OnMarketData

Both were run on separate 1 Tick charts. The OnMarketData code has no multi-Dataseries defined and is a plain call on "Last" .

[NinjaTrader_DaveI]

Thank you for clarifying. I'm not certain what may be causing this based on the information at hand. Could you please share a simplified sample script which would reproduce the issue, so that I can test further?

[koganam]

Sorry for not being clear...
Left side is multi data series....(Last/Ask/Bid) Right Side is OnMarketData

Both were run on separate 1 Tick charts. The OnMarketData code has no multi-Dataseries defined and is a plain call on "Last" .

You may want to put a timestamp on the left hand readings for complete comparison. You may simply be seeing the effect of serial data processing. No matter how much lipstick we put on the pig, and how many fancy adjectives we use to describe it, a single computer, with a single clock, processes data one datum at a time, gated by said clock.

[photog53]

Thanks Koganam....(I always value your thoughts and appreciate the help)....

I will redo the comparison with TimeStamps and take a look.
I am not sure how NT8 downloaded data gets stored when multiple Bid/Ask events occur on the same millisecond....or how that might differ from data actually captured 'live'.

[photog53]

Dave,

Pretty simple code for both indicators.....

OnMarketData Indicator checks for MarketDataType.Last and then pulls the Bid / Ask info from that record...
Ex: if (e.MarketDataType == MarketDataType.Last)
{
eData_price = e.Price;
eData_volume =e.Volume;
eData_time = e.Time;
eData_lastAskP = e.Ask;
eData_lastBidP = e.Bid;
}

The DataSeries indicator simply read BarsInProgress....with CurrentBars[0] being a 1 tick chart looking for "Last".

Ex: if (BarsInProgress == 1) // 1 tick Ask DataSeries
{
myAskPrice = Closes[1][0];
myAskVol = Volumes[1][0];
}
if (BarsInProgress == 2) // 1 tick Bid DataSeries
{
myBidPrice = Closes[2][0];
myBidVol = Volumes[2][0];
}
if (BarsInProgress == 0) // 1 tick Last DataSeries
{
myLastPrice = Closes[0][0];
myLastVol = Volumes[0][0];
}
==================================
For many records, the millisecond timing ensures that the most recent Bid and most recent Ask will be the one that 'matches" across both indicators. The only differences that I see are when we get multiple Bids and/or multiple Asks in the same millisecond.....which begs the question....which of the multiple Bids and/or Asks does NT8 select to populate the record.

Ex: YM gets a "last" record @ price 17310.....
for this example we read 2 Ask records in the same millisecond...and my DataSeries
BarsinProgress code reads the first ask @ 17310 and the second ask @ 17311.
If I use the 'most recent' record read, the I am "above the Ask'.
If I use the first Ask record read, then I am "at the Ask".

What does NT8 do with multiple records in the same millisecond???
...(do you use what you deem to be the 'most recent' of the records read in the same time frame....or....do you look for a 'hit the bid' or 'lift the ask' regardless of sequence within the same millisecond and ignore aboveAsk/belowBid unless that is all there is for that millisecond??
--------------------------------------------------------------------------
In the overall scheme of things, this is probably not a huge issue, but I have clients that will need to know the method used to make this determination. For better or worse, the current "order flow fad' and the addition of millisecond timing, has home retail traders thinking we can replicate the CME..... I am not looking to cause problems or complain, just need to understand how NT8 makes the determination on Bid/Ask when the data flies in and doesn't leave a clear single record trail.
.

[NinjaTrader_DaveI]

Hello,

I've been attempting to create sample scripts using the snippets you provided, but I'm not seeing anywhere near the same results. Can you please take a look at the two sample scripts attached and make any edits needed to bring them in line with the actual scripts you are using to test? Or a new set of sample scripts which will reproduce the issue would work, as well. I imagine my unexpected results come from me not printing in the exact same way as you are in your tests.

I'll also need to know which interval you are using for your BarsArray[1] on the multi-series script.

Thank you.

[photog53]

Dave.... I have to assume that the code you sent was sample only. The DSIndi has a secondary DataSeries of Minute (1)...and no secondary DataSeries for Tick.... Bid or Ask.

I have emailed modified versions of both indicators with your name in the email header.

Both indicators should be run on a 1 Tick Chart..(Price based on 'Last').

The OMDIndi only cares about the primary chart DataSeries, and triggers a Print on each tick.


The DSIndi simply reads thru Bid, Ask, Last....with primary DataSeries [0] "Last" triggering the comparison. The comparison is oversimplified in this code...using whichever Bid or Ask was the
most recent record read and compares that to the primary Dataseries[0][ 'Last.

The only differences will be when multiple Bids and/or Asks come in on the exact same millisecond.
As noted in a prior post, in this case, simply selecting the last sequential Bid/Ask in the secondary dataseries can trigger a difference.

....on yes... I also modified the "IsOverlay" to true....(not necessary to run but it bugs me having a panel two pop up. Also made a few more tweaks....

[NinjaTrader_DaveI]

Thank you for sending that along. I've got the modified scripts, and I will dig back in and investigate further. I'll post back here with my findings or any insights/explanations.

[photog53]

I think the issue we will find with the overly simplistic DSIndi is that when multiple Bid/Ask records hit in the same millisecond, the code will have to parse thru looking for an AtBid or AtAsk hit with 'Last"....rather than simply following the implied sequence and use the last one read.

I spoke to the VeloxPro guys (who do their own order flow BookMap)....about how they handle the various data feeds and trading platforms...(they have an NT7 compatible version of BookMap). I may have already decided how we intend to handle this.....but if you find something interesting by all means pass it along.

[NinjaTrader_DaveI]

I can see the results clearly after your edits, thanks again. I do have a bit of insight in addition to what you are seeing, which may shed more light on the reason you are seeing what you are seeing. Based on what I've found, what we're seeing with the specific code will be expected, but there are potential ways around it.

When you add the Ask and Bid data series, you are subscribing to a feed of data that includes all Ask and Bid updates on historical data. Thus, as in a live market, it is possible for an Ask or Bid price to be higher or lower than the Last traded price at any given time.

With Tick Replay, on the other hand, we are not storing every Ask and Bid update that occurred on historical data, as that would astronomically increase the amount of data stored/provided by Tick Replay, and would be overkill for the majority of use cases. With Tick Replay, each time there is a new Last price update, a "snapshot" is taken of the Bid and Ask price at that moment, and that is the data being provided in OnMarketData(). Thus, whenever a new Last price is provided, that Last price will necessarily be identical to either the Bid or the Ask at that moment.

This means that if the Bid or Ask were to change before a new Last update, you would see a difference in the Last price vs. the Bid/Ask (above Ask, below Bid, between both) when using the AddDataSeries() technique, but that changed Ask or Bid would not be captured until a new Last price triggered (at the Ask or Bid) when using the OnMarketData() technique.This is further guaranteed in our sample scripts by filtering for only MarketDataType.Last in OnMarketData().

You should be able to combine both techniques into a single script by adding your bid/ask data series, then filtering for OnMarketData updates for MarketDataType.Bid and MarketDataType.Ask. This should allow you to utilize OnMarketData while still capturing Bid/Ask events as they occurred on historical data.

[photog53]

Thanks Dave....I appreciate the response!!
(we took off during the holidays....but back at it now)