Difference between realtime & Historical

[andrew56]

First: In realtime using OnMarketData....I printed to Ninja Output 1 minute worth of OnMarketdata....printing each Bid, Ask, and Last....with Price and Time down to millisecond. I then saved this to a text file.

Second: For the same chart, I ran a historical print to Ninja Output...(using two AddDataSeries, one for Bid and another for Ask)....of the data for that same minute.
I saved this to a text file.

I ran a binary text compare between the files (to simply finding differences)....
.....(see attached screen grab for sample side-by-side comparison).

RESULTS:
As Expected:
Last Tick matches COMPLETELY on both prints in both Price and Time


I don't Understand ??????
Many of the Bid/Ask differ in milliseconds....
(this is OK if simply a timing difference between my PC clock & the stream)

In some cases....but not all....Price is different by 1 tick between realtime &
historical.

In a very few instances....a bid or ask from RealTime is completely absent from
historical.

Could be I have done something very dumb...(if so tell me)....otherwise I need help.

[andrew56]

Difference between realtime & Historical

First: In realtime using OnMarketData....I printed to Ninja Output 1 minute worth of OnMarketdata....printing each time OnMarketDataType = marketEventDataType Bid, Ask, Last....including Price and Time down to millisecond.
I saved this to a text file.

Second: Using a 1 tick chart (Last), I ran a historical print for this exact same minute, to Ninja Output...(using two AddDataSeries, one for Bid and another for Ask)....of the data for that same minute...Printing on each BarsInProgress tick for each individual Tick.
I saved this to a text file.

I ran a binary text compare between the files (to simply finding differences)....
.....(see attached screen grab for sample side-by-side comparison)...
............Real Time is on the left Historical on the right

RESULTS:
As Expected:
Last Tick matches COMPLETELY on both prints in both Price and Time



QUESTIONS:
#1: Many of the Bid/Ask differ in milliseconds....
(this is OK if simply a timing difference between my PC clock & the stream)

#2: In some cases....but not all....Price is different by 1 tick between Real Time &
historical. Real Time shows trades lifting the Ask....but Historical shows Last
Between Bid & Ask....

#3: In a very few instances....a bid or ask from RealTime is completely absent from
historical.

Could be I have done something very dumb...(if so tell me)....otherwise I need help.

[NinjaTrader_Matthew]

Do you have the screenshot you mentioned?

What provider/instrument(s)?

[andrew56]

I accidentally posted twice....the screen grab is in the first post

....here it is again....

The data feed is Continuum/NinjaBroker....market is the YM

[NinjaTrader_Matthew]

Thanks.

For my understanding, in your test did you use the marketDataEvent.Time or were you using the system DateTime.Now?

[andrew56]

for Real Time I used
marketEvent.Time.ToString()+" "+marketEvent.Time.Millisecond.ToString()

for Historical is was the usual
Time[0].ToString()+" "+Time[0].Millisecond.ToString() )
for each of the BarsInProgress

[NinjaTrader_Matthew]

Thanks. I will need to test this in real-time when the market opens back up on Monday. I should have answers to all your questions once I can observe/reproduce any differences you're seeing.

[NinjaTrader_Matthew]

Sorry mate, I was not able to finish this today during market hours... Will resume back tomorrow! Thanks for your patience.

[andrew56]

No worries.... I was actually chasing down a completely different issue when I created this piece of comparative analysis code. (I know how the work day gets away)

[NinjaTrader_Matthew]

Here is what I observed:

Two test scripts running at the same time:

Script #1: Calls OnMarketData and filters ask/bid/last
Script #2: Calls AddDataSeries and filters BarsInProgress in OnBarUpdate to filter ask/bid/last

When I compare the two scripts in real-time the results match perfectly.

Now, if I reload Script #2 and just analyze historical data, the results are very similar, however there are cases where the sequence is out of order.



This attributed to receiving multiple updates in the same millisecond, which when compared live vs historical, the results could subtly change as there is no way to know which one happened first. That is expected when using a data provider who only sends updates to a Millisecond (which is currently all of our adapters).

One side effect noted in my testing:

If I combine all of these methods in one script, the results are drastically different, where you get an OnMarketData call in real-time for reach series which are Added to the script. This means when you compare real-time OnMarketData to historical AddDataSeres(), you would see many more calls to to the real-time OnMarketData handler since there is a subscription for each series added via AddDataSeries().

My results are inconclusive with your report: my question is, when you were testing this in real-time, were you running *only* OnMarketData in real-time, or did you have at AddDataSeries in your script? If so, that could attribute the difference.

I've attached the two scripts I used in my testing with 3 files comparing all 3 tests. Please feel free to test this on you end and let me know if you do/do not see the same. (Barring in mind that comparing real-time to historical, there are cases still where the exact sequence can come out of order).

Note: I also included volume in my testing so we could keep track of the change that forced the market data to be called.

[andrew56]

My results are inconclusive with your report: my question is, when you were testing this in real-time, were you running *only* OnMarketData in real-time, or did you have at AddDataSeries in your script? If so, that could attribute the difference.

I will have to go look to make sure....but the script I ran was "quick & dirty" analysis code I was using to debug an entirely different issue. The code had both OnMarketData and the additional historical Bid/Ask data series....but I will have to look to see if they were both active or if I had one commented out.

Lots to evaluate!!!

Thanks for your work....I will spend some time with this and let you know what I find out.

[andrew56]

I rechecked the code....I did comment out the code calling the the additional Bid/Ask data series so it was not running concurrent with the OnMarketData, but I forgot to comment out the AddDataSeries for them. That could have been the issue....but I'm not quite sure I understand fully what happened.
.......
I do an a related question....what is the impact of having "Tick Replay" checked on all this??
Thanks...

[NinjaTrader_ZacharyG]

Hello andrew56,

Tick replay is a property that can be optionally enabled on NinjaScript indicators and strategies which will ensure that the market data (bid/ask/last) that went into building a bar is loaded in the exact sequence of market data events. This guarantees that your indicators and strategies are historically calculated tick-per-tick exactly as they would have been if the indicator/strategy was running live during a period. Tick replay can be enabled for indicators used in Chart, Market Analyzer, Strategies, historical strategy order processing, as well as strategy backtesting with the Strategy Analyzer.

More information about Tick Replay can be found in the NinjaTrader 8 help guide here: http://ninjatrader.com/support/helpG...ick_replay.htm

Indicators and Strategies will only be able to take advantage of tick replay if they have been explicitly programmed to calculate these market data events. If you are a programmer and would like to learn how to use Tick Replay with your custom scripts, please see the using tick replay section of our NinjaScript Help Guide: http://ninjatrader.com/support/helpG...ick_replay.htm

Please, let us know if we may be of further assistance.