Market reply vs real time data

[staycool3_a]

Hi

I'm fairly stumped by my findings. I have a strategy that seeks arbitrage opportunities between two instruments. So it's fairly a high frequency strategy.

For today's data. According to market replay data I had 568 orders generated and 588 trades executed. But in real time data I only had 12 orders generated and 13 orders trades executed. My settings are same exact settings for market replay data and real time data.

Settings:

Realtime data = 1 minute data series. COBC = false. So I would imagine the orders are analyzed on tick data.

Market replay data = 1 min data series. COBC = false.

Since I believe market replay data is actually aggregated on 1 second data. In theory, I would expect MORE trades executed on real time data. Since real time data is tick data. So Im ready to pull my hair at this finding.

In my strategy I am utilizing bid/ask data to analyze my data and submit my orders. And my settings are identical. I would had been less surprised if my profit loss had some discrepancy but to have a huge different in # of orders generated and executed just stumps me. Any idea? What is the data source for aggregating market replay data? What service provider do you guys use? Do you think I should set my strategy setting to COBC = true and load 1 second data series to mimic the market replay data?? I'm not trying to work backward. But I'm at least trying to make sense of this finding. Could it be the time series for two different instruments don't overlap one another exactly?? Meaning on can be a second behind the other etc? Also my data provider is CQG. I believe their data is filtered. So if market replay data is unfiltered. Maybe that could also cause issues. Again my strategy is seeking arbitrage using bid ask data. So it's very important to get the right data

[sledge]

How is your SuperDom in Market Replay?

Mine gets all LOST and CRAZY.. Not sure if that is what is happening to your market replay...and bid/ask examinations...

[staycool3_a]

How is your SuperDom in Market Replay?

Mine gets all LOST and CRAZY.. Not sure if that is what is happening to your market replay...and bid/ask examinations...

i have not used superdom. never used it. not sure what the normal behavior of that would be lol.

have you experienced similar problems in real trading vs replay data?

[sledge]

i have not used superdom. never used it. not sure what the normal behavior of that would be lol.

Have you experienced similar problems in real trading vs replay data?

Put it on during your market replay. If the bid/ask go far away from the last - you have a problem. (which might be my problem too). The ladder doesn't move - but the bid/ask is far away and the trades are far away.. It's really easy to see. I originally thought it was a hard disk failure issue - but even my replacement ssd's have had the same issue in market replay.

I don't use bid/ask data in trading - so I haven't seen your issue per se.. but if NT's SuperDom is a mess - then I would imagine further failure down the chain of things.

My laptop is circa Feb 2008.. $800 gem at the time... dual core intel t something processor... pata connection adapter in use.. so it's like sata1.

I remember posts here about NT recording replay data at .25 seconds or something...(I could be wrong)..but it's not totally exact. with live data...

Not sure what you have going on though .. 500+ trades in M.R. vs .13 in real time is a strange beast. (Which is why I suspect some sort of data corruption.......???)

[staycool3_a]

Not sure what you have going on though .. 500+ trades in M.R. vs .13 in real time is a strange beast. (Which is why I suspect some sort of data corruption.......???)

exactly, makes zero sense.

NT any input?

1) What is the data source for aggregating market replay data?
2) Do you think I should set my strategy setting to COBC = true and load 1 second data series to mimic the market replay data?
3) Could it be the time series for two different instruments don't overlap one another exactly?
4) Also my data provider is CQG. I believe their data is filtered. So if market replay data is unfiltered. that could cause issues. Is market replay unfiltered?
5) any idea what is causing my issue?

[NinjaTrader_CalH]

Hello Calhawk01,

1) CQG and Gain are used to record the market replay data. CQG for Futures and Gain for Forex instruments.

2) You can try this test to see what results would be different.

3) I would not expect so in this case. All L1 events are guaranteed to be in the correct order for L1 events

4) CQG is unfiltered data

5) Market Replay will increment by one second but will have all the events in that happened in that 1 second. This is not filtered data but will plot the events for that one second as they were recorded. My recommendation would be to test this with the SampleMACrossOver and if you get the same issue.

[staycool3_a]

All L1 events are guaranteed to be in the correct order for L1 events.

i would not say that. there is clearly something off with market replay data vs real time data.

since market replay data is aggregated using CQG. And CQG is unfiltered data. I would not expect this huge of a discrepancy with orders generated in market replay vs real time. i have no idea what to do right now. if anyone out there knows what to do, please let me know. these markets are immensely manipulated. and at this point i think, the real time data i am getting is being stripped of the arbitrage opportunities but in market replay data they are not. thats the only thing that can explain this. What are the known discrepancies between market replay vs real time?

[NinjaTrader_Brandon]

Hello calhawk01,

Thanks for your reply.

I would like to look further into the data you are testing on my end.
What instrument and expiry are you testing?
For what date?

[staycool3_a]

Hello calhawk01,

Thanks for your reply.

I would like to look further into the data you are testing on my end.
What instrument and expiry are you testing?
For what date?

well in order to test, you would need to run a strategy for the entire day.. for example today, and then download market replay data tomorrow, and run the same strategy.

what is your email address? I will send you a sample idea for the strategy. unless you already have a strategy that utilizes bid/ask data from two instruments and then executes the orders to buy/sell both instruments.

[NinjaTrader_Brandon]

Hello calhawk01,

Thanks for your reply.

You can email me at platformsupport[at]ninjatrader[dot]com
In the message include a link to this forum topic
Please still provide information about a date and instrument (including expiry) that you are testing this on as what I want to do is verify the bid and ask data is good for that day since the order execute based on bid and ask data.