Data Cleaning

[davidt07]

I'm getting my DataFeed currently through from IQFeed and I am using Ninja Trader 7.

I also have a subscription to eSignal and I have noticed many difference in the tick charts between the two.

I am wondering, is it possible that Ninja Trader interface somewhere is doing a clean up of the data streaming in in Real Time?

Notice that these do not appear to be CID (Corrections, Insertions & Deletions) sent from the exchange as I am watching both charts in real time.

Notably, I've seen this happen quite a lot over this week on YM Futures Dec 2011.

Anyone can give me a heads up on what might be going on??

Thanks

David

[NinjaTrader_Matthew]

Hello David,

Different data feeds produce different charts, especially when using tick based intervals vs time based intervals. Market data vendors each employ various methods for tick filtering, throttling and time stamping. As a result, no data stream is 100% identical and thus can cause subtle differences in charts. Since NinjaTrader supports many of the leading brokerage and data feed technologies, it is highly likely that two different data feeds will have minor differences when plotting the same market and time interval.

[davidt07]

Hi Matthew,

Thank you for this, I'm just getting into using RangeBars and I am using it in conjunction with TimeBased charts for a strategy.

Are the different ways that data vendors filter their data available for public knowledge? I am trying to find a way to filter the raw data so that I can find a common ground between multiple vendors.

I mean just Tuesday for some reason there were so many more ticks from IQFeed at around 10:43:43 than eSignal tick data that caused me to have an extra 5x RangeBars(3) on the YM Dec 2011 Futures.

I haven't checked yet with how RangeBars are formed because Ninja Trader uses strict RangeBar rules, (I'm trying to get the RangeBar Alt to work in NT 7 right now) but I am wondering, will I see the same RangeBar bars in NT and eSignal even despite the difference in tick data?

Thanks

David

[NinjaTrader_Matthew]

Hello David,

There is no public documentation on how this data is filtered. You may want to check with each vendor individually for more information.

Your range bars may change depending on the ticks that are stored on the data servers.