looking for a live data feed that closely resembles IB's historical data

[brynprice]

Good morning/afternoon/evening

I am going crazy as this is the one issue that I still need to figure out before I can go on to make my millions.. haha

The situation is this. I have done years and years of meticulous backtesting using IB historical data. At some point I finally realized that there are significant differences between IB's live data and their historical data (I am aware that this has already been addressed in a number of posts). However I continued to backtest using the historical data - this is what I had always done .. my entire system is built around this data.

I am now in the process of attempting to automate my strategy using IB's API. It is crucial that the live data I receive is as close as possible to that which shows up when one "refreshes" one's charts to get "historical data", if you see what I mean. Whenever I try to trade manually, I always refresh manually before placing an order, for this reason.

Therefore, now that we are trying to code my trading rules, we have looked at the possibility of firing off "historical data requests" at regular intervals in order to mimic what I would be doing if I were trading manually, thus making sure that we are using exactly the same data as that which I have always used for my backtesting.

The problem with this, of course, is that not only is this method very ugly and inefficient, but IB have restrictions in place regarding the number of historical data requests that can be made in a given time period. Therefore this method is not at all practical, and I really need to look for a new data feed.

What I would like to ask is NOT "which is the best datafeed?" or "which is the most accurate datafeed?" but rather: "which datafeed would give me live data that most closely resembles the data used by IB when they form their historical charts?"

Of course, I can try them all out one by one but that will be quite costly.
Might it be the case that they are all pretty close so it doesn't matter?
Any ideas? I'd really really appreciate any ideas at all!

This may sound like a ridiculous question but I hope you can see why I need this. My backtesting has been done, I didn't realize what I was doing at the beginning and it is too late to change now - and the differences can be very significant indeed. There have been so many times in the past that I have lost money on a trade trying to trade off IB's live data feed, only to find that If I had refreshed the charts so that I'd received historical data, there would have been no entry in the first place as the data is totally different!

Thank you

[NinjaTrader_PatrickH]

Hello brynprice,

Thank you for your post.

In general a live data feed that is not aggregated or filtered would have the same values historical and real-time. Our preferred data feed provider is Kinetick which is unfiltered data. However, we do support several data feed providers and brokerage providers. You may wish to ask around (someone can comment here on this thread of course) on other users' experience with the different data feeds.

Please let me know if you have any questions.

[brynprice]

Thanks for the reply Patrick.
do you know if Kinetick is compatible with IB's API?
I wonder has anyone else made the mistake of using IB's historical data for backtesting and then had problems when they've tried to trade live? How similar is IB historical data to the real-time (unfiltered) tick data provided by the likes of Kinetick? Is it significantly different?

[NinjaTrader_PatrickH]

Hello brynprice,

Thank you for your response.

The two connections are compatible in that you could connect to Kinetick first to receive real-time and historical and then Interactive Brokers or vice versa.
Interactive Brokers real-time data through the API is aggregated data, while Kinetick is not. However, the historical data should be similar in most cases.

Try connecting to the Kinetick EOD Free connection to see how the Daily data matches between the two provider. For information on Kinetick EOD Free connection please visit the following link: https://www.youtube.com/watch?v=cjmQ53nJX7I

[koganam]

What I would like to ask is NOT "which is the best datafeed?" or "which is the most accurate datafeed?" but rather: "which datafeed would give me live data that most closely resembles the data used by IB when they form their historical charts?"

That is the same question, better put as: "Which data feed's historical data most closely matches its own live data?"

The answer is: "any data feed that shows raw data, not snapshot or aggregated data."

There are any number of such feeds, and they all advertise themselves as such. For obvious business reasons, the only feed that I ever recommend is Kinetic.