IB historical data request limit error

[rassom]

I have a market analyzer with 15 instruments and am connected to IB data feed, real-time.

After having the connection on for a few minutes, I begin getting error messages like this one
"Error on loading chart data for 'XX symbol Default': Interactive Brokers: Ninjatrader has reacehd an IB limitation for historical data requests. IB may reject all historical data requests for the next five minutes: Historical Maret Data Service error message: Historical data request pacing violation."

How can I fix this?

[NinjaTrader_PatrickH]

Hello Rassom,

Thank you for your post.

The error comes from IB and indicates you reached the IB limitation on how much historical data you can load. Please wait 5 minutes and you will be able to request additional data again.

Please let me know if I may be of further assistance.

[rassom]

I have all historical data already downloaded through Kinetick EOD, so all I am using IB for is intraday data, which with only 15 instruments in my market analyzer and 3 open charts (daily, 30 min, 5 min) shouldn't cause these problems (I presume) ?

[NinjaTrader_PatrickH]

Hello Rassom,

Thank you for your response.

Please note that the EOD connection provides access to historical daily data only, allowing you to access Daily, Weekly, Monthly and Yearly data for free.

The amount of historical data that is being requested by your NinjaTrader 7 Platform from Interactive Brokers is reaching a limitation on historical data requests.

I recommend not loading the current amount of instruments on your workspace. A few workspaces that are loaded in succession once the previous workspace completely loads will allow for Interactive Brokers to process your requests more effectively through NinjaTrader, rather then one workspace with several requests at once.

In addition, changing the days to load to a lower setting will also improve upon historical data loading. To change the days to load setting, right click within your chart > Select Data Series > Change the 'Days to load' setting to a lower number. You will need to do this for each instrument.

Please let me know if I may be of further assistance.

[rassom]

Thanks for the tips. I have the following data series settings for the 3 charts:

Daily chart: 365 days
30 min chart: 30 days
5 min chart: 5 days

The daily data comes from Kinetick (presuming I connect to Kinetick every night after the close, which I do), even when only connected to IB in the daytime, right?

[NinjaTrader_PatrickH]

Hello Rassom,

Thank you for your response.

Correct, if you are connecting at night to load Daily historical data from the Kinetick EOD Free connection.
However, if you reload historical data on your Daily chart (right click > Reload All historical data) while connected to Interactive Brokers during the day then that data would be reloaded from Interactive Brokers.

For information on Reloading Historical Data please visit the following link: http://www.ninjatrader.com/support/h...rical_data.htm

Please let me know if I may be of further assistance.

[rassom]

I dont' use the Reload All historical data, so that's not the "problem".

My approx. 15 symbols in the Market Analyzer is the list of stocks I am monitoring on any given day for intraday moves. Having several market analyzers seems going a long ways to solve a simple and unnecessary problem. In other charting packages having 15 symbols in some sort of scanner/watchlist is no problem at all.

Is there a way to fix this? Ie. do you know if IB can open up to allow for more historical data? Or can NT be optimized to not make so many "data calls" to IB? Or something else?

Appreciate your help on this

Thanks,
Rasmus

[NinjaTrader_PatrickH]

Hello Rasmus,

Thank you for your response.

Correct, however when you connect to your data feed provider the data will be loaded for your Charts and Market Analyzers.

Decreasing the "# of bars to look back" property within the Market Analyzer will improve performance and decrease the amount of data requested. You can access this property by right clicking in your Market Analyzer > select Properties > change the '# of bars to look back' property > OK.

For information on the Market Analyzer properties please visit the following link: http://www.ninjatrader.com/support/h...properties.htm

Please let me know if I may be of further assistance.

[rassom]

Decreasing the "# of bars to look back" property within the Market Analyzer will improve performance and decrease the amount of data requested. You can access this property by right clicking in your Market Analyzer > select Properties > change the '# of bars to look back' property > OK.

I did that too

Do you know if IB can open up to allow for more historical data? Any other suggetions?

Thanks very much

Rasmus

[NinjaTrader_PatrickH]

Hello Rasmus,

Thank you for your response.

I am not aware of any option for this, however I would recommend speaking with Interactive Brokers on this item.

Please let me know if I may be of further assistance.

[rassom]

OK, will try to contact IB.

Would I have the same problem with Kinetick real-time data?

[NinjaTrader_PatrickH]

Hello Rasmus,

Thank you for your response.

Kinetick real-time data supports up to 100 instruments. The 100 symbol limit means you can access 100 symbols at any one time, such as in a Market Analyzer. This is not a specific list of symbols, and you will have access to all the symbols for which are included real-time with your subscription, and delayed and historical data for those that you are not subscribed to.

For more information please visit the Kinetick website: http://www.kinetick.com/

Please let me know if I may be of further assistance.

[rassom]

Hi Patrick,

I've now received the following answer from IB:
------------------------------------------------------------------------------------------------
It looks like you are running into our API historical data limitations (http://www.interactivebrokers.com/en...imitations.htm), more specifically into the following limitation:

"Do not make more than 60 historical data requests in any ten-minute period."

Unfortunately there is no possibility to increase this number. Even though you only monitor 15 instruments it's very possible, depending on the number of lookback bars and the size (1 min, 15 mins...) of your bars, that more than one request is required per instrument. Please check the table at the end of the link for a table of the maximum duration of one request depending on the bar size.

The good news is that Ninjatrader may save the bars locally once you download them so if you want to download, for example, one year's worth of 1 minute bars for 15 instruments that will be many hundreds of requests, and it will take a long time to complete.

However, that's only the first day. If Ninjatrader saves the bars you have downloaded somewhere on your computer, and I imagine they do (although you should check with them), the second day when you open your 15 instruments you will only need to download the data that has changed from the first day to the second. You will only need 15 requests for that and you will not run into the problem again.
------------------------------------------------------------------------------------------------

How do I make sure NinjaTrader saves the data it receives from IB to lessen the number of requests to IB?

Also, is there a place I can monitor the requests / number of requests to gauge how much we need to tweak things before not hitting the request ceiling anymore?

Thanks,
Rasmus

[NinjaTrader_PatrickH]

Hello Rasmus,

Thank you for your response.

Please go to Tools > Options > Data > and enable the option for Save Chart Data As Historical > OK.
This will save all real-time data as historical data, but you must be viewing the instrument for the data to be saved.
For information on the Options menu please visit the following link: http://www.ninjatrader.com/support/h...t7/options.htm

Please let me know if I may be of further assistance.

[rassom]

Please go to Tools > Options > Data > and enable the option for Save Chart Data As Historical > OK.

Done. But will that have any "harmful" effects on the data I get from Kinetick EOD?