Announcement
Collapse
No announcement yet.
Partner 728x90
Collapse
NinjaTrader
NT8 Charts Slow to load when using Interactive Brokers Connection
Collapse
X
-
NT8 Charts Slow to load when using Interactive Brokers Connection
I am using NT 8.0.21.1 64-bit and connecting to Interactive Brokers via TWS API for order management and also market data. The charts are really slow to display when switching between symbols using Real-Time data. TWS has no problems. I have a very powerful machine, so CPU & memory should not be an issue. It takes about 2-3 seconds for the charts to switch which is too slow. Can you advise what the problem could be?Tags: None
-
Two or three seconds seems relatively normal especially considering that Interactive Broker's historical data is relatively slow when compared to other feeds. I suspect this is the limitation you're running into. What is the 'Days to load' on your chart that you're switching? To check, right-click on your chart > Data series > What is 'Days to load' set to?
-
Patrick is correct. I had the IB data feed when I first started up with them and it was super slow and it didn't feel like live trading due to the lag. Back then, it was $4/month for the feed. I cancelled the live feed subscription in a day and went with TDA, which is free feed and L2 for a funded account. TDA's not perfect but almost as good as kinetic feed. I had the opportunity to test Kinetic, thanks to the folks at Ninja and it was the best feed I tried in terms of speed, accuracy and scanner (high of day and pre-mkt percent gainers) options. Good luck.
Comment
-
This is not a limitation of NinjaTrader at its core - NinjaTrader's ability to load your workspace is being handicapped by the data feed you're using. Interactive Brokers documents this limitation on their website:
"Important: the above limitations apply to all our clients and it is not possible to overcome them. If your trading strategy's market data requirements are not met by our market data services please consider contacting a specialised provider."
If you find that the speed of Interactive Brokers data is too slow for your uses, I suggest that you try a data feed that does not have this limitation (Kinetick, for example).
Comment
-
Thanks Patrick
This documentation says “Note:
1. At this time Historical Data Limitations for barSize = "1 mins" and greater have been lifted. However, please use caution when requesting large amounts of historical data or sending historical data requests too frequently. Though IB has lifted the "hard" limit, we still implement a "soft" slow to load-balance client requests vs. server response. Requesting too much historical data can lead to throttling and eventual disconnect of the API client. If a request requires more than several minutes to return data, it would be best to cancel the request using the IBApi.EClient.cancelHistoricalData function.“
I would therefore like to see how to setup NT8 to stay within these boundaries to get acceptable performance. Are there any log files or settings that turn on logging so that I could see what API calls are being made and the responses from IB market data servers? It feels like this is an issue many users of NT8 have experienced.
Comment
-
The table in IB documentation says
Step Sizes
A step size is defined as the ratio between the historical data request's duration period and its granularity (i.e. bar size). Historical Data requests need to be assembled in such a way that only a few thousand bars are returned at a time. The following table exemplifies this concept:Should I therefore restrict 1 min charts to 1 day of data on the NT8 side? If so, is there a way I can build up 1 min historical data each day on the NT8 side using a script for my instrument list. Hopefully that would mean NT8 only ever goes to IB market data for current day and all prior period data comes from NT8 historical data and therefore charts would be much faster.60 S 1 sec - 1 mins 120 S 1 sec - 2 mins 1800 S (30 mins) 1 sec - 30 mins 3600 S (1 hr) 5 secs - 1 hr 14400 S (4hr) 10 secs - 3 hrs 28800 S (8 hrs) 30 secs - 8 hrs 1 D 1 min - 1 day 2 D 2 mins - 1 day 1 W 3 mins - 1 week 1 M 30 mins - 1 month 1 Y 1 day - 1 month
I would be happy to spend time with a developer or support person if API logging can be turned on so that I can see where the bottleneck is.
Comment
-
I am doing some more tests on NT8 charts using Interactive Brokers and seeing unexpected behaviour:
Background
I am using TWS 972.1y which is the latest build (though I don't think this makes any difference) which has 6GB of memory allocated to it and only using 297MB during the test.
I have 3 NT8 charts:
1 Day (365 Days to load) - I assume 365 Bars of data
1 Min (1 Day to Load) - I assume maximum of 1440 Bars of data
15 Min (2 Days to Load) - I assume maximum of 96 Bars of data
All 3 charts have Trading hours to "Use instrument settings". For the US stock instruments I am testing with I can see there is full "Last" data for both Minutes and Days on NT8 historical data.
At this current time as I type the instruments are outside of trading hours.
Result - Without an Interactive Brokers connection
I am able to switch very quickly between instruments as would be expected. i.e. NT8 is not able to make a call to the market data and is therefore using locally cached historical data as the data source.
Result - With an Interactive Brokers connection
However as soon as I make a connection to Interactive Brokers, then NT8 charts show the "Loading" message for the same instruments even though it has locally cached historical data. The "Loading" message is longest on the 1 Day chart. This makes me think that NT8 is making an API call to IBKR market data even though it has all the data locally.
In this test, it should not violate the restrictions of the IBKR API that "Historical Data requests need to be assembled in such a way that only a few thousand bars are returned at a time".
So exposing the logging of the API calls in a controlled test would I think identify the cause of the delay, which I suspect is that NT8 is making too many API calls to IBKR even though it has the historical data and/or that it is not potentially adhering to the pacing violation rules
A Pacing Violation1 occurs whenever one or more of the following restrictions is not observed:- Making identical historical data requests within 15 seconds.
- Making six or more historical data requests for the same Contract, Exchange and Tick Type within two seconds.
- Making more than 60 requests within any ten minute period.
1. Is there a way to turn on logging of API calls to IBKR market data to see what is happening
2. Are IBKR's Pacing Violations programmed into the logic that NT8 uses and if so can these be logged so that a user can see the issue
3. What are NT8's rules for when it makes a call to market data for historical dates versus when it uses local historical data
I'd be happy to spend time on this to debug the issue with the NT support team or application developers.Last edited by trader-ap; 04-29-2020, 01:13 AM.
Comment
-
Here is a link with more information about how/when historical is requested from a data provider:There is no way for you as an end user to 'expose the API calls'. When I load an ES 06-20 1 minute chart with 5 days to load 'from scratch' (no locally stored data), it loads in approximately 3 seconds. A daily chart (365 days to load) it takes approximately 12 seconds). If I then switch back to 1 minute, this takes around 2 seconds. Here is a video example:All of these times would be within an expected loading time for any data feed. If you're experiencing delays longer than this, this must necessarily mean there is more complexity to your workspace and/or another variable that is causing further delay. The bottom line here is that the Interactive Brokers API will be relatively slow on historical data requests in general and there is not much you can do beyond reducing the amount of data you're requesting in order to mitigate this expected behavior.
A new Interactive Brokers is in development as we speak which may improve this behavior, however, NinjaTrader is at the mercy of what Interactive Brokers allows via their API.
Comment
-
Patrick
Here is a screencast of my NT8 screens where I switch from one instrument (ZM) to another (WH) and it took about ~20 seconds for the data to load.
Are there any diagnostic tools (log files?) on the NT8 side which can show me what is happening? I have to think there is a problem somewhere on my setup or how NT8 is requesting data from IBKR. I cannot believe that IBKR would implement an API interface used by so many people that would throttle requests to this extent of unusability. Perhaps someone can TeamViewer onto my system to take a look.
Thanks
Comment
-
As explained by email. This issue I am trying to get to the bottom of is where the problem is between NT8 and IBKR. The IBKR documentation clearly says that
“1. At this time Historical Data Limitations for barSize = "1 mins" and greater have been lifted.”
So there has to be a way of getting to the bottom of this problem. Either NT8 is coded inefficiently based on the IBKR specification or there is a problem with IBKR.
I have already reduced the data requests to be within the parameters of the IBKR documentation for bar sizes.
I used to be a software developer and at this time I believe the problem is on NT8 side.Last edited by trader-ap; 04-30-2020, 02:07 PM.
Comment
-
I just got NinjaTrader and I too am having major issues using I.B, it's either not working or extremely slow to load. If I use the WTS on my desktop, which uses the same data feed everything is very fast and as expected.... The Data feed from TD works great on NT, but I only get equities, not sure why I am not getting Futures. I do have futures on Thinkorswim.
Comment
Latest Posts
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by johntraderuser2, Today, 02:34 AM
|
0 responses
6 views
0 likes
|
Last Post Today, 02:34 AM | ||
Started by IanS00, 03-27-2024, 11:24 AM
|
2 responses
12 views
0 likes
|
Last Post Today, 02:00 AM | ||
Started by Mariawilcoxi, Today, 01:32 AM
|
0 responses
5 views
0 likes
|
Last Post
by Mariawilcoxi
Today, 01:32 AM
|
||
Started by gleichnerany, Today, 01:17 AM
|
0 responses
5 views
0 likes
|
Last Post
by gleichnerany
Today, 01:17 AM
|
||
Started by WHICKED, 03-27-2024, 12:33 PM
|
4 responses
47 views
0 likes
|
Last Post Today, 12:28 AM |
Comment