How to auto manage IB-Gateway daily restart without causing NT8 to stop working

[NinjaTrader_Jim]

Hello timko and NT-Roland,

We used to have to use IB Gateway to achieve 24/5 connections, because TWS would perform daily disconnects. However, IB Gateway should no longer be needed for 24/5 connections because NinjaTrader should be able to automatically reconnect with both. I am looking into both TWS and IB Gateway related reconnect issues.

In the past, only 24/5 connections were possible IB Gateway and we would always get disconnected over the weekend for server maintenance. I had asked our Quality Assurance team to check source code if there is a timer involved with the disconnect process. There is a timer that will stop reconnecting if the connection cannot be completed in 3 minutes. Thus, we cannot expect the connection to re-establish over the weekend.

I am currently testing to reproduce the reconnection issues that were reported last week.

[pmn100]

I have started having problems with this test build, though it seems to work for the most part I've had a couple of instances of failure to update price data after a disconnect even though Interactive Brokers connection has the green light. Occured yesterday evening and early this morning, once after the IB server restart and once after my daily TWS restart as set in the configuration options. Will send trace and logs to Jim.

[Coral]

Do we have any news or timelines on this until when it will be resolved? Still cannot use our 24/5 strategies (what was the reason why we bought NT).

[pmn100]

Has been working for me for the most part but this evening again, after my automatic restart of TWS at 20:00, reconnected to IB fine but no real time data flow occuring. The following errors in my trace at this time:

2021-05-04 20:01:07:222 (My Interactive Brokers) IB.Adapter.OnError: id=-1 errorCode='MarketDataFarmConnectionOk' errorMessage='Market data farm connection is OK:usfarm'
2021-05-04 20:01:07:222 (My Interactive Brokers) IB.Adapter.ConnectionStatusHandler: ibStatus=Connected historical=Connected marketData=Connected
2021-05-04 20:01:07:222 (My Interactive Brokers) IB.Adapter.OnError: id=-1 errorCode='MarketDataFarmConnectionOk' errorMessage='Market data farm connection is OK:eufarm'
2021-05-04 20:01:07:222 (My Interactive Brokers) IB.Adapter.ConnectionStatusHandler: ibStatus=Connected historical=Connected marketData=Connected
2021-05-04 20:01:07:274 (My Interactive Brokers) IB.Adapter.OnError: id=492 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:492'
2021-05-04 20:01:07:309 (My Interactive Brokers) IB.Adapter.OnError: id=493 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:493'
2021-05-04 20:01:07:714 (My Interactive Brokers) IB.Adapter.OnError: id=494 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:494'
2021-05-04 20:01:07:714 (My Interactive Brokers) IB.Adapter.OnError: id=495 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:495'

Had to disconnect/reconnect IB connection in NT to get realtime data flow again.

[pmn100]

Another failure today to restart realtime data from IB after my daily TWS restart. API connected but no realtime market data flowing.

2021-05-07 20:15:44:309 (My Interactive Brokers) IB.Adapter.OnError: id=674 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:674'
2021-05-07 20:15:44:310 (My Interactive Brokers) IB.Adapter.OnError: id=675 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:675'
2021-05-07 20:15:44:332 (My Interactive Brokers) IB.Adapter.OnError: id=676 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:676'
2021-05-07 20:15:44:353 (My Interactive Brokers) IB.Adapter.OnError: id=677 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:677'
2021-05-07 20:15:44:383 (My Interactive Brokers) IB.Adapter.OnError: id=678 errorCode='CantFindEid' errorMessage='Can't find EId with tickerId:678'
2021-05-07 20:15:44:655 (My Interactive Brokers) IB.Adapter.OnError: id=-1 errorCode='MarketDataFarmConnectionOk' errorMessage='Market data farm connection is OK:eufarm'

[NinjaTrader_Jim]

Hello everyone,

I have been testing on various nights and had reproduced the issue once, but the results were inconclusive for further action.

We are needing further details for cases, so I would appreciate it if those that are testing, write in with their log and trace files and note the specifics on when the disconnect occurred which could not be recovered.

Please also include the following so we can all necessary information present in the reply:

1. Confirmation if you are ever able to reproduce the issue when changing the restart time to try and make the issue more reproducible, and out of how many times tried you encountered the issue
2. Confirmation if this is happening only with the nightly reset
3. Confirmation the issue was reproduced with the test build and TWS/IB Gateway 978.2c
4. Confirmation that the account was not logged into from an external source like a mobile application
5. Details if any positions were held (Please include screenshot of Positions tab of Control Center.)

Providing direct answers to questions #1-#5 in email will help to consolidate this information.

I am also arranging next week to set up a virtual machine and to lock our Interactive Brokers paper trading account so I can perform extended testing over a 24/5 period.

Thanks for the continued patience regarding this matter, and I will follow up again after the extended testing to report my findings or if I have any other news to share.

[PWidera]

Is there any update on this issue?

[NinjaTrader_Jim]

Hello PWidera,

We are still investigating this internally. We do not have any new information to share at this time.

[bc24fl]

Can we get an update where we are at with this? I too was affected by this issue and as a result moved to TWS v 978c awhile ago but today received an error from Interactive Brokers that this version is no longer supported and exited out forcing me to download the latest "supported" version by NT of TWS v.985.1g. That said, I would prefer running IB Gateway and hence wanting an update if there is one so I can switch back to Gateway.

Thanks

[NinjaTrader_Jim]

Hello bc24fl,

We updated the supported version of TWS and IB Gateway to use with NinjaTrader to TWS 985.1g and IB Gateway 981.3c.

We have had far fewer reports about incomplete reconnects using these versions while ensuring that inbound and outbound exceptions are added in Window Firewall.

Search “Firewall” on windows toolbar → Click Firewall & Network Protection → Click Advanced Settings → Create inbound and outbound exception.

Program paths for these programs when installed from our Connection Guide can be found below.

• TWS 985: C:\jts\985\tws.exe
• Gateway 981: C:\jts\ibgateway\981\ibgateway.exe

Connection Guide - https://ninjatrader.com/ConnectionGu...nnection-Guide

If you are still having issues with this configuration, please write in to platformsupport [at] ninjatrrader [dot] com with your log and trace files. There were some changes made to our internal build that may improve the situation further if you are still affected.

[pmn100]

Same thing happened to me. IB no longer supporting the TWS version that NT is telling us to use. Have gone back to the stable version of 981.3c

[NinjaTrader_Jim]

Hello pmn100,

You have been working closely with me and our QA team. We were under the impression that the situation greatly improved after updating to the versions of TWS/IB Gateway we have listed in our Connection Guide, and also applying inbound and outbound firewall exceptions to TWS/IB Gateway.

We are seeing some other feedback regarding TWS/IB Gateway versions too.

Could you elaborate on your latest comment where Interactive Brokers is telling you to update again?

We would not expect any auto updating take place when installing the TWS/IB Gateway versions from the connection guide either.

Please fill me in on any details you may provide. If there is an important reason Interactive Brokers is giving you to not use these versions, we will want to discuss that with Product Management.

[fernlicht]

Yes, working with you was perfect and we solved the issue and it worked( you can read this in my last post) This morning EU time when I checked the Server TWS was down/closed and a prompt was displayed that I should download the latest TWS since this one is not supported any longer… and as you may remember we installed latest TWS together on our server Again, no blaming on NT,
just again a forced shutdown and installation of IB.
I will run this setup the whole next week and revert back.

[pmn100]

Hello pmn100,

You have been working closely with me and our QA team. We were under the impression that the situation greatly improved after updating to the versions of TWS/IB Gateway we have listed in our Connection Guide, and also applying inbound and outbound firewall exceptions to TWS/IB Gateway.

We are seeing some other feedback regarding TWS/IB Gateway versions too.

Could you elaborate on your latest comment where Interactive Brokers is telling you to update again?

We would not expect any auto updating take place when installing the TWS/IB Gateway versions from the connection guide either.

Please fill me in on any details you may provide. If there is an important reason Interactive Brokers is giving you to not use these versions, we will want to discuss that with Product Management.

Hello Jim

There has been some improvement as there have been some fixes in recent NT versions but I'm still having failed reconnects every so often. According to Alan (Q&A?) there are a few more fixes that exist on internal builds but I'm still waiting for these to be put into a public release. I've asked Alan a couple of times when we might expect that to happen but haven't had a reply.

The fix I'm specifically talking about is:

NTEIGHT-14865 NinjaTrader won't reconnect to Interactive Brokers Gateway after Auto-Restart occurs. “The requested name is valid, but no data of the requested type was found”.

This error I see in my trace files whenever I get the CantFindEid error, so I think they're related. I believe this fix is on internal build 25.0 I was running until it started giving errors with FXCM and I was forced to roll back to NT 24.3. Running build 25 with a later version of TWS is what seemed to cure all issues for me, but I can't run 25.0 because of the bugs with FXCM. So still I wait!

The issue with TWS this morning was not an update. TWS just gave a message window saying this version of TWS is no longer supported and the latest verison needs to be downloaded and installed. So TWS didn't even startup. Not good when trades are in progress but I'm guessing IB might be more at fault here than NT unless NT overlooked that IB can do this. I'm now running TWS 981.3c.

If you scroll to the bottom of the page here: https://www.interactivebrokers.com/en/index.php?f=14099 981.3c is described as their "time tested version that is updated less frequently". Maybe NT should focus on this one.

[NinjaTrader_Jim]

pmn100,

Quality Assurance is not a customer facing team so they may not be as responsive to customer messages if you end up working with them. I sent you an email regarding a test build that has the development build changes for Interactive Brokers that also contains the Release 24.3 changes for FXCM.

This should be tested with the versions of TWS or IB Gateway from our Connection Guide (TWS 985.1g and IB Gateway 981.3c)

If you have anything to report there please follow up with me over email.

Everyone else reading the thread, our recommendation is the same as what is mentioned in post 85 which fernlicht confirms to be resolved for them.

If that recommendation does not work for you, please reach out to platform support [at] ninjatrader [dot] com with your log and trace files so a technician can work with you more closely.