(re)connecting to IB

[redwoood]

I've been having issues with connecting to IB, especially if TWS dropped the connection for some reason and there are a bunch of charts, market analyzer etc open.

What happens is that all NT windows are very unresponsive for up to minutes.
If you right-click on a window in the task bar, it won't even show the context menu.

I suspect there is some problem with backfilling.
It seems to me that backfilling should be done asynchronously by a separate thread. Keyword here being asynchronous... no use if the other threads join with that thread right away. That's how it's done, no?

Wrt the problem I'm having, when it occurs, I often have the message:
MArket Data for <some Index> ... exceeds max price gap
in the log, sometimes many of these messages

Redwoood

[NinjaTrader_Dierk]

Some clarification:
- as you experience a lockup: is there a box popping up saying "Loading data.."? If not, then there is no data loaded from IB
- the "exceeds max price gap" message is a result of buggy/incorrect data being received from IB by the replay recorder. It's filtered out by the recordedr and you will notified.

Could you provide steps on how to reproduce the lockup problem?

Also: it might help consulting your ISP on why you get the disconnects?

[redwoood]

Some clarification:
- as you experience a lockup: is there a box popping up saying "Loading data.."? If not, then there is no data loaded from IB

Ok, I just had another lockup. All windows were stuck, ie not updating. I right-clicked on the control center icon in the taskbar 6-7 times and suddenly the windows updated and the 'loading Data' window appeared.

Could you provide steps on how to reproduce the lockup problem?

I have about 4 charts open, fdax, fesx, es, ^dax. I let it run constantly, so I'm not sure if you need to have it open for a while to happen. I also have the
Market Analyzer open with 17 symbols open.
Then, even if I disconnect from IB manually and reconnect again, NT gets stuck... sometimes.

Also: it might help consulting your ISP on why you get the disconnects?

I look at the NT log and it shows me that the connection to TWS has been dropped. The internet connection is fine. TWS and NT run on the same host.

Redwoood

[NinjaTrader_Dierk]

>> it shows me that the connection to TWS has been dropped

What's the exact message in the logs?

[redwoood]

here are the last few entries from the log.
There has not been a lockup since I wrote the message earlier.

---snip---
6/15/2007 1:17:40 PM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 1:17:27 PM,Connection,inter: Primary connection=ConnectionLost, Price feed=ConnectionLost,
6/15/2007 1:17:25 PM,Connection,inter: Primary connection=Connected, Price feed=ConnectionLost,
6/15/2007 10:41:31 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 10:41:18 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 10:41:18 AM,Connection,inter: Primary connection=ConnectionLost, Price feed=ConnectionLost,
6/15/2007 10:41:17 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 10:41:06 AM,Connection,inter: Primary connection=Connected, Price feed=ConnectionLost,
6/15/2007 9:51:58 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 9:51:57 AM,Connection,inter: Primary connection=Connected, Price feed=ConnectionLost,
6/15/2007 9:51:55 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 9:51:50 AM,Connection,inter: Primary connection=Connected, Price feed=ConnectionLost,
6/15/2007 9:28:00 AM,Default,MarketData (Ask) for '^SP500' at 6/15/2007 9:28:00 AM exceeds max price gap (1993.65/1560.81),
6/15/2007 9:27:41 AM,Default,MarketData (Last) for '^TICKQ' at 6/15/2007 9:27:41 AM exceeds max price gap (479/165),
6/15/2007 8:11:28 AM,Default,Market data farm connection is inactive but should be available upon demand.:hkfarm 2108,
6/15/2007 8:05:00 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 8:04:54 AM,Default,New account data requested. API client has been unsubscribed from account data. Resubscribe2AccountData,
6/15/2007 8:04:49 AM,Connection,inter: Primary connection=Connected, Price feed=ConnectionLost,
6/15/2007 8:04:46 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 8:04:46 AM,Connection,inter: Primary connection=ConnectionLost, Price feed=ConnectionLost,
6/15/2007 8:04:42 AM,Connection,inter: Primary connection=Connected, Price feed=Connected,
6/15/2007 8:04:36 AM,Connection,inter: Primary connection=Connected, Price feed=ConnectionLost,
--- snip ---

[NinjaTrader_Dierk]

a) As of your logs TWS looses connection quite frequently and reports that to NT.
b) The "exceed map price gap" log entry is a result of erronous price info sent by TWS e.g. SP500 spikes from 1993.65 to 1560.81 within 2 ticks. I suggest turning of the replay recorder in case you do not need replay data. Your chart data will not be effected.

[redwoood]

a) As of your logs TWS looses connection quite frequently and reports that to NT.

I'll look into it, maybe IB knows what's going on

b) The "exceed map price gap" log entry is a result of erronous price info sent by TWS e.g. SP500 spikes from 1993.65 to 1560.81 within 2 ticks. I suggest turning of the replay recorder in case you do not need replay data. Your chart data will not be effected.

the log entry itself doesn't bother me, neither do the frequent disconnections (well, actually they do), what bothers me the most is when NT locks up after trying to connect to TWS.
Any suggestion what I could do about that ?

[NinjaTrader_Dierk]

The lockup you experience would be explainable only if you saw the "loading data" box.

However "loading data" box should not appear on a temporary loss of connection but only as you (a) manually (re)connect or (b) add a new instrument to the MA or pull up a new chart etc., since only then historical data is requested from IB.

We never observed this behavior and I would need a reproducible scenario in order to analyze.

[redwoood]

Ok, but let's say I reconnect manually.
Why does NT lock up then?

As I wrote earlier, when I reconnected, NT was locked up for over a minute and probably would've stayed like this indefinitely.
Only after I started clicking 'wildly' on the taskbar entry, the loading data box appeared and NT came to life again.

[NinjaTrader_Dierk]

Let's try this: Please attach your workspace file <my documents>/NinjaTrader6/workspaces/Default.xml to your post and we'll take look.

[redwoood]

sure thing.

enjoy

[NinjaTrader_Dierk]

Sorry, the workspace file you attached only holds the CC no chart not MA.

[redwoood]

well, I got it exactly from where you told me.

Do I have to shut down NT in order for it to write the other info?

[NinjaTrader_Dierk]

>> Do I have to shut down NT in order for it to write the other info?
Correct

[redwoood]

>> Do I have to shut down NT in order for it to write the other info?
Correct

Ah ok, see attached.
May I add another suggestion to periodically save the workspace information (in case NT crashes) ?


Redwoood