DLL does not work properly after recovering from data outages

[molecool]

So over the past few years I have been running into a specific problem which I (unfortunately) replicated on each VPS or system I installed NinjaTrader:

My strategy sends out emails to myself and a few others about each step it takes. This allows us to be notified when things happen. This is done by querying my DB running on my website and then sending out those emails to the retrieved recipients.

Now strangely if the data connection or the feed drops a few times and then reconnects/recovers again I intermittently see situations when the DB query fails. I don't know why it fails but it returns 0 records (which is what I'm logging). When I run a manual query on the server it will be able to connect and execute the query just fine. Only NinjaTrader stops being able to run the SQL query once it gets 'screwed up' somehow. The only available measure has been to restart NinjaTrader after which the DB queries execute just fine.

The DB service is a simple service DLL I wrote myself and which works without problems. Yes, I know you guys do not support outside DLLs but I still would like to know what could possibly cause this type of behavior. Why would a few data outages put Ninja into a mode where it somehow detaches the DLLs or is unable to call them properly?

Any ideas would be very welcome as this has been a major annoyance for years.

[NinjaTrader_JC]

Hello molecool,

Does your Strategy get disabled when this happens?

By default NinjaTrader will disable your Strategy if it there is 4 connections losses within 5 minutes.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?maxrestartattempts.htm

[molecool]

Hello molecool,

Does your Strategy get disabled when this happens?

By default NinjaTrader will disable your Strategy if it there is 4 connections losses within 5 minutes.
http://www.ninjatrader.com/support/h...rtattempts.htm

No, which makes it very difficult to anticipate this. It just keeps running but I stop getting emails. Also, I have recently created another DLL that writes trade logs to a Google spreadsheet. When it happened today it also was not able to write a log after being disconnected - so I think it's related to any external DLL resource.

[sledge]

How often do you open connection to database?

Each task?

and there are no messages in output window about dll?

No, which makes it very difficult to anticipate this. It just keeps running but I stop getting emails. Also, I have recently created another DLL that writes trade logs to a Google spreadsheet. When it happened today it also was not able to write a log after being disconnected - so I think it's related to any external DLL resource.

[NinjaTrader_JC]

Hello molecool,

Not really familiar with using External DLL's but is it possible to try to establish a connection to your External DLL inside of the OnConnectionStatus() so that when NinjaTrader gets connected back to your Data Provider that it checks the Status of your External DLL's and tries to connect to it again if it comes back as failed?

[molecool]

How often do you open connection to database?

Each task?

and there are no messages in output window about dll?

No, nothing - no error messages of any kind. Just says that it sent 0 emails, that's it.

The code is pretty simple - the general pattern is like this:

			MySqlConnection conn;
			
			try
			{
				conn = new MySqlConnection();
				conn.ConnectionString = CONNECTION_STRING;
				conn.Open();
					
				MySqlCommand cmd = new MySqlCommand();
				MySqlDataReader results;
				
				cmd.CommandText = GET_SUBSCRIBERS;
				cmd.Connection = conn;
				results = cmd.ExecuteReader();
				while (results.Read())

				{
					// add result to list
					
				}

				results.Close();

			}
			catch (MySqlException ex)
			{
				System.Console.WriteLine("error" + ex.Message);
			}
			
		}

Yesterday I did a netstat -an and noticed that the affected ports remain open/attached after the call (which happens only once every few hours. So I have a theory that somehow connectivity drops and NinjaTrader somehow detaches and then manages to recover. But those sockets perhaps are now corrupt. I'm not sure if that is a NinjaTrader issue or if it's my code above. As you can see I do not explicitly call conn.close(). But the pertinent documentation was very explicit about the connection supposedly closing as soon as the script executes.

So I am surprised to see quite a list of connections to port 3306 (mysql remote port) when doing a netstat -na. Perhaps it would be better for me to close explicitly and thus address this problem. But it also happened via my Google logging DLL and that one uses a completely different API.

Any thoughts welcome - thanks for the input so far.

[molecool]

Hello molecool,

Not really familiar with using External DLL's but is it possible to try to establish a connection to your External DLL inside of the OnConnectionStatus() so that when NinjaTrader gets connected back to your Data Provider that it checks the Status of your External DLL's and tries to connect to it again if it comes back as failed?

Woow - you lost me there. Could you please give me an example of that? Also please see the response I just posted which may provide further hints as to what's happening.

[NinjaTrader_JC]

Hello molecool,

I do not have any examples of this, but what ever method you are calling that returns a 0 can you check this in OnConnectionStatus()?

I don't know why it fails but it returns 0 records (which is what I'm logging). When I run a manual query on the server it will be able to connect and execute the query just fine.

The basic idea that I had is that if you can check the connection state of the "sql" and it is not connected then you can try to re-establish a connection inside of the OnConnectionStatus() when NinjaTrader becomes connected to the data provider.

[molecool]

Hello molecool,

I do not have any examples of this, but what ever method you are calling that returns a 0 can you check this in OnConnectionStatus()?



The basic idea that I had is that if you can check the connection state of the "sql" and it is not connected then you can try to re-establish a connection inside of the OnConnectionStatus() when NinjaTrader becomes connected to the data provider.

Not sure I follow but I may have found a fix. I just added conn.Close() after results.Close() and then ran it in Market Replay. Afterward I did a netstat and now there were no remaining connections to port 3306. So I'll deploy this today and then see if it happens again.

Thanks to everyone who responded.