Permanent Connection loss

[CKWSTrader]

Hi!

I have a problem with my NT platform and Kinetick data feed.
During the day, there has been a permanent Connection loss - starting at 07:05:44:720 local time (Timestamp from trace file) and creating a new log entry every second for established connection followed by another connection loss.

Trace file can be sent but is about 50MB big - all fileld up with connection loss and established messages.

This is not the first time this happened and it did on various computers. What to do?

[NinjaTrader_PatrickG]

There are many Kinetick users who do not experience this symptom, and I am not experiencing this symptom when I test a Kinetick connection, so we'll need to find out more about what is causing this symptom on your end. We may need to take this conversation to email so we may gather your log and trace files, but we may be able to gather some useful information on the forums.

What version of NinjaTrader 7 are you using? Please provide the entire version number. This can be found under Help -> About (Example: 7.0.1000.X)

Is there any way to 'resolve' this symptom? For example, what occurs if you restart your PC?

You mention this occurs on 'various computers'. Do all of these computers use the same source of internet? If so, are they all on WiFi or are they hardwired?

Has any general network troubleshooting been performed so far? Here are some great steps to take when experiencing symptoms like these:

• Restart NinjaTrader
• Restart your modem and (if applicable) router
• Restart your PC
• Clear your DNS
  • Open the Command Prompt
  • In the command prompt, run this command: ipconfig /flushdns
• If you are not already and the option is available to you, connect to the internet using an ethernet cable. Wireless connections are significantly more unreliable.
• Ensure exceptions have been added into any firewall/antivirus/backup programs for the Documents > "NinjaTrader 8" folder.
• If the above steps do not help, contact your ISP and let them know you are experiencing a highly unstable connection. An issue of connectivity may exist on their end.

[CKWSTrader]

Hello Patrick!
Thanks for your reply.

Here is some more information you requested.

NT version: 7.0.1000.35
To temporarilly resolve the problem, I disconnect the data feed and then connect again. Connection then is established and stable for a while, even if connection problems have occurred all the time until the disconnect. From the trace file, it seems to me (being software developer for about 20 years now) that there may be a problem in the Kinetick.Adapter.L1Worker process. It looks like it got some kind of "hickup" or has problems to find to a stable state.

The problem occurs on different computers, all running Windows 7 but with different internet connections. "Main" problem computer is connected to T1 via cable, second (where the problem doesn't occur that often) uses mobile internet with UMTS/HSDPA data stick. Internet connection is available without interruption all the time even if connection problems show in NT log window.

If further information is needed, feel free to ask. I also can provide all files necessary for troubleshooting.

[NinjaTrader_PatrickG]

I see that you're running an older and unsupported version of NinjaTrader 7. Are all PCs running older versions of NinjaTrader 7? The current version is 7.0.1000.40.

The newer version contains many critical updates to the database. Please update NinjaTrader 7 and reset your instruments:

First, to update NinjaTrader, please follow the steps below:

• First get your license key from NinjaTrader under Help> License Key then exit NinjaTrader
• Click on the link: http://ninjatrader.com/PlatformDirect
• Enter your license key and press Submit
• Select 'NinjaTrader 7'
• Select 'Download'
• Critical: Before running the installer, ensure NinjaTrader is closed.

Lastly, to reset your instruments, please follow the steps below:

• Disconnect from your data feed: NinjaTrader Control Center> File> Disconnect> Select your data feed
• Reset your Instruments: NinjaTrader Control Center> Tools> Options> Data tab> Reset Instruments>OK
• Restart NinjaTrader> Connect (NinjaTrader Control Center> File> Connect> Select your connection

Please let me know if issues persist after these steps.

[CKWSTrader]

Hello Patrick!

Thanks, will do. Maybe it will solve the problem.

[CKWSTrader]

Hello Patrick!

Today it happened again. I also added some logging functionallity to my Strategy so it checks internet connection at every data feed connection loss. As expected, internet connection still has been available, the Strategy was able to send data to a remote server where it also has been logged.

So it has to be a problem within NinjaTrader that causes this "hickup" on data feed connection loss.
I attached log file and trace file from today.

log.20190207.txt trace.20190207.txt

[NinjaTrader_PatrickG]

I've downloaded and then removed your log and trace files. In the future, DO NOT share these files publicly. They may contain sensitive information. If you want to share your log/trace files, please contact us via email so that we will be the only ones with access to the information.

NinjaTrader and data feeds are extremely sensitive to loss of quality of connection. Disconnects in NinjaTrader may occur even when all other aspects of the internet appear to be working. For example, you may be able to watch a YouTube video while having internet quality issues (because that information can buffer) but NinjaTrader would likely report a disconnect.

I do see that you frequently lose connection to Kinetick. Here is one example from your files:

07.02.2019 05:00:39|1|2|Kinetick: Primary connection=ConnectionLost, Price feed=ConnectionLost

I believe i've found your strategies connection loss print in the log as well:

07.02.2019 05:00:43|1|4|06.02.2019 17:54:00 FDAX - CONNECTIONSTATUS - Order Status: ConnectionLost
07.02.2019 05:00:43|1|4|06.02.2019 17:54:00 FDAX - CONNECTIONSTATUS - Price Status: ConnectionLost

What exactly is your strategy testing to confirm connectivity?

[CKWSTrader]

Hello Patrick!
The Strategy is a "normal" trading Strategy with additional Server communication to log every trade. I added the OnConnectionStatus() function and there send additional log entries every time connection status changes.
Within OnConnectionStatus(), the Strategy sends a HTTP web request to the Server using WebRequest class.

What I think is strange is that when connection is lost, it is established immediately again and then lost again. This loop seems to continue for quite long. In the trace file, you can see it started at 2019-02-07 17:00:39:151 and then connected and lost connection within about 0.5 seconds each until 17:01:42:443 when I manually disconnected and reconnected again - which then worked as it always does.

[NinjaTrader_PatrickG]

I'd recommend using the IQFeed adapter with your Kinetick credentials to see if this mitigates the connection issue:

https://ninjatrader.com/ConnectionGuides/IQFeed-Connection-Guide

[CKWSTrader]

OK will do. Thank you for your assistance!

[awesometrader]

I just experienced the same issue you described with the kinetick connector within the last month. I Finally made the switch to IQFeed and my VPS had a ddos attack which knocked the IQFeed connection offline. The IQFeed connection remained disconnected, while the continuum connection was successfully re-connected.

Is it possible to control reconnection settings for IQFeed? Why did it get stuck in a disconnected state?

Would I get a more consistent and reliable behavior if I switched to the Continuum data feed as the continuum connection actually ended up being re-connected??

2020-02-06 21:45:47:462|1|2|A_IQ_KT: Primary connection=Connection lost, Price feed=Connection lost
2020-02-06 21:45:54:512|3|4|Strategy 'XXXXXX' lost price connection for more than 10 seconds and will be restarted.
2020-02-06 21:45:54:512|1|4|Disabling NinjaScript strategy 'XXXX'
2020-02-06 21:45:55:092|1|2|A_IQ_KT: Primary connection=Connected, Price feed=Connected
2020-02-06 21:46:04:460|1|2|A_IQ_KT: Primary connection=Connection lost, Price feed=Connection lost
2020-02-06 21:46:18:361|1|2|A_Continuum: Primary connection=Connected, Price feed=Connection lost
2020-02-06 21:46:30:908|1|2|A_Continuum: Primary connection=Connected, Price feed=Connected
2020-02-06 21:47:32:166|1|2|A_Continuum: Primary connection=Connection lost, Price feed=Connected
2020-02-06 21:47:55:984|1|2|A_Continuum: Primary connection=Connected, Price feed=Connected
2020-02-06 21:47:56:164|3|2|Connection lost to the NinjaTrader Historical Data Server: hds-us-nt-001.ninjatrader.com
2020-02-06 21:48:06:176|1|2|Using HDS (hds-us-nt-001.ninjatrader.com/31655)
2020-02-06 21:48:07:546|1|2|Connection restored to the NinjaTrader Historical Data Server: hds-us-nt-001.ninjatrader.com
2020-02-06 21:48:47:503|3|4|Failed to resolve instrument for market data: Resolution of 'F.US.VX.F00' timed out.
2020-02-06 21:49:03:019|3|4|Failed to resolve instrument for market data: F.US.VX.F00 could not be resolved.
2020-02-06 21:49:44:869|1|2|A_Continuum: Primary connection=Connection lost, Price feed=Connected
2020-02-06 21:49:51:416|1|2|A_Continuum: Primary connection=Connected, Price feed=Connected
2020-02-06 21:54:46:438|3|2|Connection lost to the NinjaTrader Historical Data Server: hds-us-nt-001.ninjatrader.com
2020-02-06 21:54:56:451|1|2|Using HDS (hds-us-nt-002.ninjatrader.com/31655)
2020-02-06 21:54:56:548|1|2|Connection restored to the NinjaTrader Historical Data Server: hds-us-nt-002.ninjatrader.com
2020-02-06 21:58:42:146|1|2|A_Continuum: Primary connection=Connected, Price feed=Connection lost
2020-02-06 21:58:42:654|1|2|A_Continuum: Primary connection=Connected, Price feed=Connected

[NinjaTrader_PatrickG]

The symptom discussed in this thread was about Kinetick being unable to connect at all even after some network troubleshooting. Please clarify - are you able to manually reestablish the IQFeed connection? Or are you reporting that it does not seem to be working at all after the DDOS attack?

Also, could you clarify what "...if I switched to the Continuum data feed as the continuum connection..." means?

[awesometrader]

Kinetick - I had a similar issue where the connection would be lost and it would keep trying to reconnect with the accumulation of large trace files. That required manual intervention to reconnect - I think I had to restart NT. This would randomly happen every other week or so.

IQFeed - I logged in to the VPS about an hour later from the posted log file logs, restarted NT and the connection was successfull.

Continuum Data Feed - NT Brokerage provides a data feed that you can sign up for instead of using Kinetick - https://ninjatrader.com/MDE . That's what I meant by that...would that be more reliable as it would be the same connection that also handles the orders?

It sucks to log in and see that NT is stuck in a disconnected state, your account is in a position and your strategy is no longer running.

I am aware that you can set the respective strategy "On connection loss" settings + Strategy.StartBehavior to mitigate this, but if the connection its self is never restored, then they would never execute.

[NinjaTrader_PatrickG]

It sounds like in your specific instance that the NinjaTrader Continuum feed is able to reconnect more reliably when an outage occurs, so your logic is sound - routing both data and orders through NinjaTrader Continuum would be a valid work-around.

[awesometrader]

Thanks, I might give that a try if IQFeed keeps having issues. I've heard that it is reliable from other people, so I am hoping it doesn't happen often. I'll also program it so that an e-mail is sent if OnConnectionStatusUpdate gets called so I can stay on top of things.

Since the continuum data pricing is cheaper than kinetick, do you know if the quality of the data the same?