NT8 and FXCM

[NinjaTrader_Matthew]

Same here... however, note that volatility and speed of trading has declined over those 4 days. We already know/suspect that the issue has to do with handling a large volume of data coming in during volatile times.

NT support, it might be worthwhile to set up a server with an FXCM connection going to $US30 running all night and into the non-farms payroll news release at 8:30 AM EST this Friday morning upcoming. If it happens to be volatile, and if the data freezes on you, it could be a good test case.

Thanks again for your suggestion. We'll keep an eye out during hte pay roll event.

With that said, our historical data server runs a NinjaTrader client connected to the same FXCM adapter you are using and we have not experienced what you are reporting. If this was something related to volatility or other market dynamics, we would experience the same behavior; unfortunately not the case. While our HDS is not without issues, what you are reporting is not observed in that environment.

You mentioned at one point that you use other applications on the API (which you said are working flawlessly).

I'd like to take a moment to remind of disclaimer #4

4) Using the FXCM account in NinjaTrader and FXCM’s Trading Station at the same time can cause oddities in both software since they share the same API connection. If you closed an instrument in FXCM’s Trading Station, that instrument will no longer receive quote updates in NinjaTrader.

This is something we have noted in our testing, and does cause issues under certain circumstances. For example if you have data open in FXCM and NinjaTrader, and you switch the dealing rate in FXCM or close the subscription, it'll also stop in NinjaTrader without any warning. That is just one example off 'oddities' from sharing the same API.

Our historical data server does not share the API with any other programs, so we do not experience that behavior - which could explain why you are seeing issues, but we do not see issues in our development environments.

If there is something in particular you are doing (such as login into a specific application, or using a specific feature of a 3rd party), we'd be happy to look into that scenario to understand why this is exhibiting issues so frequently for you.

[Trader_55]

wow it's been nearly 4 days without a connection freeze.

Same here... however, note that volatility and speed of trading has declined over those 4 days. We already know/suspect that the issue has to do with handling a large volume of data coming in during volatile times.

NT support, it might be worthwhile to set up a server with an FXCM connection going to $US30 running all night and into the non-farms payroll news release at 8:30 AM EST this Friday morning upcoming. If it happens to be volatile, and if the data freezes on you, it could be a good test case.

[radicaltrader]

wow it's been nearly 4 days without a connection freeze.

[NinjaTrader_Matthew]

Below is my result. Only took a minute or so, is this normal?



Also please see trace file showing lost connection this afternoon below

https://www.dropbox.com/s/mbkkizniv8ulx4c/trace.20160129.00001.txt?dl=0

Here is the screenshot I just took showing still connected despite no data coming into charts for hours.
https://www.dropbox.com/s/uypphbj1bm...52-54.jpg?dl=0

No, that does not look like it is resolving all the way to the FXCM servers.

Also, your comments about Kinetick/Fxcm -> Kinetick just uses FXCM data. You do not connect to Fxcm servers when using the Kinetick server. You connect to the Kinetick servers which process FXCM data.

To clarify, are you saying you see the same issue (e.g., connection is not lost, but price freezes at 'random' times) on both Kinetick and FXCM?

[marty087]

Below is my result. Only took a minute or so, is this normal?

Microsoft Windows [Version 10.0.10240]
(c) 2015 Microsoft Corporation. All rights reserved.

C:\Users\BLINK>pathping fxcorporate.com

Tracing route to fxcorporate.com [204.8.240.52]
over a maximum of 30 hops:
0 BLINK-PC.gateway [10.0.0.115]
1 dsldevice.gateway [10.0.0.138]
2 * * *
Computing statistics for 25 seconds...
Source to Here This Node/Link
Hop RTT Lost/Sent = Pct Lost/Sent = Pct Address
0 BLINK-PC.gateway [10.0.0.115]
0/ 100 = 0% |
1 6ms 0/ 100 = 0% 0/ 100 = 0% dsldevice.gateway [10.0.0.138]

Trace complete.

C:\Users\BLINK>

Also please see trace file showing lost connection this afternoon below

https://www.dropbox.com/s/mbkkizniv8ulx4c/trace.20160129.00001.txt?dl=0

Here is the screenshot I just took showing still connected despite no data coming into charts for hours.

https://www.dropbox.com/s/uypphbj1bmydcp2/2016-01-29_17-52-54.jpg?dl=0

[radicaltrader]

Hi Marty I have had to install time snapper to record my Profit targets etc so I can manually enter them in the morning. I might have 30 different limits to manually enter.

[radicaltrader]

I'm using FXColo as my VPS (direct connection to FXCM)

Microsoft Windows [Version 6.0.6002]
Copyright (c) 2006 Microsoft Corporation. All rights reserved.
C:\Users\xxxxxxxxxx>pathping fxcorporate.com
Tracing route to fxcorporate.com [204.8.240.52]
over a maximum of 30 hops:
0 No resources.

[marty087]

Thanks for the reply.

Just to be clear to you... on a permanent loss of connection e.g. longer than 90s the connection will remain green permanently.

I am only running one $AUDUSD

This has been happening to me for years on multiple machines and installs of NT7. I have been running NT7 with Kinetick, all my charts are FX and I presume the data comes from FXCM as that is what the Kinetick site states. I go to my charts in the morning to find the green connection light but charts stopped received incoming data many hours ago. There does seem to be a link between high vol periods and this occurring as it is common for this to happen around US open etc.

I will endevour to collect this diagnostic information and send it though. In the meantime I am running a timer event driven script that emails me when there is no data activity so I can go to my computer and disconnect/reconnect.

Really hoping we can work together to get to the bottom of this!

[radicaltrader]

Thanks for the reply.

Just to be clear to you... on a permanent loss of connection e.g. longer than 90s the connection will remain green permanently.

I am only running one $AUDUSD

[Trader_55]

As a side note: it sounds like you guys may be using multiple applications with your fxcm account connection. There is another limit on the API to bare in mind which is the number of requested symbols you can received data for concurrently. I believe this is defaulted to 20, but you can have that number changed by an FXCM rep should you so wish...

I only mention this because we have on many occasions started chasing connection issues only to find that users were looking up new symbols, which then put them over the limit unknowingly. In doing so, data for some symbols would stop, while others would flow.

To understand what exactly is happening, here is what I suggest:

Thanks, I will follow these steps the next time I see the issue occurring and let you know what is going on. For reference, I am only streaming 2 symbols from FXCM, namely $US30 and $GER30_CFDEUR (both CFD products). I can also say that they both stop updating at exactly the same time, which means that it's a global issue with the adapter. I will let you know what I find the next time the freeze happens.

Also, one more interesting note: when the freezing occurs, I am actually still able to enter/exit positions. For example, I can close out a position if I am in one... this implies that the connection to the trade server is OK, it's just the data streaming that is the issue.

[Trader_55]

The connection status is reported (green / orange / red) is reported by the API.

However, with FXCM - It takes the API 90 seconds to even report to the NinjaTrader client that the connection has been lost. This means that you can experience periods where you are NOT receiving data (since you have lost connection), but the API has not yet reported this to NinjaTrader. As a result, the connection status is green and connected.

Just a note: with the condition that is occurring here, the real problem is not the fact that the disconnect it taking time to be reported in the status bar, the problem is that the disconnect is happening at all, and on a regular basis. It is a known condition that the API takes up to 90 seconds to report the issue to NinjaTrader, which I suppose is a problem for automated strategies, but if you are watching it manually then it becomes quite obvious when the connection is frozen with or without the status field changing color.

[NinjaTrader_Matthew]

This is correct... while it did crash on me once or twice, I cannot say for sure that it is directly related to the FXCM adapter.

The situation described above, however, does happen relatively frequently. I have also noticed that it seems to happen more often when there is a lot of data streaming in very quickly, during volatile times.

For example, yesterday about 20 minutes before the S&P close, FXCM stopped transmitting data when the indices were quickly moving in a volatile downtrend. Similarly, about 10 minutes after today's 9:30 open, the data also froze during a volatile period. I have a feeling that somehow NT's FXCM adapter code may not be able to handle high volumes of trades coming in.

I would suggest that the development team make sure that it can handle this during NT 8 testing. Ask them if they can just turn on an FXCM connection and have it displaying data on a few minute + tick-based charts (range, second, whatever), and make sure that it runs without a glitch during volatile trading periods (news releases, opens and closes) on instruments like $US30 (dow jones 30). Just leave it running and see what happens, maybe with an NT 7 connection side-by-side. If there is a case of frozen data which occurs, they can investigate it further for NT8.

I appreciate your advice; however we do performance testing pretty frequently and this is just simply not something that comes up in our testing. We need your help to figure out what is going wrong.

As a side note: it sounds like you guys may be using multiple applications with your fxcm account connection. There is another limit on the API to bare in mind which is the number of requested symbols you can received data for concurrently. I believe this is defaulted to 20, but you can have that number changed by an FXCM rep should you so wish...

I only mention this because we have on many occasions started chasing connection issues only to find that users were looking up new symbols, which then put them over the limit unknowingly. In doing so, data for some symbols would stop, while others would flow.

To understand what exactly is happening, here is what I suggest:

1. Open a Market Analyzer in NinjaTrader
2. Right click-> Columns->
3. Add the "TimeLastTick" column on your market analyzer
4. Right click again-> Instruments
5. Add exactly the number of symbols you plan on viewing / you have configured on your account.

This is going to give you a display of all the instruments you are viewing data for and the time they have last updated.



During periods of issues, if all instruments are not updating, then it would indeed be an adapter/connection issue. We would then need some diagnostic information.

• open command prompt Start > All Programs > Accessories > Command Prompt
• type in the following
• pathping fxcorporate.com

This will immediately give you a brief trace route to the fxcm servers that NinjaTrader is currently using.

It will then start to compute statics for about 500 seconds (that's over 8 minutes, so this takes a while)

Once it is complete, it will give you an output of the average response time to each ISP node between NinjaTrader and the FXCM severs we NinjaTrader uses. This will be crucial in us understanding what is happening in your environment during these periods where the adapter is not receiving data. (Note: as you have said you have other programs working fine, I'm sure your internet is also working fine, this is simply some diagnostic information about your environment we would need to analyze to understand why this is occurring)

You can right click and select all > copy on these results, and then paste them into an email or another text file and send these results to our support team.

Example of a pathp ping result

C:\Users\MyAccount>pathping fxcorporate.com

Tracing route to fxcorporate.com [204.8.240.52]
over a maximum of 30 hops:
  0  my.computer [x.x.x.x]
  1  x.x.x.x
  2  173-14-19-14-Colorado.hfc.comcastbusiness.net [173.14.19.14]
  3  96.120.12.17
  4  xe-2-0-1-sur01.denver.co.denver.comcast.net [68.85.221.221]
  5  ae-20-ar01.denver.co.denver.comcast.net [68.86.128.169]
  6  be-33652-cr01.1601milehigh.co.ibone.comcast.net [68.86.92.121]
  7  be-11719-cr02.denver.co.ibone.comcast.net [68.86.86.77]
  8  be-11724-cr02.dallas.tx.ibone.comcast.net [68.86.84.230]
  9  hu-0-10-0-0-pe03.1950stemmons.tx.ibone.comcast.net [68.86.86.170]
 10  ae13.er1.dfw2.us.zip.zayo.com [64.125.13.185]
 11  ae11.cr1.dfw2.us.zip.zayo.com [64.125.20.65]
 12  ae27.cs1.dfw2.us.eth.zayo.com [64.125.30.180]
 13  ae0.cs2.dfw2.us.eth.zayo.com [64.125.28.123]
 14  ae5.cs2.iah1.us.eth.zayo.com [64.125.28.102]
 15  ae3.cs2.dca2.us.eth.zayo.com [64.125.29.44]
 16  ae4.cs2.lga5.us.eth.zayo.com [64.125.29.30]
 17  ae27.cr2.lga5.us.zip.zayo.com [64.125.30.253]
 18  ae3.er4.lga5.us.zip.zayo.com [64.125.31.246]
 19  208.185.12.62.IPYX-065351-ZYO.above.net [208.185.12.62]
 20  www.fxcorporate.com [204.8.240.52]

Computing statistics for 500 seconds...
            Source to Here   This Node/Link
Hop  RTT    Lost/Sent = Pct  Lost/Sent = Pct  Address
  0                                           my.computer [x.x.x.x]
                                0/ 100 =  0%   |
  1    0ms     0/ 100 =  0%     0/ 100 =  0%  x.x.x.x
                                0/ 100 =  0%   |
  2    0ms     0/ 100 =  0%     0/ 100 =  0%  173-14-19-14-Colorado.hfc.comcastb
usiness.net [173.14.19.14]
                                0/ 100 =  0%   |
  3   10ms     0/ 100 =  0%     0/ 100 =  0%  96.120.12.17
                                0/ 100 =  0%   |
  4   10ms     0/ 100 =  0%     0/ 100 =  0%  xe-2-0-1-sur01.denver.co.denver.co
mcast.net [68.85.221.221]
                                0/ 100 =  0%   |
  5   10ms     0/ 100 =  0%     0/ 100 =  0%  ae-20-ar01.denver.co.denver.comcas
t.net [68.86.128.169]
                                0/ 100 =  0%   |
  6   10ms     0/ 100 =  0%     0/ 100 =  0%  be-33652-cr01.1601milehigh.co.ibon
e.comcast.net [68.86.92.121]
                                0/ 100 =  0%   |
  7   13ms     0/ 100 =  0%     0/ 100 =  0%  be-11719-cr02.denver.co.ibone.comc
ast.net [68.86.86.77]
                                0/ 100 =  0%   |
  8   27ms     0/ 100 =  0%     0/ 100 =  0%  be-11724-cr02.dallas.tx.ibone.comc
ast.net [68.86.84.230]
                                0/ 100 =  0%   |
  9   27ms     0/ 100 =  0%     0/ 100 =  0%  hu-0-10-0-0-pe03.1950stemmons.tx.i
bone.comcast.net [68.86.86.170]
                                0/ 100 =  0%   |
 10   28ms     1/ 100 =  1%     1/ 100 =  1%  ae13.er1.dfw2.us.zip.zayo.com [64.
125.13.185]
                                0/ 100 =  0%   |
 11   27ms     0/ 100 =  0%     0/ 100 =  0%  ae11.cr1.dfw2.us.zip.zayo.com [64.
125.20.65]
                                0/ 100 =  0%   |
 12  ---     100/ 100 =100%   100/ 100 =100%  ae27.cs1.dfw2.us.eth.zayo.com [64.
125.30.180]
                                0/ 100 =  0%   |
 13  ---     100/ 100 =100%   100/ 100 =100%  ae0.cs2.dfw2.us.eth.zayo.com [64.1
25.28.123]
                                0/ 100 =  0%   |
 14  ---     100/ 100 =100%   100/ 100 =100%  ae5.cs2.iah1.us.eth.zayo.com [64.1
25.28.102]
                                0/ 100 =  0%   |
 15  ---     100/ 100 =100%   100/ 100 =100%  ae3.cs2.dca2.us.eth.zayo.com [64.1
25.29.44]
                                0/ 100 =  0%   |
 16  ---     100/ 100 =100%   100/ 100 =100%  ae4.cs2.lga5.us.eth.zayo.com [64.1
25.29.30]
                                0/ 100 =  0%   |
 17   59ms     0/ 100 =  0%     0/ 100 =  0%  ae27.cr2.lga5.us.zip.zayo.com [64.
125.30.253]
                                0/ 100 =  0%   |
 18   61ms     0/ 100 =  0%     0/ 100 =  0%  ae3.er4.lga5.us.zip.zayo.com [64.1
25.31.246]
                                0/ 100 =  0%   |
 19   64ms     0/ 100 =  0%     0/ 100 =  0%  208.185.12.62.IPYX-065351-ZYO.abov
e.net [208.185.12.62]
                                0/ 100 =  0%   |
 20   61ms     0/ 100 =  0%     0/ 100 =  0%  www.fxcorporate.com [204.8.240.52]


Trace complete.

C:\Users\MyAccount>

If you need any help with this, our support staff can help you setup these tests over a remote session if needed.

[NinjaTrader_Matthew]

In NT7 the disconnect is never reported. "connection remains green".

That is precisely what I mean...

The connection status is reported (green / orange / red) is reported by the API.

However, with FXCM - It takes the API 90 seconds to even report to the NinjaTrader client that the connection has been lost. This means that you can experience periods where you are NOT receiving data (since you have lost connection), but the API has not yet reported this to NinjaTrader. As a result, the connection status is green and connected.

NT7 doesn't crash, you can still navigate around the application. The application just stops receiving data from FXCM.

I would be surprised to see an adapter cause NT to completely crash, but I'm just going off language used in this thread. This may be something different that trader_55 is reporting.

It sound's like it is just Trader 55 and I that's having the issue..... I'm happy to open a support case with Trader55

I would not assume you're both experiencing the same issues and would treat them different.

It would be great if you both could create separate tickets by emailing our support staff at platformsupport[at]ninjatrader[dot]com and we could work on these cases individually to make sure we're not talking about apples and oranges.

Including your log and trace files from a period you received a disconnect is a great way to start.

[Trader_55]

NT7 doesn't crash, you can still navigate around the application. The application just stops receiving data from FXCM.

This is correct... while it did crash on me once or twice, I cannot say for sure that it is directly related to the FXCM adapter.

The situation described above, however, does happen relatively frequently. I have also noticed that it seems to happen more often when there is a lot of data streaming in very quickly, during volatile times.

For example, yesterday about 20 minutes before the S&P close, FXCM stopped transmitting data when the indices were quickly moving in a volatile downtrend. Similarly, about 10 minutes after today's 9:30 open, the data also froze during a volatile period. I have a feeling that somehow NT's FXCM adapter code may not be able to handle high volumes of trades coming in.

I would suggest that the development team make sure that it can handle this during NT 8 testing. Ask them if they can just turn on an FXCM connection and have it displaying data on a few minute + tick-based charts (range, second, whatever), and make sure that it runs without a glitch during volatile trading periods (news releases, opens and closes) on instruments like $US30 (dow jones 30). Just leave it running and see what happens, maybe with an NT 7 connection side-by-side. If there is a case of frozen data which occurs, they can investigate it further for NT8.

[radicaltrader]

Thank you for responding.

We do know of an API limitation where a disconnect event isn't reported for 90 seconds. This was true in NT7 and remains true in the new NT8 adapter as well.

In NT7 the disconnect is never reported. "connection remains green".

I do not know of any issues that could cause the application to crash as a result of FXCM adapter alone.

NT7 doesn't crash, you can still navigate around the application. The application just stops receiving data from FXCM.

It sound's like it is just Trader 55 and I that's having the issue..... I'm happy to open a support case with Trader55