Connection lost to the NinjaTrader Historical Data Server

[reach4thelasers]

That's what I'm thinking. Here we gooooooooo

Surely it will gap fill

[NinjaTrader_James]

Hello Everyone,

I have spoken with our IT and Development teams in order to provide an update.

We restored data on the majority of futures instruments at the close of CME exchange yesterday which did restore the missing price data. Unfortunately, we ran into a volume issue where volume is simply too high for the 18th and 19th on restored data for minute and daily data. This issue will prevent us from restoring additional futures data by this weekend. We are working through this now and will have an update early next week.

For a brief update on the original issue, we run multiple historical data servers for redundancy and unfortunately we had a simultaneous failure event which caused all NinjaTrader 8 historical data servers to not record historical data for 1 hour period after 1:40 AM EST on 1/19. We have identified the cause and will be taking preventative measures. We have technology in place to quickly backfill data at the end of the day now with NinjaTrader 8 between our historical data servers however in the case where all NinjaTrader 8 historical data servers are missing data it requires slower backfill from a third party, which is the process are working through now.

Thank you for your patience.

[tradesmart]

Unfortunately, we ran into a volume issue where volume is simply too high for the 18th and 19th on restored data for minute and daily data. This issue will prevent us from restoring additional futures data by this weekend. We are working through this now and will have an update early next week.

And just how bad the volume issue is you can see in the attached screenshots: millions and millions of ES contracts traded every minute.

[fredrey]

Hi

Can someone please confirm this for me to make sure I understand it right:

CQG, Continuum and Rithmic data feeds only provide real time data.They use the Ninjatrader historical data server to download historical data.

Kinetick, IQFeed, BarChart and eSignal all provide both real time and historical data which means historical data is not downloaded from the Ninjatrader historical data server but form their own servers.

So this means Kinetick and IQ Feed users would not have had the volume issue yesterday. Is that correct?

[NinjaTrader_PatrickG]

Hi

Can someone please confirm this for me to make sure I understand it right:

CQG, Continuum and Rithmic data feeds only provide real time data.They use the Ninjatrader historical data server to download historical data.

Kinetick, IQFeed, BarChart and eSignal all provide both real time and historical data which means historical data is not downloaded from the Ninjatrader historical data server but form their own servers.

So this means Kinetick and IQ Feed users would not have had the volume issue yesterday. Is that correct?

This is correct.

[soulfx]

From what i can see in trying to reload all data .. the missing data for 18/19th period is still a problem and bid/ask tick replay data at issue still. Has this or when is this expected to be completed?
thanks

[NinjaTrader_PatrickG]

Our development team is looking into the imported data volume issue for the 18th and 19th. Ideally we will have an update on this in the next few days. Bid/ask stamped tick data will unfortunately not be backfilled since our third party backfill provider does not have that feature.

[soulfx]

Hello thanks for the reply, however given that bid/ask tick data is a key feature of N8 and the likelihood of a server issue occuring again is realistically high ; would it not be prudent and expected of ninja users to have this backfill of bid ask data - have other solutions or backfill providers been considered?
thanks

[soulfx]

Hello there, i notice there was no reply to my last post here.
Also today i have experienced repeated disconnections to N8 and the reconnect doesnt seem to work or take hold and requires shutting down. The last time around time of this thread 18/19th there was noted internet outage issues ontop of ninja server issues. has there been any ninja server issues today?
The disconnect reconnect does NOT seem reliable : i have ninja 7 running alongside and it at least eventuall managed to reconnect (appears cleanly after several attempts) - how is this different in N8?
thanks

[NinjaTrader_PatrickG]

The Development Team has considered all options available to restore the data, however, at this time it cannot be restored.

I would suggest writing into PlatformSupport[AT]NinjaTrader[DOT]com and reporting this disconnect behavior so we can troubleshoot what you're experiencing.

[sobertradingpartner]

Constant reconnections

Hello there, i notice there was no reply to my last post here.
Also today i have experienced repeated disconnections to N8 and the reconnect doesnt seem to work or take hold and requires shutting down. The last time around time of this thread 18/19th there was noted internet outage issues ontop of ninja server issues. has there been any ninja server issues today?
The disconnect reconnect does NOT seem reliable : i have ninja 7 running alongside and it at least eventuall managed to reconnect (appears cleanly after several attempts) - how is this different in N8?
thanks

I'm having the same problem today, constantly losing the connection and re-connecting(to the HDS server - #2). Some of my charts will not update (like ZB). While I'm normally very forgiving with certain faults of Ninjatrader, being unable to perform its main function (trading) is, frankly, not very acceptable. Can someone on the Ninjatrader end please clarify what is happening today?

[NinjaTrader_PatrickG]

The NinjaTrader historical servers have experienced no downtime today and are currently up and running as expected.

I can connect to HDS2 successfully using a NinjaTrader Continuum demo. I can also load ZB real-time and historical data without issue.

Internet health is relatively poor at the moment which may be contributing to your individual experiences:

http://www.internethealthreport.com/

I would also advise writing in so we can troubleshoot your individual experience.

[brohnhdon]

Besides bid/ask stamped tick data, is the volume issue still being worked on?

[NinjaTrader_PatrickG]

Besides bid/ask stamped tick data, is the volume issue still being worked on?

current issues with volume are being addressed, however, I have no update or ETA at this time.

[NinjaTrader_PatrickG]

First, I want to thank everyone for your patience while we work on resolving these historical data items. Here is the latest update:

• Futures volume issues are now resolved.
  
• Forex gap issues are now resolved.
  
• Bid/ask stamped tick data will remain an issue and is currently unresolvable since our backfill provider does not supply that data. We're considering alternative backfill sources so that if we have a case like this in the future we could backfill that data as well.