Comparing Continuum and Rithmic Data Feeds

[Jim W.]

One of the reasons that I wrote my Data Feed Monitor http://www.ninjatrader.com/support/f...d=4&linkid=652 was so I could compare Continuum and Rithmic Data Feeds. This morning I ran my first comparison over the 9:30 AM ET open. Will add screen shot showing the results.

The Rithmic NT running with a High Process Priority is on my physical computer with Windows 8.1 Pro. The only other application running is VMWARE Workstation 11 with Unity View running at Above Normal Process Priority. One Windows 10 VM is running and under that is NT using Continuum running at High Process Priority. VMWARE Tools is set to Synchronize guest time with host.

As you see from the physical computer's Resource Monitor, my work computer is configured to handle loads like the one above. The cpu was running at 20% and only 31% of physical memory is required, so there is no paging on the physical and VM computer. My Internet connection is FiOS 75/75 Mbps.

P.S. Upload of snapshots fail. Will upload when Support Forum upload fixed.

[NinjaTrader_PatrickH]

Hello Jim W,

Thank you for uploading that indicator.

Different servers are used for each of these datafeeds and users will see varying performance over their ISP, based on their location. Both datafeeds provide real-time unfiltered data.

Please let me know if you have any questions.

[Jim W.]

Please let me know if you have any questions.

Thanks for the offer. Have reached the point where all five of my Shared Indicators are working and are functionally complete. Checking them to see if I missed any low hanging fruit. Looking at the NT Help, OnMarketDataEventArgs I see:

Methods and Parameters

MarketDataType
Possible values are:
MarketDataType.Ask
MarketDataType.Bid
MarketDataType.DailyHigh
MarketDataType.DailyLow
MarketDataType.DailyVolume
MarketDataType.Last
MarketDataType.LastClose (prior session close)
MarketDataType.Opening

Q1: Besides NinjaTrader Help Topic: Historical & Real-Time Data, is there any other documentation available about Continuum, Rithmic, and other Data Providers?

Q2: Are DailyHigh, DailyLow, and DailyVolume the ones referenced in above Help Topic Tips 1. "Not all connectivity providers support all MarketDataTypes." ? I was thinking of displaying / printing these values every 15 minutes or user specified interval. Will put in some test code to watch for their arrival on Continuum and Rithmic.

Q3: Do you see anything missing from this or my other Shared Indicators? Interested on feedback on this from all.

[Jim W.]

Have added optional Daily High, Low, and Volume print to Output Window. Only first data of each type per minute printed. Will update Shared version soon. Thinking about changing it to last per minute instead of first. Will probably do that before I upload and change doc.

[NinjaTrader_PatrickH]

Hello Jim W,

Thank you for your response.

Q1: Besides NinjaTrader Help Topic: Historical & Real-Time Data, is there any other documentation available about Continuum, Rithmic, and other Data Providers?

Information on the Fundamental Data provided can be found at the following link: http://www.ninjatrader.com/support/f...ad.php?t=51946

Other than Fundamental Data, I do not have any other links to information on the providers.

Q2: Are DailyHigh, DailyLow, and DailyVolume the ones referenced in above Help Topic Tips 1. "Not all connectivity providers support all MarketDataTypes." ? I was thinking of displaying / printing these values every 15 minutes or user specified interval. Will put in some test code to watch for their arrival on Continuum and Rithmic.

It is possible the data feed provide is not providing certain MarketDataTypes or that they are only provided at certain times of the day, such as after the close of the day.

Q3: Do you see anything missing from this or my other Shared Indicators? Interested on feedback on this from all.

I have not tested your shared indicators. Anything specific you are looking for assistance with?

[Jim W.]

Revisited my 02-10-2015 Comparing Continuum and Rithmic Data Feeds post

Revisited my 02-10-2015 Comparing Continuum and Rithmic Data Feeds post http://www.ninjatrader.com/support/f...80&postcount=1 .

The above post compared Rithmic NT running with a High Process Priority on my physical computer with Windows 8.1 Pro. The only other application running was VMWARE Workstation 11 with Unity View running at Above Normal Process Priority. One Windows 10 Pro Technical Preview VM ran NT using Continuum. VMWARE Tools was set to synchronize guest time with host.

Since 02-10-2015 there have been no hardware changes but several software updates:

1) An update applied to my VMWare Workstation 11
2) Two new builds installed for Windows 10 Pro Technical Preview
3) Several changes to my DataFeedMonitor Indicator to improve its performance,
including charting the Exchange Data Age information instead of Printing it to the Output Window

To see how much performance degradation running in a VM caused, before doing a Continuum to Rhythmic comparison I created an NT with Continuum configuration on my physical computer. This was done by having two Documents\NinjaTrader 7\Config.xml files. One for Rithmic and the other Continuum and renaming the one that I wanted to test to Config.xml. The Continuum User name had to be different from the one used on the NT Continuum VM because NT does not allow two connections from the same user concurrently.

Yesterday at about 10 AM I started running VMWare and set its Process Priority to Above Normal. NT running on the physical computer was at High Priority. Both NTs had identical Indicators and Workspaces.. Neither NT had any hidden Workspaces. Both were running a Workspace with only one chart with Instrument ES 06-15 in a one minute chart. I kept my hands off the mouse and keyboard on the physical computer until after 4:30 PM. I did not enable Unity view on the Workstation. Comparing the two NT charts showed only a few bars that were not identical. These bars were a tenth of a second different, within the margin of error.

This morning about 8:15 I started my Rithmic to Continuum comparison. Midday I made a change to the Indicator, adding Session Max Age value and adding a volume indicator to the chart. These changes were copied to the VM NT and both continued for the rest of the trading day. Snapshots of the tests are shown below. The top chart is physical and the bottom VM.

The tests were done with the Data Feeds only providing one Instrument. With many Instrument there may be different results. Only the Data Feeds of Rithmic and Continuum were tested. I have no way of testing the Order Entry servers of Rithmic and Continuum. Under the conditions tested, the Rithmic and Continuum Data Feeds performed almost identically.

Going to try to test with more Instruments and hopefully will hit a fast market when a difference could become apparent in the two Data Feeds. Will post when I learn more.

The other side of the coin, one DF getting orders to the Order Servers faster than the other is something that I can't see a way to test. But, a little edge there could mean more profit or not missing a fill.

DataFeedMonitor is very sensitive. Watching it chart made me aware of the affect of NT's design and operation considerations.

o Running NT at the default Normal Process Priority for live trading makes no sense. At Normal it competes with every other application for the CPU. My Indicator Set NinjaTrader's Process Priority to High (SetNinjaTradersPriorityHigh) Links and Downloads Manager - NinjaTrader Support Forum gives NT preference over the other applications that are running. It is not needed in a Workspace with a Chart that already has my Indicators LEDOnMarketDataLast, LEDOnMarketDataBidAsk, or DataFeedMonitor.

o NT 7 is single threaded and NT 8 will be multi-threaded. Being single threaded only one NT 7 task at a time can run, no matter how many cores the CPU has. Read and follow the recommendations in NT Help "Performance Tips".

[pakricard]

Hi: I just did a test on 2 Amazon EC2 identical instances, Windows Server 2008 running only NT7. The same exact configuration except one is getting data from Rithmic and the other from Continuum.

In a full FDAX trading session, Rithmic is losing contracts so at the end of the day, continuum shows around 5% more contracts traded than Rithmic.

Did the same test 2 days and the result was roughly the same, so Rithmic receives less ticks than Continuum.

regards,
pak

[Jim W.]

Rithmic receives less ticks than Continuum.

Interested in your conclusion.
What Processing Priorities were the two NT 7 run at?
Will you post the test tick data from both systems for comparison?

[pakricard]

Hi Jim:

The Processing Priorities were both set by default to "normal". But if it were a problem of process priority on average both datafeed should loose the same number of ticks (on identical instances, as it's the case of the test)., not Rithmics always left behind.

Truth is we don't know if Continuum looses ticks, but in any case Rithmic looses more than Continuum.

I will not post the tick set as I did not save it and already deleted the instances. Anyway it's a simple test.
1) Set 1 Amazon EC2 instance with NT7
2) Create a AMI for this instance
3) From this AMI, create a 2nd instane which is a clone of the first.
4) start both of them before the beginning of trading session
5) Grab a beer and sit to see the differences in volume in market analyzer :-)

Regards,
Pak

[sledge]

Hi Jim:

The Processing Priorities were both set by default to "normal". But if it were a problem of process priority on average both datafeed should loose the same number of ticks (on identical instances, as it's the case of the test)., not Rithmics always left behind.

Truth is we don't know if Continuum looses ticks, but in any case Rithmic looses more than Continuum.

I will not post the tick set as I did not save it and already deleted the instances. Anyway it's a simple test.
1) Set 1 Amazon EC2 instance with NT7
2) Create a AMI for this instance
3) From this AMI, create a 2nd instane which is a clone of the first.
4) start both of them before the beginning of trading session
5) Grab a beer and sit to see the differences in volume in market analyzer :-)

Is this related to how these different connections rollup some of the data? I'm not sure of the status of that.. but it should be all the same at this point.....

I'm only on rithmic... so I have nothing to compare to...




I googled this up: http://ninjatrader.com/support/forum...t=73719&page=3

[Jim W.]

Pak,

I call to your attention my 03-19 post comments :

o Running NT at the default Normal Process Priority for live trading makes no sense. At Normal it competes with every other application for the CPU. My Indicator Set NinjaTrader's Process Priority to High (SetNinjaTradersPriorityHigh) Links and Downloads Manager - NinjaTrader Support Forum gives NT preference over the other applications that are running. It is not needed in a Workspace with a Chart that already has my Indicators LEDOnMarketDataLast, LEDOnMarketDataBidAsk, or DataFeedMonitor.

o NT 7 is single threaded and NT 8 will be multi-threaded. Being single threaded only one NT 7 task at a time can run, no matter how many cores the CPU has. Read and follow the recommendations in NT Help "Performance Tips".

At the beginning of this year, I did extensive testing comparing the data feeds and the CME broadcast session contract total volume, using the procedures documented in this thread.

When NT's Processing priority was set to High and the NT data collection was not preempted by competing NT tasks, I found that Continuum and Rithmic Data Feeds matched tick for tick and both agreed with the CME broadcast session contract volume totals.

Jim

[Jim W.]

Sledge, See my reply to Pak below.