ES Historical Data Inconsistent Between Computers

[InfraredBusa]

Hi,

A friend and I are working on a strategy together for the ES (at different locations). We're having a terrible time getting the same results from backtesting while using the same strategy. For example, when backtesting the strategy from 4/1 - 4/22 we sometimes see a *huge* difference in Net Profit.

We're both running the same version of NinjaTrader, the same strategy with the same settings. We've deleted our data cache (\Documents\NinjaTrader 8\db/ (deleted folder/files within 'cache', 'day', 'minute', and 'tick').

After much experimentation (comparing chart bars, times, etc...), we're finding that no matter how many times we clear our caches and reload historical data, we end up with different data.

We actually compared charts that were 1 tick charts and there is definitely inconsistency between our charts (missing ticks).

I realize that historical data isn't as complete as live data, but having missing chunks from one computer to the next makes it nearly impossible for us to work together or accurately optimize on backtesting because of suspect data.

We've thought of having one of us download the data and then send the cache files to the other. That would at least sync our historical data, but we still may not be working with 100% accurate data so optimizations won't be accurate.

Is there any way to resolve this so that we have clean and complete historical data?

Thanks!

[NinjaTrader_Jason]

Hello InfraredBusa,

Can you please tell me to what broker/data feed you both connect in NinjaTrader. Could it be you connect to different broker/data feed connections in NinjaTrader? You can experience differences in data when you compare different broker/data feed connections. Please see the link below.
https://ninjatrader.com/support/help...derstandingThe VariablesInvolvedInBuildingChartBars

Discrepancies Between Different Data Feeds
Different data feeds produce different charts, especially when using tick based intervals vs time based intervals. Market data vendors each employ various methods for tick filtering, throttling and time stamping. As a result, no data stream is 100% identical and thus can cause subtle differences in charts. Since NinjaTrader supports many of the leading brokerage and data feed technologies, it is highly likely that two traders using NinjaTrader on different data feeds will have minor differences when plotting the same market and time interval.

You can export historical data as per the instructions at the link below. The other person can then import the historical data on their end.

Export

https://ninjatrader.com/support/helpGuides/nt8/?exporting.htm

Import

https://ninjatrader.com/support/helpGuides/nt8/?importing.htm

[InfraredBusa]

We both use the same: "My NinjaTrader Continuum".

If we remember correctly, we both cleared out cache files/folders at least 3 times and then used "Reload Historical Data". Over the course of our experimenting, my friend got at least 3 different sets of values from our strategy backtests using the various historical data downloads.

We then opened the same chart days (all of this is ES 06-21) and compared bar by bar, tick by tick until we found discrepancies in the data, which we always found.

Through all of our testing, we never were able to get exactly the same historical data on both our machines.

Assuming we're both using the same data provider, what would explain the discrepancies?

Are there multiple historical data servers that feed different client requests that don't have properly sync'ed server data?

How is the data fed to NinjaTrader?

Is it streamed with a non-correcting/connectionless protocol (similar to UDP) that allows for errors to be introduced?

Is the data transferred as files that should resist errors during transfer?

With the data disagreements, he has trades on days that I don't. Sometimes I have winning trades on days when he has losing trades.

As mentioned, this is disconcerting because not only can we not arrive at the same backtesting results, but it also makes our optimizations suspect because we don't know whose data is the most accurate or if either of them are accurate.

Thanks for your help!

[NinjaTrader_RyanS]

Hello,

If loading tick based charts, it is likely the two charts will never match exactly due to how tick bars are created. I've included a link from our help guide with more information below:

https://ninjatrader.com/support/helpGuides/nt8/index.html?how_bars_are_built.htm#UnderstandingWhyAChartCanLookDifferentAfterReloadingHistoricalDataFromTheServer

[InfraredBusa]

The link you sent would make sense for live data (time stamp differences, etc...), but is it applicable to historical data?

When we compared data at the tick level for historical charts, my friend was missing many ticks that displayed on my chart. This meant his bar patterns formed differently than mine (and changed the strategy results). Shouldn't the historical data be the same no matter who downloads it and when they download it?

I'm assuming that historical data is stored in a database on your or the data provider's servers. Why would there ever be variations in that data if it has already been recorded and saved?

Again, my friend repeatedly cleared his cache of data and reloaded historical data and got varying results. He never did get the same data/results that I get (after I've cleared my cache files and reload data).

Thanks!

[NinjaTrader_RyanS]

Hello,

Since Tick charts are not time based, the reference point used to load data will be the instant the chart is requesting data from the historical server. So if one platform requests data, and another platform requests the same data but a half of a second later, the latter will be using ticks that occurred in that half second that the former will not. So if 20 ticks occurred, it is skewing the historical bars by 20 ticks each.