Annoyance led to discovery

[eDanny]

A couple of years ago I was able to match a volume chart with one in a trade room and it would plot exactly the same unless one of the charts suffered from a data glitch (internet blip). A reload of historical data would normally bring the two charts into alignment again. Of course clocks had to be in sync and session times the same. The problem now is I have a PC alarm tell me to reload historical data every 1/2 hour since the historical ZF data is so different from the realtime data. Then I have to go back and align all draw objects that are misaligned from the previous 1/2 hour. Annoying and inaccurate. So, I also reload the data on 5 min charts since it seems they suffer to a smaller degree, or so I thought. Today I wanted to check out a 2 1/2 min chart but since there is no such thing I set up a 300 second chart to compare to a 5 min. as a test. To my surprise the two charts start plotting differently very quickly. I thought the fault would be in the 300 second chart so I refreshed it and there was no change. I refresh the 5 minute chart and it changes to match the 300 second chart! Amazing. So, as it stands now, the most accurate chart I have is not the Volume chart or the Minute chart but is the Second chart. This certainly doesn't seem right. (6.5.1000.14)

[NinjaTrader_RyanO]

Hello eDanny,

Thank you for your post.

Can you please let me know who your data provider is so that I can test this issue on my end.

[eDanny]

Amp/Zenfire. I just set up a 60 second chart and a 1 min. chart and after 2 bars they start to differ. The 60 second has a 1 tk wick at the top af candle at close and the 1 min closes at the high. Then some bars after that look correct and some not.

[NinjaTrader_RyanO]

Hi eDanny,

I tested this on my end using both your 5 minute example and the 1 minute example, and I do notice the same thing. What that means is that the issue is related to the data feed itself.

The seconds chart builds its bars using tick data, while the minute chart builds its bars using minute data bars. So, as data is incoming from the provider it creates minute bars based on your PC time clock. And with the seconds chart it simply uses each incoming tick for that time interval.

Another factor in this is that Zen-Fire tick data is time-stamped meaning that rearranging historical data will be the norm when the data is based on larger intervals than tick data.

Unfortunately, this is to be expected and would explain why your seconds chart and your volume charts, since they each use tick data to build the bars, were the most accurate when compared to historical data.

Also, please see the following link.

http://www.ninjatrader-support.com/HelpGuideV6/helpguide.html?HowDoesNinjaTraderBuildChartBars

[eDanny]

I didn't say the Volume charts were accurate, what I did say was that they use to be years ago but now I have to refresh every 30 min and still the chart and indicators change drastically. The only chart that seems accurate is the Seconds chart. If volume and seconds are built on the same data then there is a bigger problem here.

[NinjaTrader_RyanO]

Hello,

I must have misunderstood your initial statement regarding your volume charts, but the principle still remains the same.

Even though volume charts and seconds charts use the same foundational data to build there bars there may be differences.

The reason for this is that the volume chart is a two step process, categorizing incoming ticks based on time and then creating volume bars based on the volume data included with incoming ticks. Sometimes, then a tick comes in that is time stamped differently than your local PC clock and thus reshuffles the bars that were built in real time.

With the seconds chart there is no second step to creating a chart, it just plots incoming ticks within the time frame of your chart, and even then there may be discrepancies due to Zen-Fire time-stamping their ticks.

[eDanny]

No matter how everything is done, the results are bad indeed. If my indicator gives a signal and the resulting trade is bad, then a refresh of the data shows the signal was not actually supposed to occur, what can I trust? From what I can tell, other platforms do not have similar problems.

[NinjaTrader_RyanO]

Hi,

Unfortunately, this is to be expected as related to the link that I provided earlier in this thread.

The NinjaTrader application only plots the data as it is reported by the API you are connected through. NinjaTrader behaves like a radio receiver in that it plots data from the connected data provider.

[eDanny]

Yes I've heard all that before and understand. Why is it that ZenFire cannot send in real time the same data as it provides historically to your historical server? This seems like nonsense to me. Also I just had an issue crop up that has before. I reloaded hist. data and got one big bar as last candle. Try again, same. Disconnect, reconnect, same. Shutdown NT, restart, all is well. Has happened before as I said.

[NinjaTrader_RyanO]

Hi,

The reason that the historical data can be different with Zen-Fire is because they time-stamp their incoming ticks. So the time-stamped ticks from Zen-Fire may conflict with your local PC's time when it is building the bars in real-time.

As for your second issue please go to Tools-->Options-->Misc-->Repair DB to make certain that this does not occur in the future.

[eDanny]

It is indeed too bad I can't get accurate real time charts even though my clock syncs every half hour. I'll try the DB repair.