Poor Kinetick data quality?

[Beauregard]

Hello

After some attempts last year, which I stopped, because of data problems with Kinetick (see kinetick streams old data on bank holidays: answer still pending, Kinetick: Download all futures contract ) I thought to try it again.
I downloaded the continous corn contract (ZC ##-##) and compared it to PiFin.Within the last year I see the following wrong data:
1. 8th and 11th July 2011: No data from kinetick (see screenshot)
2. On 30th of May, which seems to be a banking holiday, Kinetick sends a record with opoen/high/low/close of 758.5, which is the close of 27th May. As reported already 5 months ago, this problem still remains. (see screenshot)
3. 11th of May: No data record
4. 21th of February: Same wrong data as in point 2 above
5. 17th of January: Same wrong data as in point 2 above
6.I also downloaded the monthly contracts. Rollover data is defined per 29th Nov 2011. But quotes in the continous contract are different from current month contract already from 14th December on and earlier.
Why is this not the same data?

NinjaTrader is a great product, I like it, it is good made and you have good support. But with your Kinetick partnership you put your good name in danger (as I understand kinetick is somehow even connected with NT). I am using only EOD data, but I need reliable data. I do not want to imagine how realtime kinetick data looks like?

Or do I overlook something?

[NinjaTrader_Brett]

Hello,

Thanks for the note.

The data for the Kinetick service comes from Telvent/DTN servers. If there is any incorrect data they will correct any dates that you mention.

Please send an email into support at kinetick dot com with the below information and the data integrity team will check into the dates you supplied and the questions you have on data. Data should be correct from the Kinetick service.

-Brett

[Beauregard]

I will notify them.
But: My point is not, that I found one bug which should be fixed. My point is that it seems that the data stream is not reliable. Should I compare all historical data from Kinetick with other sources for all instruments in last 10 years? The work is not to fix this, the work is to identify data problems.... My conclusion therefore is, that I will not take Kinetick. That's no problem. And my posting here is a message to NinjaTrader, that you probably should take a look on data quality. Because NT is in my opinion a real good product.
Unless I miss something?

To make sure, that the corn contract was not just an unlucky example I compared now also the wheat contract. Same problems:
- 12th and 14th September are missing
- 11th,12th,13th May are missing

[NinjaTrader_Brett]

Thanks for the feedback.

-Brett

[Beauregard]

I investigated this more in the last days, including opening an account with IQFeed and comparing also other data providers such as pinnacle, zenfire, pifin.
Summary
- Not adjusted data looks quite ok, if you don't mind for differences up to 1% between data providers and therefore probably to reality. I assume a reason is close/settlement price differences.
- Adjusted data from kinetick/IQFeed may be a problem in the corn contract. IQFeed still investigates, see details below
(see details and answers from support below).

Details
1. 8th and 11th July 2011: No data from kinetick (see screenshot)

From Kinetick Support:
Regarding the data that is missing for the ZC and ZW contracts from July 8th, July 11th and May 11th, the reason this data is not present in NinjaTrader is because it is filtered out based on out of range values. On daily bar data Kinetick uses the settlement price for the close value, which in some cases is outside the range established by the day's trading.
For the ZC contract:
July 8th: Open - 652 4/8, High - 670 6/8, Low - 652 4/8, and Close - 672 2/8
July 11th: Open - 672 4/8, High - 680 6/8, Low - 663 4/8, Close - 681 4/8
May 11th: Open - 708, High - 711 2/8, Low - 669 2/8, Close - 668 6/8
In each of these cases the close price based on the settlement from the exchange is either higher than the day's high or lower than the day's low. Because of this NinjaTrader filters the bar so that it does not display, but if you were to view an intraday chart this data would be present. Going forward we are going to seek to include an alert in the log tab when NinjaTrader does this type of data filtering.
The nature of this issue is derived from the fact that NinjaTrader does not recognize a settlement price, but a close value. With Kinetick the close value that is generated is not based on the close price, but on the settlement price and it is displayed as the close price. Because of this there are instances where the close price as available to NinjaTrader, based on the settlement, is outside the bounds of the bar's other OHL values. When this is the case NinjaTrader filters the bars from a chart.

2. On 30th of May, which seems to be a banking holiday, Kinetick sends a record with opoen/high/low/close of 758.5, which is the close of 27th May. As reported already 5 months ago, this problem still remains.

From Kinetick Support
As for holiday's as they are reported by the exchange Kinetick merges that daily data as part of the previous session and following session. Take for example the most recent holidy on February 20th, the data for the times when the CME markets were open were included as part of the previous night session, so that that the data generated for 2/20 would be calculated based on the 2/19 overnight trading session. Upon the market re-open the data then generated for that evening was included as part of the 2/21 daily bar. This only applies to the CME markets, and thus the ZC and ZW contracts, and is expected.

3. Backadjusted Data
In the IQFeed ( and Kinetick Feed) is on 15th July 1996 a change of 15.3% to the previous day. Looking at the unadjusted data as well other providers data this ist quite strange. I compared the unadjusted with the adjusted data as well the month contracts (downloaded from turtle page). As you can see in the PDF enclosed there was neither in the July nor in the September contract any movement compared to the backadjusted data series.
IQFeed Support currently still investigates this point.

4. 0-Values or 259 Values
In the downloaded data from IQFeed/Kinetick Feed there are sometimes days with 259 values (adjusted data) or 0 values (not adjusted data). I guess I have not to say that bugs like that are unpleasant as well, because they could be identified quite easily.

[kemin]

In each of these cases the close price based on the settlement from the exchange is either higher than the day's high or lower than the day's low. Because of this NinjaTrader filters the bar so that it does not display, but if you were to view an intraday chart this data would be present. Going forward we are going to seek to include an alert in the log tab when NinjaTrader does this type of data filtering.

I too find this problem annoying, where missing data abound for more volatile contracts in the grains. Is there a way around this for NT to ignore the settlement price that is outside the daily OHL? Am manually inputting these "outside" close for missing bars but everytime I refresh the historical data from Kinetick, all previously input values will be deleted again! Please share your experience.

[NinjaTrader_Brett]

Is there a way around this for NT to ignore the settlement price that is outside the daily OHL.

I'm open to suggestion to send into development on what to do in this scenario. Currently the best way we see to handle bar data that the close is outside of the HIGH or LOW is to filter it out as this bar is invalid and cannot be charted.

-Brett

[kemin]

I'm open to suggestion to send into development on what to do in this scenario. Currently the best way we see to handle bar data that the close is outside of the HIGH or LOW is to filter it out as this bar is invalid and cannot be charted.

-Brett

One compromise is to use the High or Low of the day as Close if price settled outside of this range, ignoring the settlement, this may not look like the real picture but is definitely better than a missing bar. Another way is to extend the High or Low to match the Settlement price, creating an artificial longer bar.

[NinjaTrader_Brett]

Not sure if we can do that as then the data is incorrect still. We would have the same problem and at the same time be displaying incorrect close prices. Instead we opt here simply not to display the bar. We will think on this to see if this is viable.

-Brett