When analyzing ES 06-22 tick data from Kinetick, I have noticed some discrepancies in live data versus historical data and a possible error (possibly my own coding error).
In my export script, I write the OnMarketData MarketDataEventArgs 'e' elements to a text file for later analysis in Excel. When I write the data to a file, I use e.Time.ToString("MM/dd/yyyy HH:mm:ss:fffffff"), e.Price, e.Volume, e.Bid, and e.Ask.
Comparing live data to historical data, I noticed the timestamps are different between the live and historical data exports while the tick sequence and volume, price, bid, and ask are all equal. For a specific example, on April 8, 2022 at 12:11:54 (Pacific Time), I have a series of ticks that have different timestamps. Upon closer inspection, it appears that the sub-second values in the live data have a leading zero while the historical data pulled from Kinetick do not have this leading zero.
Attached is a picture showing the data (pdf of my excel file that I cannot upload per website attachment restriction). The left table contains the tick data recorded live while the right table contains the tick data pulled as historical data from Kinetick (both live and historical data are coming from Kinetick). I have highlighted the timestamps that match in green and the timestamps that don't match in red. Also, the column "Milliseconds" in both tables shows the milliseconds part of the timestamp. As stated earlier, it appears that the timestamps recorded live have an extra leading zero compared to the timestamps recorded with historical data.
Have you seen this issue before, and do you know what is going on? Is my time formatting of the MarketDataEventArgs 'e' correct (i.e. e.Time.ToString("MM/dd/yyyy HH:mm:ss:fffffff"))?
Thank you for your help, and please let me know if anything is unclear.
Edit on May 8, 2022:
To add, I've looked further into this. In live recorded tick data, I see plenty of timestamps with HH:mm:ss:0ff timestamps (millisecond begins with a zero), but this rarely occurs in historical tick data. Out of the 209,661 records of tick data that I recorded live from 11:24am PT to 1:00pm PT on April 8th, 2022, 27,166 records had millisecond timestamps beginning with 0 (e.g. 026, 094, 055). However, looking at the same time segment in the historical tick data, only 54 records had millisecond timestamps beginning with 0 (e.g. 026, 094, 055).
Statistically, assuming that the trades' timestamps are coming in randomly, I would expect 100 out of 1000 trades (or 10%) of trades to have a timestamp with the milliseconds component being between 000 and 099 (ranging from 000 to 999 for all timestamps). For my live tick data, 27,166 records with timestamps having milliseconds between 000 and 099 out of 209,661 total records was about 13% of the total. For the historical tick data, 54 records with timestamps having milliseconds between 000 and 099 out of 209,661 total records was less than 0.03% of the total.
This supports that there is something wrong with the timestamps when they come in live versus when they come in with historical data, or my formatting string e.Time.ToString("MM/dd/yyyy HH:mm:ss:fffffff") is adding a leading zero when it shouldn't.
Could you please help me with understanding what is happening here?
Thank you again for your time and help.
Best regards,
traderUS
Comment