Are there any known issues with downloading replay data?

[forrestang]

Hello.

I've been downloading replay data for MNQ. I have noticed several gaps in bars(For data two weeks ago), which I thought was due to the contract rollover, so I downloaded the PRIOR contract of MNQ-06-22(Current is MNQ-09-22), but those seemed to have problems also.

Another issue, I was replaying the data that occurred during the NY session for 15 June, and there was no data from about ~9:30EST to 9:45 EST.

This seems to persist for both NQ and MNQ.

I'm not particularly worried about it... but am wondering if there are KNOWN issues with replay data, or days when the data may be incomplete, or something else? Is it common, is it a rarity, etc?

In the same way that you can check data of a day to see the quotes, is there a way to know if a downloaded replay will have problems.

Is it likely just on my end? Just trying to get an understanding of what may be happening?

Thanks,
FP

NOTE: This is a repost as my original thread got messed up.

[NinjaTrader_Zachary]

Hello forrestang,

Thank you for your inquiry.

Aside from your request, we have not yet received reports of this specific issue. However, I'd be happy to help look into this and get to the bottom of it. With regard to the first part of your inquiry, so that I may download and test the data, which specific dates and times of day are you noticing the gaps that you are referring to? When you state that there are gaps, do you mind elaborating a bit so I may better understand what I'm looking for once I download the data?

With regard to the issue you are experiencing on June 15th, is this for the 06-22 contract, the 09-22 contract or both?

Lastly, I wouldn't consider issues like this common but they are not unheard of. Until I am able to test, I'm not sure if the problem is on our end or is local to you.

I look forward to assisting you further with this.

[forrestang]

Howdy.

I have an example of the issue. The contract MNQ 09-22, for the trading date of 14 June 2022. The times between @9:05CST to ~9:17.. there is missing data.

[forrestang]

Here is another one, same symbol, on 15June, right at the open 8:30CST, there appears to be another gap in the data... where on my platform, I can see the time still clicking off in the playback window, but the data is just paused at the point.

--edit--
data was missing for about 6 minutes

[NinjaTrader_Zachary]

Hello forrestang,

Thank you for your patience as I worked to test our Playback data.

After downloading this data, and comparing it to the historical data from multiple sources, it reflects exactly what I was seeing from those historical data providers that I was comparing to both on the 14th and the 15th. Specifically, the first candle that you circled in yellow on the 14th appears on all of the charts that I reviewed. With that said, there are some minor differences between what I am seeing on my end and the screenshot that you provided. I'd suggest deleting the playback data from those dates and redownloading from the server.

Please let us know if we may assist you any further.

[forrestang]

Thanks for checking on this Zachary.

Just to be sure we're on the same page with what you were seeing...

The issue isn't necessarily the bar itself... it's just that once it hits 9:05CST, the data timer is still counting, as in I can still see the replay window counting the time....

But the chart stops updating. Which makes me think there is just a gap in the data?

So if you replay that day... does your chart continually update that entire time? As my chart just stops, and doesn't receive updates for about 10 minutes.

Just for clarification, 9:05CST == 10:05EST/NY time.

[NinjaTrader_Zachary]

Hello forrestang,

Thank you for your reply.

I'm not in EST personally, I'm in MST but I changed my PC time to CST while downloading the data and conducting these tests to ensure that I was able to exactly reproduce the conditions. My charts for both dates updated continuously between 9:05 to 9:20 AM CST on 6/14 and between 8:30 AM CST to 8:45 AM CST when I was using replay. Do you experience the same when you delete the existing replay data and redownload it for those dates from our server? There are very slightly differences between your chart in the screenshot and mine, and it makes me wonder if perhaps missing data was backfilled at some point after you downloaded the data.

I look forward to your response.