Market Replay issues for 30-Jan-2017

[NinjaTrader_Jason]

Can you please send me your db-->data folder, so I can test it on my end. Please right click the data folder and select Send to-->Compressed (zipped) folder. Subsequently upload the .zip file via sendspace.com or a similar service and send it to my email.

[ntuser2017]

Can you please send me your db-->data folder, so I can test it on my end. Please right click the data folder and select Send to-->Compressed (zipped) folder. Subsequently upload the .zip file via sendspace.com or a similar service and send it to my email.

Hi Jason

Since the entire data folder is nearly 1 GB, would you prefer if I sent you just the affected days?

[NinjaTrader_Jason]

Yes. If the folder is that large, please send me only the affected days.

[ntuser2017]

Yes. If the folder is that large, please send me only the affected days.

Jason, I have sent the zipped file via Sendspace.

The filename is CL-MarketReplay.zip


If you have any need to communicate to me via email please continue to use the other email address and do not use the one used for Sendspace.

Thanks

[NinjaTrader_Jason]

Thank you, I downloaded the file.

Sounds good, I will respond to that email.

[ntuser2017]

Thank you, I downloaded the file.

Sounds good, I will respond to that email.

Hi Jason

I sent an email last night saying that by following your instruction the problem had been addressed.

However, I keep incurring in the same issue if I try to download more days.

It appears the issue somehow originates when Market Replay files are being downloaded.

I tried to disable Antivirus and Firewall but that didn't seem the cause.

I am thinking it could be my router that perhaps is interpreting some of the packets as intrusions and corrupting the data.

Is there a specific port or setting I should configure that you know of?

[NinjaTrader_Jason]

I do not think it is related to you blocking a port or IP address. If this would be the case, you would not be able to download Market Replay data at all. The server from where Market Replay data is loaded uses port number 31654.

Am I correct you download Market Replay data at Control Center-->File-->Utilities-->Download Replay Data or do you obtain the data in a different way?

Are you able to test a different router?

[ntuser2017]

I do not think it is related to you blocking a port or IP address. If this would be the case, you would not be able to download Market Replay data at all. The server from where Market Replay data is loaded uses port number 31654.

Am I correct you download Market Replay data at Control Center-->File-->Utilities-->Download Replay Data or do you obtain the data in a different way?

Are you able to test a different router?

Yes, I download Market Replay in the way you described.

I'll see if I can dig out an older router and try with that one. I'll keep you posted.

[ntuser2017]

Update

Hi Jason

I have managed to get hold of a separate PC.

On it I downloaded NinjaTrader 7 from scratch. Note: downloading and installing NT7 was fine.

I then attempted to download CL Market Replay for Nov. 21st 2016 (CL 01-17).

L1 download appeared to go fine. However the PC stopped in the middle of downloading L2 data.

I waited over half an hour with nothing happening. I then decided to close NT7.

I deleted the db\data\20161121 folder which contained the partially downloaded data.

I then attemped to download again. This time it did download.

Again, on this PC, the data downloaded exhibited the same behaviour as on my own PC.

Note: when this failed download took place it reminded me that I had a similar occurrence on my own PC, before which I had not experienced Market Replay issues.


This is leading me to think to one of this possible causes

• It could be a router issue. Certain Market Replay days may have data packets that my router interprets as a malicious signature and attempts to block them or corrupt them somehow
• Something taking place during the download that corrupts the data in transit
• My own Continuum account may have some corruption of its own and may be in the need of resetting

I am a bit at wits' end on this. Anyway, tomorrow I will try with the old router as advised.

[ntuser2017]

More updates

Hi Jason

I got a colleague to download the 21st nov data and send it to me.


Once again, the data shows the same anomaly.

This, contrary to my previous post, would exclude a router or a download issue.


I have run out of logical explanations.

[ntuser2017]

Hi Jason

I went past the 21st Nov and tried downloading the 18th Nov 2016 (again CL).

That plays fine.


Could it be that the 21st is actually corrupted at NT's end?

[NinjaTrader_Jason]

When I download and replay November 21st 2016 for the CL 01-17, I experience the issue as well. Unfortunately Market Replay data for this date is corrupted.

[ntuser2017]

When I download and replay November 21st 2016 for the CL 01-17, I experience the issue as well. Unfortunately Market Replay data for this date is corrupted.

Thank you for checking Jason. I am relieved in a way because at least that was a red herring.

I will keep monitoring the situation.

For now, if a day downloaded with issues I will keep trying using the 'rename data folder' technique to download that individual day and integrate with the existing folder structure.


Many thanks for all your help so far.

[ntuser2017]

When I download and replay November 21st 2016 for the CL 01-17, I experience the issue as well. Unfortunately Market Replay data for this date is corrupted.

Hi there Jason

Unfortunately that Market Replay issue is reoccurring on yesterday's CL data (13th Feb on CL 03-17).

I tried renaming the data folder and also renaming the entire db folder and re-downloaded the 13th of Feb twice but it still shows the issue.

Would you be able to check that the data is not corrupted at your end please?

[NinjaTrader_Jason]

I am able to reproduce the issue on my end when I download and replay February 13th as per the CL 03-17. L1 and L2 data gets out of synch around 12:40 AM EST. I will report the corrupted data to our server team.