Data price disruption that makes no sense

[pstrusi]

Hi NT,

I don't still know why this issue continues to happen. Like the old saying " an image explains better than thousand words". Please take a look at that chart attached. I use that scale to perceive better what I'm trying to show. It's a simple and abrupt price change that it's not a normal market move, it's an issue cause there was not any kind of error, malfunction of NT, PC...etc

I've tried in the past all your instructions for avoiding this but still...

Any ideas?

Thanks in advance

[pstrusi]

I've found a couple more of these interruptions that are not registered either in NT Log or trace files. As you can see tin those charts, there is 8 minutes of not data, despite my pc and connections were fine. At first I thought that was something cause my normal working, editing, compiling..etc; but now I can see these event occurred exactly an hour of difference, so there must be something between IB and NT. I was checking the IB's API log files in that day, but I've not found any possible recurrent cause.

So, this is a matter quite urgent to address, because this creates a complete havoc in my data.

I repeat my platforms:
Windows 10 home, 64b
NT 7.0.1000.30
IB TWS and Gateway 944.3
Java , last: 8.66

Thanks in advance

[pstrusi]

Since this is frustrating issue, I continued to search for any possible cause, so in order to give you all the info available, I update this post.

The issue began this week 22-27 , but before that, there was not any issue at all, so I discard any trouble with IB since is the same connection. That weekend, just before the issue, I had to reinstall NT cause doing a heavy backtest, the application hanged. Now I wonder what could have gone wrong with that fresh reinstall ?

I decided to uninstall and reinstalling NT again, deleting any cache files. I hope not to see this again.

Any ideas, suggestions?

Thanks

[pstrusi]

Updating post:

After uninstall, delete all NT folders and install NT again, I've been loading data seamlessly so far. However, if you have any additional review, idea or procedure to verify, please let me know.

Thanks

[NinjaTrader_PatrickH]

Hello pstrusi,

It is quite inconsistent and as we discussed without the ability to investigate first hand it is hard to determine the cause.

Please let us know of any occurrences. I would also ask that we keep one thread (possibly this one) open on this rather than three.

[pstrusi]

My apologies for multiple posts.

What is inconsistent ? As you can see in the charts attached, it happened in that way. I'm pretty aware is not easy for you determining the exact cause without being present, still I thought that perhaps you could have some suggestions or basic procedures to do a kind of"reset" .

No further occurrences, otherwise I had updated this post. What I did yesterday seems to have addressed the problem.

Thanks anyway

[NinjaTrader_PatrickH]

Hello pstrusi,

No worries on the multiple posts. I meant that the case occurrences seemed to not be connected to a simple item.

I believe you have already created a completely new installation in the previous case. The following would be my next steps.

With the fact that the tick data is not supplied to NinjaTrader from the API, if there is any drop out of the connection to Interactive Brokers then the data would not be present.
If you see that the connection is stable and no disconnects are logged in the Log tab of the NinjaTrader Control Center, you can check to see what version of TWS you are running as NinjaTrader 7 currently supports TWS 944.3 only.

If the installation is new and not using the directory (My Documents\NinjaTrader 7) from the previous install, meaning you removed or renamed it, and TWS is the supported version with no disconnects logged I would consider checking the tick data in TWS to see if the data is not present there as well.

If the data is present then something is preventing the data from being communicated to NinjaTrader 7 from TWS. Programs that scan files and/or applications can prevent data from being communicated between two applications (NinjaTrader and TWS for example).

[pstrusi]

Hi Patrick,

Unfortunately, it happened once again. Briefly:
NT was all Friday doing simple simulations and loading data seamlessly. Market closed at 17:00 pm ( Forex ) and I checked all was in order. In that state I decided to do a backup ( it takes a couple of minutes ) when finished then I closed NT. Then I start NT again and once again, the frustrating surprise to see data loaded only till 16:51:15, incredible. Not connection failures, nor hangups or anything that justifies that. I checked Log and Trace files and there's not any message about it.

The only feasible cause that I might think is that during that time, I was checking some price action hours before ( in the very same chart where data is loading ) simply by moving backwards through the chart but shortly set to the most recent price.

This is really annoying, frustrating besides that now I'm afraid to work in my codes when markets are opened.

Any ideas, suggestions, solutions ? A help here would be highly appreciated.

Thanks in advance

[NinjaTrader_PatrickH]

Have you followed the instructions below?

I would consider checking the tick data in TWS to see if the data is not present there as well.

If the data is present then something is preventing the data from being communicated to NinjaTrader 7 from TWS. Programs that scan files and/or applications can prevent data from being communicated between two applications (NinjaTrader and TWS for example).

[pstrusi]

Hi Patrick. Thanks for your attention.

As I've written earlier "All data was in order and completely loaded till 5:00 pm, so there's no any communication problem between TWS and NT cause I had all data in my chart. The issue comes when you close NT then in any moment that you need to start it up again, some data is missing, so how to explain this? It seems that there's some problem or interference with some "cache" file. I really don't know what's happening, that's why I come here. Having said this, if you have further new ideas, please share to try'em out.

Thanks in advance

[pstrusi]

Hi NT,

Once again a very annoying, mysterious and still not solved data issue happened to me yesterday.

Brief description:
Markets closed, I checked all data charts were completely loaded till final time session and immediately I did a backup "just in case". Well, being Friday, I procedure to turn off all my programs and pc ( HP notebook ). Saturday in the morning, started the pc then NT, bam! once again $EURUSD data just loaded till 16:21 and ES 12-15 even worse, till 9:30. Seeing this, I decide to restore the very last backup from Friday night and still the database continues to be the wrong database with a lot of disappeared data.
How to explain even the backup ( with full data ) fails?

How to solve it?

I've followed all your instructions but the problem continues.

I've been checking all Windows 10 internal procedures, just in case, one of this might be interference with NT: Time sync, Optimizations...etc but nothing yet.

Any ideas, suggestions?

[NinjaTrader_PatrickH]

Hello pstrusi,

When was the last time NinjaTrader was open on Friday?
Was the EURUSD and ES 12-15 open during that time and viewed on the screen?

[pstrusi]

Hi Patrick, yes, they were uploading data till session finals, without any trouble, all data loaded, but whenever I close NT and later start it up again, then the issue: data disappeared. It's like a random issue, not same times, in fact ES erased data backwards till 9:30am...from 5pm , ???? EURUSD was backward from 5pm till 4:21pm

[NinjaTrader_PatrickH]

I would try the free Forex demo at the following link and see if you have the same problem with a data feed provider that supports historical tick data: http://ninjatrader.com/Forex#!Forex_Commissions

[pstrusi]

Patrick, as you have read, the complete data session is loaded in the chart, then obviously in the database, so it shouldn't disappear without any reason. A provider that supports historical tick data would only give me the opportunity to reload missing data, but I wouldn't need it, whatsover, if I had had my NT loading my real-time data perfectly fine and visible in the right charts, so I'm still looking for reasons of this and solutions. I hope you can figure it out as expert support team.