Data Issue Last.ntd because it is used by another process

[djkiwi]

Hi Guys, I have been getting this error sometimes (see attachment).

Connection to Ninjatrader historical market data server lost. The process cannot access the file.last.ntd because it is used by another process.

When I get this I cannot get the chart up for the relevant instrument in this case TF, even repairing the database, refreshing the data and even deleting the tick data directory. Usually after a few hours I can get it to work with reboots etc which is painful. Now this hasn't happened for a couple of months and then all of a sudden it started happening again yesterday. I was thinking what did I start doing now that I was doing back then? Then I realized I started using windows process explorer again.

http://technet.microsoft.com/en-us/sysinternals/bb896653

When I shut process explorer down and rebooted I had no problems again. So I am thinking this is the cause of this very annoying issue. For some reason process explorer is using the data.

The purpose of this post is twofold.

1. Firstly if others have had the same problem try shutting down process explorer.
2. Does anyone at ninja know why this would be the case? I quite like process explorer so would like to keep using it when I use ninja.

I will monitor this issue as it is possible that it is not process explorer at all but I remember that I started to get these problems last time when I started to use process explorer about that time as well. I will report back if I still have the issue without using process explorer.


Cheers
DJ

[NinjaTrader_RyanM1]

Hi DJ,

Thanks for your observations here. This can happen with more than just process explorer. You could have security programs or antivirus cause a similar message as well. There's not much you can do unless it's possible to add an exception to these types of programs, so that they're not accessing NinjaTrader files at the same time as NinjaTrader. If process explorer has this functionality, add the folder Documents\NinjaTrader 7\ to their list of exceptions.

[djkiwi]

Thanks Ryan, it looks like I spoke too soon as the error is still occurring. This is a stripped down trading machine so does not have any antivirus software. The other thing is it seems to be on random instruments sometimes TF sometimes ES etc. At the moment it is only TF that is the problem.

The only way I god rid of this last time was to reset the database but then I lost all the replay data if I recall. Any other steps you would suggest?

Thanks
DJ

[NinjaTrader_Kyle]

Hi DJ,

Please contact us with a note to Support [AT] NinjaTrader [DOT] com. In the message, please include:

1.) A link to this forum thread.

2.) A copy of today's trace file.
You will find the file in the Documents > NinjaTrader 7 > Trace folder.
The file would be named 'trace.20120119.txt'.

[djkiwi]

Hi guys I am still having this problem all day and no matter what I do nothing works. I tried to repair the database countless times and also renamed the database to olddatabase so a new one started. Still the same problem. I suspect this is a ninjatrader issue as sometimes it is ES sometimes TF and now it is gold playing up. I also tried a new worksheet and recreated the charts and worked for awhile then corrupted again Any ideas on what this is (see below from the log)

server.Client.Loop0-5: 64.202.118.179/31654

Any other things to try from here would be appreciated..

Thanks in advance
DJ

Log items from today:


2012-01-24 17:13:08:577 (AMP) Data.Bars.GetBarsNow: instrument='FDAX 03-12' from='2012-01-24' to='2012-01-24' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0 session='Eurex Equity Index Futures'
2012-01-24 17:15:07:360 (AMP) Data.Bars.GetBarsNow: instrument='FDAX 03-12' from='2012-01-24' to='2012-01-24' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0 session='Eurex Equity Index Futures'
2012-01-24 17:18:17:617 (AMP) Data.Bars.GetBarsNow: instrument='GC 02-12' from='2012-01-24' to='2012-01-24' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0 session='Nymex Metals / Energy ETH'
2012-01-24 17:18:40:267 (AMP) Data.Bars.GetBarsNow: instrument='GC 02-12' from='2012-01-24 17:00:00' to='2012-01-24 23:59:59' period=1 Tick splitAdjusted=False dividendAdjusted=False bars=0 session='Nymex Metals RTH1'
2012-01-24 17:18:40:679 Connection to NinjaTrader data server lost: System.IO.IOException: The process cannot access the file 'C:\Users\DJ\Documents\NinjaTrader 7\db\tick\GC 02-12\201201241800.Last.ntd' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.Delete(String path)
at NinjaTrader.Data.Bars.Save()
at NinjaTrader.Data.Bars.ProcessEventArgs(BarUpdateEv entArgs e)
at NinjaTrader.Server.Client.Loop()
2012-01-24 17:18:40:679 ERROR: Connection to NinjaTrader historical market data server lost: The process cannot access the file 'C:\Users\DJ\Documents\NinjaTrader 7\db\tick\GC 02-12\201201241800.Last.ntd' because it is being used by another process.
2012-01-24 17:18:50:679 Server.Client.Loop0-5: 64.202.118.179/31654
2012-01-24 17:18:50:833 Server.Client.Loop1-5

[NinjaTrader_RyanM1]

DJ,

This message is very clear in that another program attempts to access other files at the same time as NinjaTrader. Unfortunately the message doesn't identify the program. You were likely on the right track in your first post with process explorer. Did you discontinue use of this program?

[djkiwi]

Hi Ryan, I had turned everything off on the PC. Here is what I think caused the problem. I use AMP-Zenfire and IB as my datafeeds. For some reason in the connection Options I had checked the use historical data for IB AND AMP-Zenfire. Because I am recording bid/ask data and recently doing some other things I think each connection was trying to write to the database at the same time causing the issue. All seems ok now.

Thanks
DJ

[NinjaTrader_RyanM1]

Great to hear it's all working again. I don't feel multiple connections would contribute, as there are rules that determine which connection is used for data and it is only 1 connection per instrument type. If it's something you are able to see and are sure there are no other programs contributing, please take note of the sequence that causes it and we will try to reproduce here.

[djkiwi]

Data Issue continued

Hi Ryan, I am quite sure this is a ninjatrader issue. Today I had the "historical data" disabled on both IB and AMP. I had no problems at all. Now I had a gap in the data so I enabled the "historical data" on the AMP connection. Immediately I had the problem (please see attached) with TF when I tried to reconnect with the AMP "historical data enabled". Note it is only TF this time. So not sure why only this instrument would be affected. I did not have IB historical data enabled.

Any ideas of what should I do next to fix this annoying issue?

UPDATE: I just tried to login again with data enabled and got more errors plus an error loading chart issue for GC (although the charts loaded). See data issue 4. Es, 6E and CL all show "loading data" on the chart.

Thanks in advance
DJ

[NinjaTrader_RyanM1]

Please send your trace file to support 'at' ninjatrader 'dot' com, and include a link to this thread.

You will find the trace file in the Documents > NinjaTrader 7 > Trace folder.

* The trace file will be named "trace.YYYYMMDD.txt"

[djkiwi]

File Sent

Hi Ryan, file has been sent as requested. Today I uninstalled and reinstalled ninjatrader and still no difference. Also I have only ninjatrader on this computer so no other programs that I am aware could cause it although of course it is possible. I have shut down process explorer and windows defender.

The problem only occurs when loading historical data probably because the database is being accessed.

If I uncheck the "get data from server " option in the data options then I have no problems. I also have tried settings as "do not merge" still no luck. Normally I an fix these things but this has me stuck for 2 full days because of this issue. Even renaming the database to .old didn't fix the issue. I'd be very grateful for a resolution to this.

Thanks
DJ

[NinjaTrader_RyanM1]

Thanks DJ. We received the files and will follow up with you by email.

[BradB]

I would be interested in hearing the result also as I have this problem occasionally myself. Usually I can get past it by deleting _Workspaces.xml and restarting Ninja, so Ninja starts without a workspace and then use the Historical Data Manager to reload data for the problem instrument. I also turned off windows indexing on the Minute and tick folders today to see if that might help.

[NinjaTrader_RyanM1]

Hi BradB,

We have not yet identified anything here. The only time we have seen this there have been external programs contributing. Can you please send your latest trace file to support 'at' ninjatrader 'dot' com and include a link to this thread.

You will find the trace file in the Documents > NinjaTrader 7 > Trace folder.
* The trace file will be named "trace.YYYYMMDD.txt"

[ag987654321]

Ryan,

I am also having the same issues repeatedly. Has a resolution been identified?
Would my tracefiles be helpful? I have had this issue many times today and am running NT 64B
7.0.1.0000.9 and connecting to IB for live data and IQFeed for historic data