A note for the people who manage NinajTrader development

[Harry]

Please, fix the bug!

The only way to check, whether it is still loading, is to look at the processor load within the task manager. If nothing happens any more, you better shut down NinjaTrader. Then go back to the task manager again, kill the NinjaTrader.exe process, which is still running on its own. Then try again, and it might work.

To the developpers: NT is a really professional piece of software, which I love to use. But I wonder why you would not acknowledge that there is a major bug that locks NT every day. Come on, take the challenge! First accept that there is a bug (several users do confim it), now fix it, please!

[r2kTrader]

Cunparis,

Yes, testing for data would be key here. But that does not mean that the feed will send whether or not it has the data? So it may work with some data providers and not others.

Some type of timeout in options would seem fitting. I know the issue you are talking about.

For me the main problem on this crashing and getting stuck downloading data is that one cannot tell if NT is going to successfully download the data or if it's going to be hung all day.

Couldn't the loading data dialog show some kind of progress? Couldn't it check to make sure the data is available before getting stuck? Like if my data provider has data from 5/2006 to present and I accidently demand 1/2006 to present (which is my case right now as I type this) then the loading dialog just hangs forever. So I have to exit and restart. If NT would just ask the data provider if there is data from 1/2006 it could reply no and NT could return with a msg "no data for 1/2006". ????

I have to exit and restart NT often because I mess up with my dates. it's very frustrating.

[NinjaTrader_Dierk]

@Harry: Could you please provide a reproducible scenario where you feed NT got stuck so we could look into? Thanks in advance

[Harry]

Unfortunately, the problem is not reproducible. If the problem occurs, I have to shut down NT, remove NinjaTrader.exe from the task manager and retry again. Most of the time my second try starting NT and connecting will be successful. Sometimes I have to restore the last backup, and on rare occasions I have to reinstall NT to make it work.

I only can describe what is happening (sorry for the detail, but it might help to solve the problem):

(1) Starting NT, workspace opens. Workspaces contains CC, 3 intraday charts connected to DOM, 1 daily chart, 1 intraday chart for TICK, 1 market analyzer with 20 instruments. The three intraday charts have different lookback periods between 65 and 120 days. There is no second or hidden workspace.

(2a) Connecting to IB. If everything goes well, charts and instruments of the market analyzer will update within 1 or 2 minutes. The "Loading" popup can only be seen, if I put the market analyzer on top of my desktop. While the popup reads "Loading..." in the upper left corner of the popup, all the instruments will be called one by one (instrument and last date displayed by popup). During this process the popup will alternately display the instrument loaded (including last date) and the information "Saving".

(2b) Sometimes this process gets stuck after loading the instrument displayed on the chart. The Saving popup will show forever without any processor load (nothing happens). Aborting is not possible, but will simply convert the "Saving" popup to an "Aborting. Please wait.." popup, which will also remain forever. The charts just display some horizontal lines, separating the indicator panes, but no data.

If I shut down NT in ths stage, the data base is likely to be destroyed. I found a way out of this by pretending to close NT, but then selecting "Cancel". This will remove the "Aborting" popup. Now NT will try to continue to load data for the remaining instruments of the market analyzer. This time the instruments will be displayed each without the date, and the "Saving" popup that comes after each instrument can be manually aborted. So I will click "Abort" a 15 or 20 times until all instruments of the market analyzer have had their turn. Now I will shut down the connection and close NT.

Now the desktop of my PC is empty, and you might think that NT has been shut down. Not so. It is still running in the background, so it cannot be restarted. To solve this, I have to use the Task Manager and remove NinjaTrader.exe from the running processes. Having achieved this, I can restart NT.

(3) After restarting NT and connecting again, usually it will load data without any problems. If it does not, I may restore yesterday's backup, and it is likely that everything works afterwards.

@Harry: Could you please provide a reproducible scenario where you feed NT got stuck so we could look into? Thanks in advance

[NinjaTrader_Dierk]

Please send your workspace and your zipped DB to "support AT ninjatrader DOT com", subject "for Dierk" and refer to this thread. Thanks

[Harry]

Thanks for your reply.

Thank you for your reply. Will send you DB, but first clean it up and rebuild it to make it easier to handle. May take a couple of days ... as I am not in front of my screen all day long.

[NinjaTrader_Dierk]

No problem. Take your time. The more likely it's reproducible the more likely we would able to track down this issue.

[NinjaTrader_JoshP]

Please also send in your workspace. Thank you Harry.

I have tried your instructions with the database you sent in so far, but could not reproduce.

[Harry]

Currently my NT installation works alright. I could not reproduce the problem either myself. I will try again and look what else it needs to cause the freeze and then come back to you.

[Harry]

Crash again

Did only have to wait for the next day. This morning after connection to IB, after loading ES to the charts, the famous "Saving" - popup showed and would not go away (this was a brand new installation of NT). Instruments on the MA would not be loaded. Would have to abort and then play the usual tricks as described below.

Peculiar: While connected the ES data would show correctly on the intraday charts. After shutting down and restarting NT the part of the ES data between 11:30 PM last day and the moment when the backfill was executed will not show on the charts, but leaving a hole (see attachment). The second time, I connected NT to IB, backfill was fast and easy. But the chart problem with ES remained (only when being offline). So NT has once more destroyed its database.

Not traffic related: At the moment of backfill (7:30 Central European Time or 1:30 Eastern Time) all markets were quiet and IB backfill fast and without problems. This means that the problem is not related to heavy traffic.

@Josh : I will send you tracefile (logfile does not contain any information) and workspace. Please let me know, if you need further data.

Currently my NT installation works alright. I could not reproduce the problem either myself. I will try again and look what else it needs to cause the freeze and then come back to you.

[Harry]

Problem is real and cannot be fixed

Josh, I appreciate that you have tried to help, but unfortunately without any success.

So let me summarize:

- NT hangs about twice a week, when I try to download backfill data
- a number of other customers have the same problem
- the problem is not reproducible, which means that under the same conditions it sometimes occurs, and sometimes doesn't

The only information I can provide for is that the problem also occurs during low traffic times, so it is not at all traffic or resource related. Every single time that the problem occurs the trace file shows that the downloading of data for the instruments for the charts or the market analyzer was interrupted by NT performing an IB.AccountUpdatesrequest.ProcessUpdate operation. After this request NT could not continue to update the instruments but was hanging and needed to be shut down.

I do not know why and when NT performs this Account Update, but it may cause the problem.

I also searched the EliteTrader Forum, and somebody who had similar problems(not using IB but Mirus) wrote that he now regularly resets the data base as there was negative interference between executions stored by NT and downloaded from the account.

Please let me know if you further enquire into this.

[NinjaTrader_JoshP]

Harry,

Unfortunately in our numerous tests on multiple machines with your settings we were just unable to even reproduce once. We will be glad to reinvestigate with more information, but at this time we have already looked down the IB.AccountUpdateRequests.ProcessUpdate road and do not believe that to be the cause.

[Harry]

Another suggestion

When I rollover the futures contracts, I would like to get rid of the old contracts that are no more needed. These increase the size of the data base unnecessarily.

Because this cannot be done, I need to rebuild the data base regularly, which is rather unconvenient.

I think there should be some features in NT to allow the user to modify the database!

Besides today my database was destroyed again. Why is NT so unstable?

[trader65]

data management tool

imho what Ninza really needs and this should be no 1 priority is a data management-maintanance tool, like Omega's Globalserver....where the user can edit modify delete recorded data...perform stock splits and
even rollover and backadjust futures data...

regards
Kon

[NinjaTrader_Jason]

You can remove the 03-09 instruments from your instrument list as per the instructions at the link below.

http://www.ninjatrader-support.com/HelpGuideV6/AddRemoveFromList.html

I will forward the suggestion to support a data management tool to our development team and ask them if they can add this to the list of future considerations for the software.