NT Reliability

[NinjaTrader_JoshP]

You are most likely drawing data simply because of the first rule: Format Data Series” contains the current day
or maybe the second rule: Format Data Series” contains the last day of data available in your database

This is necessary because NT doesn't know if you actually have the full day's data. You might have ended your session early. Therefore it needs to go and retrieve and patch the holes.

I suggest you just download the data you want then disconnect from your feed. Like you stated, there is nothing coming in from your feed so simply disconnecting from it will guarantee you draw from your database.

[greentrader]

This is necessary because NT doesn't know if you actually have the full day's data. You might have ended your session early. Therefore it needs to go and retrieve and patch the holes.

I suggest you just download the data you want then disconnect from your feed. Like you stated, there is nothing coming in from your feed so simply disconnecting from it will guarantee you draw from your database.

I understand the point about NT not knowing whether there might be new data, but not why it appears to be tossing out ALL the cached data for ALL symbols and then reloading it all (i.e., it's not just patching holes for specific symbols). Anyway, not a big deal for me at this point, although it could be a problem for anyone trading with NT who needed to restart NT while in a trade. It appears to me that the NT cache is not behaving optimally, even on a Saturday night with all markets closed. Hopefully this will improve in later NT updates.

With respect to staying disconnected in off-hours, that usually won't work since, during development, one is changing lots of stuff that often requires historical data access.

Thanks again for the quick response! I'm sure good support is one of the things traders most appreciate about NT.

[NinjaTrader_JoshP]

If a call to load data occurs it will occur on all instruments that hit the rules. If some instruments have charts that do not hit the load data rules they will not load. In most cases the rules are always hit no all instruments simply because people will have the data series loaded out to the current day. Thanks for your feedback though. We will take note of it for consideration in future versions.

You should be able to run backtesting without the requirement of a data feed to load data. Connect once, go through all your instruments and load data to the current day. Load up tick, min, daily etc. This will provide you with a complete up-to-date database and then you can disconnect. Provided you do not decide to add a new instrument you don't need a data feed from that point forward. If you are adding new instruments to your backtesting then unfortunately it will be a slight hassle for you to reconnect to your data provider.

[zoltran]

Right ... but if you do disconnect.. then you can't attach a strategy and/or all strategies will be removed from a chart.

Odd, odd, odd logic.

[greentrader]

some results

As mentioned elsewhere, I'm in the process of converting a bunch of stuff from Tradestation to NT. In previous posts in this thread, I mentioned being dissappointed that NT was not smarter about historical data reloading (esp. compared to Tradestation), and have learned a bit more...

My biggest surprise was to find that loading historical data based on "bars back" is always several times slower than when using "days back", even when the number of bars being retrieved is the same (using IQFeed). So, for example, if you ask for 5 days of index minute bar data, this is MUCH faster than asking for the equivalent 5 * 390 = 1950 minute bars. So, when setting up charts, it really pays to use "days back" rather than "bars back".

Unfortunately, for MarketAnalyzer windows, you're stuck using "bars back", and their data (re)loading time suffers greatly. This is compounded by the fact that such windows typically have many symbols to reload, which can make historical data reloads quite time-consuming. (And all of this is made worse by how often NT unnecessary reloads all data in such windows.)

Another thing I encountered that about doubles the time it takes to load historical data is the use of continuous futures contracts (vs. front month) from IQFeed. This is an IQFeed issue, but I actually had an MA window with several continuous contracts in it, and so it was particularly slow to reload data.

As an example of the above effects, reconnecting (and reloading) historical data in the presence of an MA window based on minute "bars back" and continuous contracts took 8 minutes 40 seconds. Converting that window to be day-based (i.e., the "bars back" became an equivalent number of "days back") dropped the total reload time to 31 seconds!

BTW, the above results were not dependent on how the minute bars were organized (i.e., 1-minute vs. 5-minute bars), nor was the CPU ever maxed out. The difference seems to be in whatever algorithm NT is using to load "bars back" vs. "days back".

Anyway, I'm happy to have found a way around the long load times, and now know a couple of things to avoid... (And sorry if this is already documented somewhere, but I didn't recall seeing it.)

[NinjaTrader_Dierk]

Unfortunately you're experiencing limitations related to the "bar back" logic. Some of them will be overcome with NT7 (beta by the end of this year).

[NinjaTrader_Jessica]

Hi Harry,

Are you able to reproduce this if you limit it down to one instrument (just for testing sake)?

Then try all of your current instruments but with no indicators or strategies in your charts.

Also what happens if you just let it sit and don't hit abort? Does it time out or go ahead and fill?

Please send me your answers to support At ninjatrader DOT com attn Jessica.

[Harry]

Hi Jessica,

I know that IB backfill is sometimes painfully slow, which of course is not caused by NT. My only concern is that NT freezes if I want to abort the backfilling. If I do not hit "Abort", the backfill will possibly be completed after some time, today it took over an hour. I basically hit the "Abort" button after deciding to do the backfilling at a later stage, when there are fewer crowds trying to do the same thing.

I will check, if this also applies to a single instrument and come back to you via Mail.

Meanwhile I got my usual Swiss Cheese chart (before connecting, see below), which is the other problem. This chart was complete before I turned off NT yesterday evening at 18:40, today after having launched NT again part of the data is missing. This often happens and this is not linked to IB, as the data was well in the data base, after I had disconnected from IB yesterday.

[NinjaTrader_Jessica]

Hello Harry,

I will watch for your note to support AT ninjatrader DOT com.

[Harry]

1 year later - note to the developpers

Starting NinjaTrader with DOM and 5 charts (out of which 4 display the same instrument with different time frames). When connecting to IB, charts will first show horizontal lines only and usually will be completed one by one. Quite often, however, one of the charts will not complete.

From that point on it is impossible to open any new chart, even not the simplest one, as it will display "Instrument 1 min" in the upper left corner and remain empty. NT has to be shut down. Restarting NT now will show the control center in a different size and location (original location). No way to make it work correctly any more. So I need to restore everything, before trying my luck again.

Also there is another current problem: Some of the frames that are displayed in the windows task bar do not show any labels. Clicking on them with the left mouse button will not redisplay the chart, this can only be achieved via the menu that can be opened with the right mouse button.

This is a really tedious behaviour of NT. I would give NT 6 out of 5 stars, if it did not crash all the time! I hope that reliability will improve with NT 7, I do not need all the nice-to-haves announced, but I would greatly appreciate if you managed to turn this capricious lady into an obedient servant.

[NinjaTrader_Kyle]

Hi Harry,

Thank you for your reply.

Please send a note to Support [AT] NinjaTrader [DOT] com with "ATTN: Kyle" in the subject line. Please include your log and trace files from a date when this occurred.
You will find your log file here: My Documents > NinjaTrader 6.5 > Log > log.YYYYMMDD.txt
You will find your trace file here: My Documents > NinjaTrader 6.5 > Trace > trace.YYYYMMDD.txt

[Harry]

Thank you for your reponse

No problems today, will come back to you with the files when problem occurs again.

One major problem of NT 6 is the ever growing data base. Is there any way to get rid of the data of outdated future contracts, if I want to reduce the size of the data base?

I saw that NT 7 will have a Historical Data Manager. Does the Historical Data Manager allow for scrapping unwanted data or eliminating bad ticks?

[NinjaTrader_Kyle]

Hi Harry,

Thank you for your reply.

1.) That sounds good. Please let us know if this does occur again.

2.) Renaming or deleting the NinjaTrader.MDB file in the My Documents > NinjaTrader 6.5 > DB folder will cause NinjaTrader to create a new database.

3.) Yes, the historical data manager in NinjaTrader 7 will allow you to do this.

[Harry]

Deleting is not an option

Deleting the database will unfortunately also destroy the data that I still want to use.....

[NinjaTrader_Kyle]

Hi Harry,

Thank you for your reply.

That is correct, renaming or deleting the database will delete historical chart data, historical execution data as well as ATM strategies.


You could, however, delete historical data for expired contracts using the Tools > Historical Data > Delete menu. For more information about deleting historical data, please see the following link: http://www.ninjatrader-support.com/H...V6/Delete.html

*Edited to include historical data deletion information.