Ninjatrader keeps downloading data it already has

[cunparis]

I run a backtest with a few stocks and NT downloads the data.
But when I rerun the backtest, without changing anything, NT downloads the data again. Sometimes it takes several minutes to get the data (that it already has) and this is quite frustrating for me.

Why would NT spend several minutes getting data it already has?

I know I can disconnect from IQ but the problem is I change the stocks and dates so I'd be constantly connecting & disconnecting.

Resetting my instrument list isn't an option because I've added a lot of instruments.

[NinjaTrader_Jason]

Hello cunparis,

Please see the link below that provides more information how NinjaTrader loads data.

http://www.ninjatrader-support.com/HelpGuideV6/WhenDoesNinjaTraderLoadHistoricalData.html

[cunparis]

Hello cunparis,

Please see the link below that provides more information how NinjaTrader loads data.
http://www.ninjatrader-support.com/H...ricalData.html

if I understand correctly, when backtesting I should ask for 1 day more of data the first time and then from then on I just use the dates I need. Since NT will have the extra day, it won't need to check for new historical data. Is that correct?

[NinjaTrader_Jason]

Correct, it should work that way, since subsequent backtests use historical data which you loaded during the initial backtest.

[cunparis]

This is driving me crazy! It's still happening. Here's what I did:

I download data for 35 stocks, minute data for a 2 month timespan. For some reason with IQFeed this takes 10 minutes or so but I suffer through it.

Then I change the backtest date for the 1st month, the very same month I already downloaded. I run my backtest and NT goes through its loading data for a long time! It already has the data, my new dates fall inside the dates of the first backtest.

Here's my theory: I changed the dates to do 1/1/08 - 1/31/08 (first run was 1/1/08 to 2/27/08). The 31st is a saturday and NT didn't have data for it so it requests it again from iqfeed. Could this be possible?

What other explanation is there? It's very frustrating and unproductive for me to wait 10 minutes for a backtest every time I run it.

[NinjaTrader_Ben]

Hello,

It should not reload if the data is already in your database.

Is the strategy working and that is what is slowing it down?

I will have Jason review this thread on Monday as he may have a better suggestion/answer.

[cunparis]

Hello,

It should not reload if the data is already in your database.

Is the strategy working and that is what is slowing it down?

I will have Jason review this thread on Monday as he may have a better suggestion/answer.

This always occurs before the strategy runs. The output goes from empty to terminated in less than a second.

When I check the trace, it is going to iqfeed. This process takes up to 27 seconds. You can see that 15 seconds average * 100 symbols is 25 minutes. The question is:

1 - why is NT going to iqfeed for data?

2 - is iqfeed being slow (the 27 seconds?) or is it NT? In my IQFeed client I see the # seconds since last request and it goes up to 27 which makes me think it's NT that's slow in processing it.

[NinjaTrader_Bertrand]

Hi, on your 'weekend theory Sat the 31st' - did you try running these backtest with 'Exclude weekend' to true?

[cunparis]

Hi, on your 'weekend theory Sat the 31st' - did you try running these backtest with 'Exclude weekend' to true?

Yes exclude weekend is always true for me.

[NinjaTrader_Jason]

Hello cunparis,

What historical data does your database contain; what was the last historical data request for the same instrument.

If you now use the same amount historical minus one day compared to the last date, it should load the data from the database.

[cunparis]

Hello cunparis,

What historical data does your database contain; what was the last historical data request for the same instrument.

If you now use the same amount historical minus one day compared to the last date, it should load the data from the database.

If it loads from the database, would it print in the trace that there is a call to IQFeed? If not (I assume no), then I can easily tell from the trace. ANd if it's going to IQFeed when it shouldn't then we can show there is a problem.

[NinjaTrader_Jason]

Hello cunparis,

Does NinjaTrader load data from IQFeed when using the 1/1/2008 - 1/30/2008 dates after loading historical data from 1/1/08 to 2/27/08?

[cunparis]

Hello cunparis,

Does NinjaTrader load data from IQFeed when using the 1/1/2008 - 1/30/2008 dates after loading historical data from 1/1/08 to 2/27/08?

From what I see Ninjatrader is always loading data from IQFeed when the dates change, even if the date is shorted by one day. Here is the trace from a few minutes ago:

2009-03-17 15:46:23:978 (IQ) Data.Bars.GetBarsNow: instrument='NQ ##-##' from='2006-03-16' to='2009-03-16' sessionBegin='00:00:00' sessionEnd='00:00:00' excludeWeekend=False period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2009-03-17 15:51:05:007 ERROR: Unable to create instance of 'NinjaTrader.Strategy.ToolAccount'. Most likely there is no default constructor defined, or the default constructor does not work correctly: Constructor on type 'NinjaTrader.Strategy.ToolAccount' not found.
2009-03-17 15:51:05:007 ERROR: Unable to create instance of 'NinjaTrader.Strategy.ToolAccount2'. Most likely there is no default constructor defined, or the default constructor does not work correctly: Constructor on type 'NinjaTrader.Strategy.ToolAccount2' not found.
2009-03-17 15:51:05:348 WARNING: Starting Strategy
2009-03-17 15:51:05:351 WARNING: Starting Strategy
2009-03-17 15:51:21:298 (IQ) Data.Bars.GetBarsNow: instrument='NQ ##-##' from='2006-03-16' to='2009-03-13' sessionBegin='00:00:00' sessionEnd='00:00:00' excludeWeekend=False period=1 Min splitAdjusted=False dividendAdjusted=False bars=0

Making matters worse is the fact that it takes 5 minutes to "download" the data. This is really frustrating to me.

ps: I just ran the backtest again and it did not load the data. So it appears it's definitely a problem with changing the dates.

[NinjaTrader_JoshP]

cunparis,

Change the dates for what? If you are backtesting just disconnect from your data provider.

Please make sure NONE of the loading requirements are hit if you do not want it to load. For instance this one: The database contains no data within 3 days of the “From” date.

Just because you've told it to start at a certain date does not mean your data provider is actually giving you that data and loading into your database.

[cunparis]

cunparis,

Change the dates for what? If you are backtesting just disconnect from your data provider.

Please make sure NONE of the loading requirements are hit if you do not want it to load. For instance this one: The database contains no data within 3 days of the “From” date.

Just because you've told it to start at a certain date does not mean your data provider is actually giving you that data and loading into your database.

I can't disconnect because I'm using NT for charting. Although I'm seriously considering purchasing another charting software for charting and using NT exclusively for strategies.

I still don't get it. I run back test, it loads data. i reduce the backtest period by one day, it reloads the data. I understand if the last day of the backtest is the current day it'll reload but if I reduce it by a day it doesn't make sense why it would reload.