Daily bars limit?

[dougfir]

When I create a chart for TMW (DJ Wilshire Total Market Index) and request 365 days of daily data I only get data back to 3/11/08.
I've also tried requesting 2 years, 500 bars, start date to stop date, etc., but the max it goes back is to 3/11/08.
I also tried other symbols such as SKF (UltraShort Financials ProShares ) with the same results.
I'm using IB's data feed and when I open a chart in TWS(v890.6) and request 2 years of daily data for those symbols, it opens just fine with all that data so I know IB has the data.

What can I do to get the data into NT's charts? (using NT 6.5.1000.7)
John

[NinjaTrader_Bertrand]

Hi John, please try this using the latest NinjaTrader 6.5.1008 release in combination with TWS version 887.2. This can be found in our IB connection guide - http://www.ninjatrader.com/webnew/su...ivebrokers.htm

To update your NinjaTrader, you can click Help > Download Site in the software.

[dougfir]

Thank you,

Not sure I want to downgrade TWS, don't know if I have to downgrade their API as well. I'll look into that.

Will this problem go away with NT 7?

John

[NinjaTrader_Bertrand]

Hi John, please try it with the TWS we recommend using from our testing in the connection guide for IB. I'm not sure how NT 7 will affect this issue, but we expect general performance to increase overall. Thanks!

[dougfir]

Hello again,

I followed your instructions and upgraded NT to v 6.5.1008 and downgraded TWS to version 887.2.

I still cannot load data past about 3/11/08 for the symbols mentioned above. IB has the data since I can load more data in a TWS chart.

What now?

John

[NinjaTrader_VinceB]

Have you tried importing the data? See if you can import it in .txt format. More info here:

http://www.ninjatrader-support.com/HelpGuideV6/Import.html

[dougfir]

Yes I did convert the data to ascii and import it. I was able to see more than the 10 or 11 months I was getting before.

However, something seems to be seriously wrong with NT's use of daily data if you have to get the data, convert to ascii, import, etc. for any symbol you want to see more than 11 months of data.

Hard to believe this is correct.

Have you tried to load more than a year's worth of daily data for TMW, AAPL, etc?

If you can, why can't I?

If you can't, why not?

John

[NinjaTrader_Bertrand]

Are you running into requesting too much data from IB at a time? In your log would then be a message stating 'NinjaTrader has reached an IB limitation for historical data requests'. You would then have to disconnect from IB and wait for approx. 10 minutes for your data status to clear with IB.

For example for AAPL I could get data back to 11/23/2007.

[dougfir]

I just closed and restarted NT with no charts open.

I connected to IB with no other requests in the past hour.

I opened a new chart for AAPL, requested 500 bars of daily data.

The first bar on the chart is 3/12/08 and the last is today.

I chose Reload Historical Data - same results.

I don't know what your data feed is but someone at NT must be able to test daily bars with IB to see where I'm going wrong.

As a test, is there a way to set my data feed for a particular instrument to Yahoo for daily data only?

John

[NinjaTrader_Bertrand]

Do you get any error in the Log tab of the Control Center?

My datafeed was IB and I was using a Live account to test this for you.

Please try to disconnect from IB and then go to Tools > Histrical Data > Delete and delete all old AAPL data.

Then reconnect and see if you can download more data now.

You can delete the data for AAPL again and then setup the account connection for Yahoo - http://www.ninjatrader.com/webnew/support_Yahoo.htm

For Yahoo on Yahoo I get backdata to 4 / 12 / 1996
For MSFT on Yahoo I get backdata to 3 / 13 / 1986

[dougfir]

I disconnected from IB, deleted AAPL, repaired the db.
I then connected to IB, went NEW - Chart - AAPL - 500 bars Day.
Got data only back to 3/13/08.

The log reads:
2/5/2009 3:04:21 PM|1|2|IB: Primary connection=Disconnected, Price feed=Disconnected
2/5/2009 3:05:49 PM|1|4|Database repaired
2/5/2009 3:05:57 PM|1|2|IB: Primary connection=Connecting, Price feed=Connecting
2/5/2009 3:05:58 PM|1|2|IB: Primary connection=Connected, Price feed=Connected

No error messages.

I then disconnected from IB, connected to Yahoo, built the same chart and all the data was there.
I know IB has the data since I can see it on another machine with a different IB connection (not thru NT).
Very strange. Sure like to know how/why you get all the data from IB into NT and I do not.

John

[NinjaTrader_Bertrand]

Hi John, are running a 32 or 64 bit OS?

You could try disconnecting from your IB connection and then go to Tools > Account Connections and delete your IB connection.

Then go to Tools > Options > Misc and Repair your DB.

After this recreate the IB connection fresh with the help of this connection guide - http://www.ninjatrader.com/webnew/su...ivebrokers.htm

Then delete old data by going to Tools > Historical Data.

Now try download a new set of data and see if you can get more backdata.

If this does not help, I would suggest trying a complete, fresh install of NinjaTrader and TWS 887.2

[dougfir]

I am using Windows Vista 32 bit.

I deleted the IB connection, repaired the DB, added a new IB connection.
Requested 500 daily bars of INTC.

Here's the relevant part of the trace file just after the request:

2009-02-06 13:21:42:538 (IB) Data.Bars.GetBarsNow: instrument='INTC' from='2006-12-21' to='2009-02-06' sessionBegin='00:00:00' sessionEnd='00:00:00' excludeWeekend=False period=Daily splitAdjusted=False dividendAdjusted=False bars=0
2009-02-06 13:21:42:553 (IB) IB.HistoricalDataRequest.Send: last=2/6/2009 instrument='INTC' from='2006-12-21' to='2009-02-06' sessionBegin='00:00:00' sessionEnd='00:00:00' excludeWeekend=False period=Daily splitAdjusted=False dividendAdjusted=False bars=0
2009-02-06 13:21:42:553 (IB) IB.HistoricalDataRequest.Send: request='INTC;STK;;0;;;SMART;ISLAND;USD;;;1 day;11 M;1;TRADES'
2009-02-06 13:21:43:220 (IB) IB.HistoricalDataRequest.Process: itemCount=228 instrument='INTC' from='2006-12-21' to='2009-02-06' sessionBegin='00:00:00' sessionEnd='00:00:00' excludeWeekend=False period=Daily splitAdjusted=False dividendAdjusted=False bars=0
2009-02-06 13:21:43:221 (IB) IB.HistoricalDataRequest.Send: last=3/13/2008 instrument='INTC' from='2006-12-21' to='2009-02-06' sessionBegin='00:00:00' sessionEnd='00:00:00' excludeWeekend=False period=Daily splitAdjusted=False dividendAdjusted=False bars=0
2009-02-06 13:21:43:221 (IB) IB.HistoricalDataRequest.Send: request='INTC;STK;;0;;;SMART;ISLAND;USD;;20080314 00:00:00;1 day;11 M;1;TRADES'

Note the 2 requests due to IB's limit of 12 months of daily data per request. This is fine but note that the first 228 bars are processed but the second request ending 3/13/2008 never gets processed. And in fact, the oldest bar on the chart is 3/14/08.

I'm positive IB has the data and is sending the data since I can request and receive that historical from the same TWS connection with my own code. The problem must lie in the way NT is receiving/processing (or not) the rest of the data past the first 228 bars.

As to the rest of your suggestions, I did a fresh install of IB. With a fresh install of NT, how do I restore my templates,workspaces, custom indicators, etc.?

I assume I could just copy back the saved template and workspace folders such as \NinjaTrader 6.5\templates and \NinjaTrader 6.5\workspaces. For the indicators I could copy back(restore) \NinjaTrader 6.5\bin\Custom\Indicator\ but would I then have to "Import" or compile each indicator?

John

[NinjaTrader_Bertrand]

Hi John, thanks for your patience with this issue. Unfortunately I cannot test this right now, as IB is down for normal maintenance. You could also try removing the symbol from your Dow30 list in the instrument manager and then adding it back and selecting only the default exchange for it. NT does seem to get the needed data, otherwise you would have seen a 'Process' statement. Did you try this also on the Yahoo connection?

Yes, you could copy the folder simply to the MyDocuments directory, for the indicators then open one up in the NinjaScript editor and hit F5 to compile them.