Historical Data in IB

[JKrider]

When running a backtest can you disable the loading of historical data from IB. For example, I have two years of minute data in the Historical Data Manager for RIMM. I start a backtest and the strategy analyzer starts downloading historical data from IB even though I already have that data. IB then hits a data limit and the analyzer quits. If I disconnect from IB everything works great. The problem is that during the morning I do not want to disconnect as I monitor my positions, but I do want to work with the strategy analyzer.

[NinjaTrader_Austin]

JKrider, you can to set the To Date to sometime before the day you are running the backtest. For instance, if today is July 21, set the end of the backtest to the 19th so it doesn't have to load the data from today.

[JKrider]

Just tried. Set the end date to 7/19/2010. I am trying for a 1 year back test and have start date as 7/1/2009. It still immediately starts downloading data again from IB and when it hits around September I get the message that there is an IB limitation for historical data requests.

[NinjaTrader_Austin]

JKrider, please try downloading the data from the Historical Data Manager before running the backtest. I just opened up the HDM, downloaded daily and minute data for DELL from 1/1/2010 to today (7/21/2010), and then I could run a backtest on both minute and daily data from 1/1/2010 to 7/19/2010 without the Strategy Analyzer re-downloading the data. Please let me know if this doesn't work for you.

[JKrider]

I did the same. HDM, download DELL 1/1/10 to 7/21/10. Watched the bottom right of Control Center, Showed "Loading data DELL at X/XX/XXXX. Ran SA and worked fast. My RIMM HDM data was from 7/19/09 to today. I also have 2008 up to 6/30/09. If I run SA from 8/09 to present, no problem. If I run from 7/09 to present it tries to download 7/09 but starts from present going backwards and runs into IB's data limit and exits. In the process I seem to have lost my 7/19 to 8/1 data. So apparently the issue resolves around asking for data prior to beginning of the stored HDM data, then tries to download everything again. I guess that would be expected behavior.

[NinjaTrader_Austin]

Hi there, your assumption is correct. If you request data from before any of the data stored on your computer, it will reload it all. This is expected behavior.

[NingNing]

may i know where the historical data manager located in Ninja?

[NinjaTrader_VinceB]

Hi,

Under the Tools menu.

See here for more info:

http://www.ninjatrader.com/support/helpGuides/nt7/historical_data_manager.htm

[NingNing]

same applies to 6.5 V? thx again

[NinjaTrader_JoshP]

No, the Historical Data Manager is a new feature in NT7. There is no equivalent in NT6.5.

[J_o_s]

Would it be possible to program an indicator through a command to disable downloading data from IB (or another broker), and instead use the data which is already in the NinjaTrader database?

My problem is that I have an indicator, which compares the current Instrument with other instruments, and, after some calculations, plots this in a new panel. However, when I'm connected to IB, it loads all the other instruments, everytime I change the chart to another instrument.

Would it be possible to tell this indicator to just use the data in the database? That would be a lot quicker and also prevents the error popups ('downloaded to much historical data, for the next 5 minutes no data') and the crashes which NinjaTrader experiences.

So far, I've come up with this code:

[COLOR=black][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2]if[/SIZE][/FONT][/SIZE][/FONT][/SIZE][/FONT][B][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2] ([/SIZE][/FONT][/SIZE][/FONT][/SIZE][/FONT][/B][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2]CurrentBar==[B] ([/B]Count-1[B]) [/B]&&!Historical[/SIZE][/FONT][B][COLOR=black])[/COLOR][/B]
[B][COLOR=black]{[/COLOR][/B]
[COLOR=black]Print[B]([/B][/COLOR][/SIZE][/FONT][/SIZE][/FONT][/COLOR][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2][FONT=Consolas][SIZE=2][COLOR=black]"This is realtime data"[/COLOR][/SIZE][/FONT][/SIZE][/FONT][/SIZE][/FONT][B][FONT=Consolas][SIZE=2][COLOR=#f3f3f3][FONT=Consolas][SIZE=2][COLOR=#f3f3f3][FONT=Consolas][SIZE=2][COLOR=#f3f3f3][COLOR=black]);[/COLOR][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][/B][FONT=Consolas][SIZE=2][COLOR=#f3f3f3][FONT=Consolas][SIZE=2][COLOR=#f3f3f3][COLOR=#f3f3f3]
[B][SIZE=2][FONT=Consolas][COLOR=black]}[/COLOR][/FONT][/SIZE][/B]
[/COLOR][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT]

This makes it possible to stop the indicator if the data is realtime, however, it does not prevent that NinjaTrader loads all the symbols in the background.

Anyone got an idea as to how to solve this problem?

Regards,

[NinjaTrader_Bertrand]

Unfortunately this can't be controlled programmatically from a NinjaScript indicator, when connected live NinjaTrader would always try to refresh the current day - http://www.ninjatrader-support.com/H...ricalData.html

[J_o_s]

Unfortunately this can't be controlled programmatically from a NinjaScript indicator, when connected live NinjaTrader would always try to refresh the current day - http://www.ninjatrader-support.com/H...ricalData.html

Hm, okay. If I understand it correctly, that means that IB as a data provider isn't suitable for stock trading? Or, put in a different way, IB and NT together aren't suited for stock trading?

I'm asking because I have a Market Analyzer window with 50 stocks, and the following columns: LastPrice, LastClose, DailyVolume, DailyLow, DailyHigh, NetChange and RSI(2,1) (with 7 60 minute bars as lookback period). I intend to use this Market Analyzer screen as an "filter" to spot interesting stocks. However, when connecting to IB it starts to load every instrument, and even though it only loads 1 or 2 days of data, I never get beyond 1/3 of the Instrument list in the Market Analyzer.

Because I can't load instruments individually in the Market Analyzer, I never get the complete list loaded.

Is there any workaround for this? It would be great to tell NinjaTrader to load one instrument, then wait for example 30 seconds, and then load the next. Because I run this Market Analyzer after hours this would work.

Because with one indicator in the Market Analyzer (RSI) I already run into the "loading difficulties", with more indicators this Market Analyzer will not work as an "screening tool".

Does anyone have an suggestion as to how to make this Market Analyzer useful with IB data? I guess I'm not the only person running into this problem, so I would love to hear some tips or hints from other users.

Regards,

[NinjaTrader_Bertrand]

J_o_s, have you tried starting out with a 'reset' data state as far as your IB connection goes (meaning no prior requests creating any taxing on your data limit levels) ?

Also: please ensure to remove all not needed tickers, including the pages you've got setup in the TWS, those would count, too towards your limit.