Strategy Analyser - Too Many Historical Requests

[tmfdouglas]

Hi,

When trying to run a backtest on the sp500 equities using tick data and IQ Feed as the data source it fails. Enabling error logging in IQ Feed the IQ Feed logs display "Too many historical requests". Is this a known bug?

Also is there any ETA for beta 12? Seems to be a number of bugs with the historical data downloader and strategy analyser which will cause the strategy analyse to freeze, historical data downloader to freeze or the whole of Ninjatrader to freeze.

Thanks

[NinjaTrader_JessicaP]

Hello tmfdouglas, and thank you for your question.

Interactive Brokers has a limitation in place for the amount of data that can be requested at one time. Loading small sets of instruments one at a time will prevent this from happening. The easiest ways to do this are to use multiple workspaces, and to use instrument lists with the market analyzer.

While there is no ETA for NinjaTrader 8, you can track our progress here,

http://ninjatrader.com/support/helpGuides/nt8/en-us/release_notes.htm

Please let us know if there are any other ways we can help.

[tmfdouglas]

Thanks Jessica although this problem relates to IQ Feed not Interactive Brokers. Attached is a screenshot of the error. I've also re-ran the back test today and the IQ Feed log is now full of "Send Failed to Client" error messages, screenshot attached.

Any thoughts?

Aprreciate there isn't an ETA for the none beta version but any news on the beta 12 release?




FROM CLIENT LookupRequest 38004 157 2016-06-14 20:20:43 HTT,EQT,20160103 190000,20160613 185959,,,,1,,

STATUS Information 18108 137 2016-06-14 20:20:43 LOOKUP SOCKET CLOSED -
STATUS Information 18552 0 2016-06-14 20:20:43 LOOKUP SOCKET ACCEPTED 158 -
FROM CLIENT LookupRequest 30892 158 2016-06-14 20:20:43 S,SET PROTOCOL,5.1

STATUS LookupError 30892 158 2016-06-14 20:20:43 S,CURRENT PROTOCOL,5.1


FROM CLIENT LookupRequest 30892 158 2016-06-14 20:20:43 HTT,EIX,20160103 190000,20160613 185959,,,,1,,

TO CLIENT Admin 18552 2 2016-06-14 20:20:43 S,STATS,66.112.156.224,60003,500,140,34,0,0,0,Jun 14 8:17PM,Jun 14 03:20PM,Connected,5.2.1.0,436664,11938.83,10.04,55 .79,1233883.03,4483.98,5765.81,

STATUS LookupError 34968 141 2016-06-14 20:20:44 Send Failed to Client: Bytes To Send 193836 : Error 10035 : First 256 bytes of data 2016-06-13 14:43:30.608,34.6950,300,2195469,34.6900,34.7000,1 387922,C,19,01,

2016-06-13 14:43:31.990,34.6900,100,2195569,34.6900,34.7000,1 388104,C,12,3D,

2016-06-13 14:43:32.001,34.6900,100,2195669,34.6900,34.7000,1 388106,C,24,01,

2016-06-13 14:43:32.00
STATUS LookupError 34968 141 2016-06-14 20:20:44 Send Failed to Client: Bytes To Send 387758 : Error 10035 : First 256 bytes of data 2016-06-13 14:43:30.608,34.6950,300,2195469,34.6900,34.7000,1 387922,C,19,01,

2016-06-13 14:43:31.990,34.6900,100,2195569,34.6900,34.7000,1 388104,C,12,3D,

2016-06-13 14:43:32.001,34.6900,100,2195669,34.6900,34.7000,1 388106,C,24,01,

2016-06-13 14:43:32.00

[NinjaTrader_JessicaP]

Hello tmfdouglas,

While I don't know IQFeed's log format too well, the message output is suggesting that there is a problem connecting Ninja to IQFeed on your local machine. So I may assist further, can you provide the following information about your system, so that I can attempt to reproduce this on my end?

• Your operating system
• Whether you are using a virtual machine (VMware, Parallels, etc) or software like BootCamp
• Your IQFeed version (unless you are using the recommended v5.1.1.3 for Ninja 8)

Please also send your (My) Documents\NinjaTrader 8\log and (My) Documents\NinjaTrader 8\trace files to platformsupport[at]ninjatrader[dot]com, referencing the following unique ID in the body of your e-mail : 1529789



Finally, also in the operating system that Ninja is running in, I would like you to perform the following steps and take a screen shot.

• Win key + e
• Right click (My) Computer (called This PC in Windows 10)
• Select Properties

To send a screenshot with Windows 7 or newer I would recommend using Window's Snipping Tool.
Click here for instructions

Alternatively to send a screenshot press Alt + PRINT SCREEN to take a screenshot of the selected window. Then go to Start--> Accessories--> Paint, and press CTRL + V to paste the image. Lastly, save as a jpeg file and send the file as an attachment.
Click here for detailed instruction

Thank you, with this information I believe it will be much easier to assist further.

[tmfdouglas]

Hi,

Thought the IQ Feed version was the problem and downloaded the version from your website but no luck, same problem happened again this evening.

Have emailed all the log and trace files and also the IQ Feed log file.

In answer to your questions, Windows 10 Pro, not using VM's.

Unfortunately feel I may need to give up on NinjaTrader soon.....

Thanks
Tim

[NinjaTrader_JessicaP]

Hello tmfdouglas,

I would like to ask you to perform the following procedure to see if it resolves your query.

• Save your current workspace
• Close NinjaTrader
• Delete your (My) Documents\NinjaTrader 7\workspaces\_Workspaces.xml
  • Deleting this file will force Ninja to load a blank "Untitled" workspace
• Connect to IQFeed
• Load only the ES 09-16 contract into your strategy backtest

If this does, this means that IQFeed has a data limit similar to TWS limit. I have been able to find a hard limit of 18 concurrent historical data requests.


What this will mean to you, is that the advice I gave for TWS should also apply to your situation,

Loading small sets of instruments one at a time will prevent this from happening. The easiest ways to do this are to use multiple workspaces, and to use instrument lists with the market analyzer.

Please let us know if there are any other ways we can help.

[tmfdouglas]

Hi Jessica,

I don't have a futures data subscription but have tested using AAPL and SPY individually and works fine.

I believe there is a historical request limit, a number of posts on the IQ Feed forum state this, I assumed that Ninjatrader would abide by this limit though as IQ Feed is a supported data provider.

Are there any plans to implement this?

Thanks

[NinjaTrader_JessicaP]

Hello tmfdouglas,


I have submitted a feature request to the product management team for the following feature :


This user would like the IQFeed connection throttled on the Ninja side so it stays within their historical data request limits.


I will follow up with more information as soon as it's available. If the feature requests already exists, a vote will be added to it.


Please let us know if there are any other ways we can help.

[NinjaTrader_JessicaP]

Hello tmfdouglas,

In order to help us reproduce on our end, I would like to ask, are you able to cause the same message to show up in your Ninja and IQFeed logs using the SampleMACrossover strategy to test with? If you are, please send screen shots of your strategy analyzer settings, along with your Ninja and IQFeed lots, to platformsupport[at]ninjatrader[dot]com . Please include this unique ID in the body of your e-mail :

1529789

To send a screenshot with Windows 7 or newer I would recommend using Window's Snipping Tool.
Click here for instructions

Alternatively to send a screenshot press Alt + PRINT SCREEN to take a screenshot of the selected window. Then go to Start--> Accessories--> Paint, and press CTRL + V to paste the image. Lastly, save as a jpeg file and send the file as an attachment.
Click here for detailed instruction

Thank you very much, this information will make it much easier to reproduce this on our end, and reduce the amount of time we spend on a solution.

[tmfdouglas]

Hi,

Sent as requested.

[NinjaTrader_JessicaP]

Thank you very much tmfdouglas, we have received this information.

Just to confirm, you did produce these logs using the SampleMACrossover strategy that is included with Ninja? We will need this to be used on both ends so that we may perform valid tests on our end.

[tdouglas]

Yes that's right. Should be a screenshot in the mail too.

[NinjaTrader_JessicaP]

Hello tdouglas,

I attempted the following procedure in the latest build of NT8 (which will become Beta 12), and it did not produce what you observed :

• I connected to IQFeed v 5.1.0.5 with "Use Back Adjusted Data" unchecked in NinjaTrader
• New -> Strategy Analyzer
  • Backtest type : Standard
  • Strategy : SampleMACrossover
  • Fast : 10
  • Slow : 25
  • Instrument : SP 500 -> Select All
  • Price based on : Last
  • Type : Tick
  • Value : 1
  • Start Date : 01JAN16
  • End date : 20JUL16
  • Trading hours : <Use instrument settings>
  • Break at EOD : checked
  • Include commission : unchecked
  • Maximum bars look back : 256
  • Bars required to trade : 20
  • Order fill resolution touch : Standard (Fastest)
  • Fill limit orders on : unchecked
  • Slippage : 0
  • Entries per direction : 1
  • Entry handling : All entries
  • Exit on session close : checked
  • Set order quantity : strategy
  • Time in force : GTC

I recommend attempting to downgrade IQFeed to the version I used. Should you continue to experience the same thing, please get in contact with IQFeed. Should this behavior continue into the next version of NinjaTrader please let us know.

Should IQFeed need any information from us, please ask that they use this unique ID when contacting us : 1529789

If you would like us to examine your system to see if we can assist further from there, or if there are any other ways we can help, please let us know.

[tmfdouglas]

Thanks Jessica, as said earlier in the post though was tested with both versions of IQ Feed. Will try again once beta 12 is released.

Thanks
Tim

[tmfdouglas]

Also - is there a rough ETA for beta 12?

Thanks
Tim