Market Replay Playback needs a queue or date range facility

[kimbee]

The new Market Replay / Playback feature needs to have the ability to download more than 1 instrument at a time / day. This is the perfect time to make this feature far more efficient, effective & productive.

Scenario:
When one is downloading 50 instruments for a month (about 20 working days), that's a 1000 downloads, and when some of them can take up to 3 minutes to download (especially now that it's only level 2 data), that can add up to far too many hours downloading data, when the computers should be able to do this more efficiently.

Does anyone else agree with this concept (that downloading one instrument at a time, and one day at a time) needs improving ?

I can think of 3 ways of making the download of Market Replay / Playback data more efficient:

1. The ability to download many instruments at one time, for a particular day.
e.g. click on the Heading "Futures", and all the instruments in your Futures list are automatically included in this date

&/or

2. Give the ability for users to enter a date "range". Even if its limited to 7 days at a time. Anything is an improvement than just one day at a time. A 31 day date range would be sensible, considering some instruments change contracts every month.

&/or

3. Add to a download Queue facility (my personal favourite). Using a queue method, one can select an Instrument (e.g. ES - which can take minutes to download), and change the date, click download, change the date, click download (repeat....) Each date / instrument download request is entered into a queue. Then the software can manage the queue (by throttling speeds, postponing to quieter times etc)
There's even a lovely empty space at the bottom of the Historical Data window / "Get Market Replay data" section - just waiting for a queue management system.

Can Ninja development staff PLEASE do something about improving the slow method that users have available to them to download Playback data.

I would love to hear other peoples comments.

Regards

[kimbee]

I found another user that has similar thoughts on speeding up Playback downloads

I found this post that supports my case...

"Date Range for Market Replay?"

http://ninjatrader.com/support/forum/showthread.php?t=30027

I consider this a very important (not quite show stopper) issue.

[NinjaTrader_Matthew]

Thanks for sharing your ideas. As you have noted, this has been requested before. At this time, we have no plans to change the methods for downloading data and the current implementation is intentional.

Access to our historical data servers (included market replay) is a free value added service that is available to all users. Hosting this data and providing access to our users takes significant server resources. The market replay files are much larger than the typical tick data. We do not want to risk the chance that users downloading gigs of market replay data would potentially limit or even take down access to our servers, preventing all users from downloading data that they need access to for live trading needs.

We have a number of partners that have developed tools and host market replay on their service which will allow you to download a wider range of data if you need to and other enhancements you mentioned such as downloading a list or queuing requests.

You may have noticed the NinjaTrader 8 feature to play back "Historical" data which will give you allow you to playback historical tick data over a range of data you have in your database. While this mode is generally less accurate, it should help users who are playing back a range of data to get an idea of how the market played out over that period.

If perhaps you needed market replay data because you wanted to backtest a strategy as if it was running live over a range of time, we now have the "Tick Replay" mode which also runs on historical tick data, including bid/ask events to give you a true idea of how a strategy may have operated during that period.

Access to our free replay servers will remain limited to one instrument and date at a time. While this could change in the future, I just would like to set expectations that this enhancement is not on our short term radar.

[kimbee]

Now that I have discovered the brilliance of downloading data from my data broker, this issue is now obsolete. This other option allows me to download an entire month, all instruments all at once, with the click of one button. Brilliant.