Can I backtest/optimize using playback data?

[edstaffin]

Is there a way to do this?
I can't get the strategy analyzer to match up with the playback trade (having dropped a strat on a chart and check trade performance). If I can do an optimization using playback data that would probably be more accurate anyhow.
Thanks ... Ed

[NinjaTrader_JessicaP]

Hello edstaffin, and thank you for your query.

So that I may have a clearer picture what is happening, can you make this happen with the SampleMACrossover strategy, and then send me a screenshot of the settings you used? Could you also let me know a time, instrument, and expiry combination I should expect to see a trade show up at?

To send a screenshot with Windows 7 or newer I would recommend using Window's Snipping Tool.
Click here for instructions
Alternatively to send a screenshot press Alt + PRINT SCREEN to take a screenshot of the selected window. Then go to Start--> Accessories--> Paint, and press CTRL + V to paste the image. Lastly, save as a jpeg file and send the file as an attachment.
Click here for detailed instruction

[edstaffin]

It'll take me a bit but I will get back to you as soon as I can.
In the meantime, is there a way to run a backtest/optimization on playback data?
Thanks ... Ed

[koganam]

It'll take me a bit but I will get back to you as soon as I can.
In the meantime, is there a way to run a backtest/optimization on playback data?
Thanks ... Ed

I will presume then that your strategy does not use Calculate.OnBarClose?

If you want the strategy to act intrabar, on NT8, you can use TickReplay or a multi-timeframe method. Do neither of those work?

What instrument? If you are using an instrument like /CL that has massive slippage, it will be close to impossible to match realtime performance in the Strategy Analyzer. In the bad old days when I really thought that I knew what I was doing, but was really a rank ignoramus, I once, in realtime suffered a 12 tick slippage on my Stop Loss when trading /CL. I am barely a tad wiser now.

Recheck your strategy using /ES. Is that a closer match between realtime and the Strategy Analyzer or chart?

[edstaffin]

12 tick slippage? Yikers! I thought I was being conservative testing for 2 tick slip.
I am using Calculate.OnBarClose.
I am using CL. In my playback test for just 16 days I had 25 trades. I run the same thing in strategy analyzer doing a backtest, I get only 11 trades. That's so far off that the analyzer loses credibility. It almost seems like the 2 are working with different data sets. I am going to re-download the timeframe for both analyzer and the playback and try again. Stay tuned.
As to your comment on "the bad old days when you thought you knew ...", I am reminded of an old joke. "Those of you who think you know everything are bothering those of us who do." :-)

[NinjaTrader_JessicaP]

IIf you would like your playback and analyzer to use the same data, I would like to suggest recording incoming data as historical data, and then playing over your historical data in the playback connection instead of using market replay data.

You can set up Ninja to record incoming ticks as historical data with Tools -> Options -> Market Data -> Record live data as historical -> OK -> OK

The Playback Connection controller will have two radio buttons, one for market replay, one for historical data.

[koganam]

12 tick slippage? Yikers! I thought I was being conservative testing for 2 tick slip.
I am using Calculate.OnBarClose.
I am using CL. In my playback test for just 16 days I had 25 trades. I run the same thing in strategy analyzer doing a backtest, I get only 11 trades. That's so far off that the analyzer loses credibility. It almost seems like the 2 are working with different data sets. I am going to re-download the timeframe for both analyzer and the playback and try again. Stay tuned.
As to your comment on "the bad old days when you thought you knew ...", I am reminded of an old joke. "Those of you who think you know everything are bothering those of us who do." :-)

I have seen that one before. Invariably, it was because I was lacking some historical data. Refreshing the data on the chart usually did not help: I had to:

1. Use the Historical Data Downloader to get the data.
2. Delete the cache.
3. Reload the chart data.

On occasion, I have actually had to change the data series back and forth, or create a completely new chart. At other times, the new data got loaded as soon as it was downloaded.

[edstaffin]

Fun, fun, fun! I'll do some playing and get back to you.
Thanks ... Ed

[edstaffin]

IIf you would like your playback and analyzer to use the same data, I would like to suggest recording incoming data as historical data, and then playing over your historical data in the playback connection instead of using market replay data.

You can set up Ninja to record incoming ticks as historical data with Tools -> Options -> Market Data -> Record live data as historical -> OK -> OK

The Playback Connection controller will have two radio buttons, one for market replay, one for historical data.

I guess my question here is, "what's the price of poker?" In other words does doing as you suggest come at a cost either performance or anything else?
Thanks ... Ed

[NinjaTrader_JessicaP]

Thank you for your question, edstaffin. There are in fact some costs to be aware of.

In terms of recording incoming tick data as historical data, recording live data uses more PC resources and is intended for connections which do not provide historical data. I am recommending this only so that you are aware of the feature, since you mentioned wanting to guarantee performance between different parts of NinjaTrader 8. Enabling this option while also using a historical data provider may result in data gaps.

In terms of using historical data instead of market replay data, this is generally less accurate than using Market Replay. Specifically, there is no level 2 market depth data in this mode.

With this in mind, while both these tools can be very useful for testing strategies, it is recommended once you have satisfactorily tested performance between playback, analysis, and live data, that you revert to using historical data from your data feed provider, and that you use market replay data to test your strategy, especially if it relies on level 2 data.