Accuracy of strats on Market Replay data

[lookOutBelow]

I was wondering. If I'm running a strategy on Market Replay data, is it as accurate as if I was running it live (in a demo account without real-world fill needs)? Or is there some kind of data not included that would make the results different?

Further, is there any sort of data dropped if I'm running a script against Market Replay data, but played back at 500x speed?


Thank you,
Rick

[NinjaTrader_Tim]

Hi lookOutBelow,

Running a strategy with the Market Replay would give you very close results to as if in live. However keep in mind the results could differ for a few reasons, including, but not limited to...
1. You are not sending actual orders to the exchange
2. You may or may not get filled in live
3. There may be latency involved

However, in general, the results should be more accurate than backtesting, in many cases.

Doing do on a higher speed should not change the results, you can test this on your end and let me know if you see a discrepancy.

[FormulaXTrader]

Hi lookOutBelow,

Running a strategy with the Market Replay would give you very close results to as if in live. However keep in mind the results could differ for a few reasons, including, but not limited to...
1. You are not sending actual orders to the exchange
2. You may or may not get filled in live
3. There may be latency involved

However, in general, the results should be more accurate than backtesting, in many cases.

Doing do on a higher speed should not change the results, you can test this on your end and let me know if you see a discrepancy.

When I run the same strategy at 30x and then again at 300x I get very different results. It appears that the speed plays a big factor in the results and that running it at anything more that 10x would give you incorrect results.

[NinjaTrader_Ben]

Hello,

I will have someone reply to you on Monday. Thank you for your patience.

[FormulaXTrader]

Hello,

I will have someone reply to you on Monday. Thank you for your patience.

Thanks. Here are the results from Replay when run at 30x and 300x so you can see the extent of the difference.

[NinjaTrader_Tim]

Hi FormulaXTrader,

Does this occur every time?
Is it close every time?

Can you provide all the necessary steps and scripts in order for me to reproduce this on my end?

[FormulaXTrader]

Hi FormulaXTrader,

Does this occur every time?
Is it close every time?

Can you provide all the necessary steps and scripts in order for me to reproduce this on my end?

Yes, unfortunately it does occur everytime and the results get even more different if I run it at 500x. For some reason I even get trade executions where according to my rules are not possible because price never hit my threshold trigger (according to the high or low of the bar).

I cannot send you the exact script I use because that is proprietary but I will try to modify it and insert standard oscillators in place of mine and send it to you. Not thru this forum though. Can you provide me your email address?

[NinjaTrader_Tim]

Hi FormulaXTrader,

You can send it to support at ninjatrader dot com, and reference this post in the email:

http://www.ninjatrader.com/support/forum/showthread.php?t=31678

[bluewave]

This should not be any surprise....the accuracy will begin to get skewed as you increase replay speed for the simple fact that some data points are missed resulting in trades being missed. Having done a fair amount of replay recording and study, the best way to keep it accurate id to play back no more than 15 to 30x....

bluewave