Running an Optimization in Market Replay mode

[dave416]

I have a strategy that requires an optimization 15 minutes before the market open. I want to test this in Market Replay mode. Will historical data beyond the 'To' day set in Market Replay be used by the optimizer?

Example:

Today is Friday 12/10/2012...

Time/Date on the Market Replay is currently set to last Monday 12/03/2012 at 9:15:00 AM. I pause the replay, then run an optimization on my strategy. Will the optimizer use historical data stored on my drive beyond 12/03:2012 9:15:00 AM to come up with the optimized parameters?

I hope this is clear.. and sorry if this has been asked before but I couldn't find any posts or help documentation that explicitly describes the expected behaviour.

Thanks -Dave

[NinjaTrader_PatrickH]

Hello Dave,

Thank you for your post.

The Market Replay 'From' and 'To' fields have no effect on the 'From' and 'To' fields in the Time Frame section of the Optimizer parameters.

No matter what dates you are running in the Market Replay Control window you will still be able to select dates outside of this range in your Strategy Analyzers.
The reason this is, is because Market Replay runs on Market Replay Data not Historical Data. Where as the Strategy Analyzer runs on Historical Data.

Please let me know if you have any questions.

[dave416]

Great thanks Patrick - What if the 'To' date on the Optimizer is the same day as the date set on the Market Replay?

example:

Market Replay time/date = 12/03/2012, 9:15 AM
Optimize Dates = 11/29/2012 to 12/03/2012

I tried this, pausing at 9:07 AM, and the optimization results do not include any trades beyond that time on 12/3/2012. I also tried setting the To date on the Optimizer to 12/04 (the next day) and the result still only go as far as the Market Replay time and date.

So even though the Optimizer is indicating that it is optimizing up to 9:07 AM on 12/3 is it actually excluding the rest of that day's historical data in the sample? Please see the screen shot below demonstrating this using the SampleMACrossover strategy.

Thanks -Dave

[NinjaTrader_PatrickH]

Hello Dave,

Thank you for that additional information.

I have tested this item on my end as well and found the exact same results, so contrary to my original response, the Market Replay date does effect the Strategy Analyzer.

This is expected behavior as the Market Replay connection is used for Market Replay, and not the Strategy Analyzer. The Strategy Analyzer is used with historical data which is not Market Replay data.

Please let me know if I may be of further assistance.

[dave416]

clarification

Patrick it is very very important that I fully understand the behaviour in detail as I will be investing many hours of testing based on the answer, so forgive me if I ask again for clarification.

1. You wrote 'Market Analyzer' in your last post, but I was referring to the Strategy Analyzer in Optimization mode, which is different. Could you please clarify if you meant Strategy Analyzer?

2. Thank you for reproducing my scenario above, however this still leaves my original question not completely answered. All we are confirming here is that

a) Strategy Analyzer will only show optimized trades up to the day and minute that the Market Replay is on.

This is fine, HOWEVER

b) I'm worried that Strategy Analyzer might use ALL the historical data for that day (12/3) to come up with optimized parameters up to 9:07 AM that day... and maybe it only displays (filters out) the trades in the results window up to that day/time.

Point b) is important, and I need to verify that the Optimizer will only use historical data up to that point in time set in Market Replay. I already got skewed results using the Walk Forward tester using session templates that span overnight (a known issue in NT), and so it appears that the background calculations are is not always as it might display in Ninja Trader, thus this inquiry.

Thanks for your time in looking into this.

-Dave

[NinjaTrader_PatrickH]

Hello Dave,

Thank you for your update on this matter.

You are correct, and I have edited the previous post to reflect the Strategy Analyzer. Thank you for pointing that out.

In any case here we are seeing that the Strategy Analyzer is producing results that do not reflect the range it is set to for the 'From' and 'To' range. This is due to the fact that the Market Replay Connection is the active Connection. The Strategy Analyzer is working as expected and it is best used when not connected to the Market Replay connection.

The entire date will not be used for that last trade or trades on that date. The trade is closed at that time indicating that it is in fact a completed trade and all calculations are complete for that trade. The entire day is not reflected in that one trade.

To see the complete results of the test make sure to run the Strategy Analyzer when you are no longer connected to the Market Replay Connection.

Please let me know if I may be of further assistance.

[dave416]

Thank you Patrick this is the good news I was looking for and will help greatly in testing my strat!

cheers -Dave

------
Side note for anyone reading this an wondering why I might need to optimize in Market Replay mode:

The WTI Crude Oil (CL) contract is a wild beast and its characteristics can shift greatly from week to week or even day to day. This, compounded with the fact that different contract months respond differently to the same strategy parameters, makes it difficult to create a highly efficient strategy. If the strat was based purely on price action (e.g. pivots, volumes, market profile, etc.) then this might not be so much of an issue, however if the strat takes into consideration historical price trends (like mine does) then this becomes more of an issue and it must be optimized more often, sometimes right up to the minute of when the strategy starts. The Strategy Analyzer only allows you to pick day-to-day which is not ideal for optimizing to Crude Oil, so I must use Market Replay to force the Strat Analyzer to only include up to a certain hour/minute of day of historical data. (FYI, I use non-merged, non-backadjusted data in my strats for CL)

-Dave

[maketoprank]

Patrick it is very very important that I fully understand the behaviour in detail as I will be investing many hours of testing based on the answer, so forgive me if I ask again for clarification.

1. You wrote 'Market Analyzer' in your last post, but I was referring to the Strategy Analyzer in Optimization mode, which is different. Could you please clarify if you meant Strategy Analyzer?

2. Thank you for reproducing my scenario above, however this still leaves my original question not completely answered. All we are confirming here is that

a) Strategy Analyzer will only show optimized trades up to the day and minute that the Market Replay is on.

This is fine, HOWEVER

b) I'm worried that Strategy Analyzer might use ALL the historical data for that day (12/3) to come up with optimized parameters up to 9:07 AM that day... and maybe it only displays (filters out) the trades in the results window up to that day/time.

Point b) is important, and I need to verify that the Optimizer will only use historical data up to that point in time set in Market Replay. I already got skewed results using the Walk Forward tester using session templates that span overnight (a known issue in NT), and so it appears that the background calculations are is not always as it might display in Ninja Trader, thus this inquiry.

Thanks for your time in looking into this.

-Dave

I want to test this in Market Replay mode.Great thanks Patrick.
Best Optimization Company

[NinjaTrader_Lance]

Hello,

I want to test this in Market Replay mode.Great thanks Patrick.

Could you please clarify what aspect of market replay you are having trouble with?

For a webinar recording on using Market Replay: http://www.youtube.com/watch?v=yVteK...D7105&index=13

For our help guide and videos on downloading and using Market Replay: http://www.ninjatrader.com/support/h...ket_replay.htm