I have been testing two strategies that provide really nice daily results using the strategy analyzer. They are time-limited strategies (meaning they only trade between 8:35 AM EDT and 3:50 PM EDT. When collecting replay data I always start the night before the actual trading day, after midnight of the previous day. I live in the Pacific Time zone but keep my trading machine on EDT. Here are the results I've gotten. I'm concerned about the wide delta between what the strategy analyzer provides as strategy performance versus what happens in market replay. I am going to also test these two strategies next week by running them 24/7 and see what results I get for a real-time comparison, a market-replay comparison, and the strategy analyzer results.
Any comments or advice you could provide regarding how to get closer results between the three approaches would be greatly appreciated. The strategy runs on 1 minute data, trades only 1 contract for each strategy, and uses market orders to enter and stop or limit orders to exit. The testing is based on the current ES contract ES-09. Here are the results:
FOR RESULTS SEE ATTACHED TEXT FILE
Actual versus test results of from -51% to minus almost 100% makes the
strategy analyzer fun to use, but not at all helpful in determining how successful a strategy is likely to be or even how tweaks to a strategy might
make it perform a little better or a little worse. Perhaps my expectations of what the strategy analyzer should provide the systems developer in terms of useful functionality is way off, but right now, it looks like the strategy analyzer is nearly a complete waste of time. It does appear that for any given strategy, if it performs better in the strategy analyzer, than another strategy, then in real-time (as proxied by the replay data performance) the better strategy is likely to do better than the other strategy but not nearly close to what one would get trading that strategy live.
Any advice or insight you could provide about how to get these two tests closer in performance or perhaps how to interpret the results would be greatly appreciated.
Thanks
DaveN
PS, I added the tabular results as an external text file since I couldn't get them to format properly in this post.
Any comments or advice you could provide regarding how to get closer results between the three approaches would be greatly appreciated. The strategy runs on 1 minute data, trades only 1 contract for each strategy, and uses market orders to enter and stop or limit orders to exit. The testing is based on the current ES contract ES-09. Here are the results:
FOR RESULTS SEE ATTACHED TEXT FILE
Actual versus test results of from -51% to minus almost 100% makes the
strategy analyzer fun to use, but not at all helpful in determining how successful a strategy is likely to be or even how tweaks to a strategy might
make it perform a little better or a little worse. Perhaps my expectations of what the strategy analyzer should provide the systems developer in terms of useful functionality is way off, but right now, it looks like the strategy analyzer is nearly a complete waste of time. It does appear that for any given strategy, if it performs better in the strategy analyzer, than another strategy, then in real-time (as proxied by the replay data performance) the better strategy is likely to do better than the other strategy but not nearly close to what one would get trading that strategy live.
Any advice or insight you could provide about how to get these two tests closer in performance or perhaps how to interpret the results would be greatly appreciated.
Thanks
DaveN
PS, I added the tabular results as an external text file since I couldn't get them to format properly in this post.
Comment