I would like to be able to run a "diff" between two optimization runs, to see how the trades differ as the input parameters to the strategy change. For example, if one parameter set is superior during optimization, but only differs from a less superior set by one trade, I wouldn't consider it statistically significant. Right now, I export the trade grids to files, pre-process them a bit, and run a text diff on them.
Another suggestion would be the ability to analyze parameter stability. This is stuff I've only seen in high end packages, but the basic feature is to be able to see (visually or numerically), within the optimization range, how performance changes as a parameter changes. A parameter that basically results in a noisy ("randomly" up/down) looking performance curve as the parameter is incremented is not stable and probably not even worth having as a strategy parameter. On the other hand, a parameter that monotonically increases or decreases performance is strongly correlated and very useful as a parameter. Some higher end tools will also allow non-linear parameter stability analysis, where you can check the fit (R^2) of a n-order polynomial to the performance curve to see if the performance forms a "mountain", "valley", "two mountains", etc. as the parameter changes. To do this in a statistically significant manner, you sometimes need to group multiple parameter values, so get enough trades in each group. For example, you would have the moving average period be 20-120 in steps of 1 during optimization, but you would make each parameter analysis group be a set of 5 consecutive values (20-25, 26-30, etc.), so you have at least 200 trades in each group and a smoother performance curve to analyze, for example.
These are probably more advanced features not everyone would use, but I'd certainly find it handy, rather than exporting to Excel and doing my own semi-manual analysis. It may even be something a third party could implement, if they had proper hooks to get to the data without users running manual exports, etc. Right now, I don't think those hooks exist in NT 6.5.
Another suggestion would be the ability to analyze parameter stability. This is stuff I've only seen in high end packages, but the basic feature is to be able to see (visually or numerically), within the optimization range, how performance changes as a parameter changes. A parameter that basically results in a noisy ("randomly" up/down) looking performance curve as the parameter is incremented is not stable and probably not even worth having as a strategy parameter. On the other hand, a parameter that monotonically increases or decreases performance is strongly correlated and very useful as a parameter. Some higher end tools will also allow non-linear parameter stability analysis, where you can check the fit (R^2) of a n-order polynomial to the performance curve to see if the performance forms a "mountain", "valley", "two mountains", etc. as the parameter changes. To do this in a statistically significant manner, you sometimes need to group multiple parameter values, so get enough trades in each group. For example, you would have the moving average period be 20-120 in steps of 1 during optimization, but you would make each parameter analysis group be a set of 5 consecutive values (20-25, 26-30, etc.), so you have at least 200 trades in each group and a smoother performance curve to analyze, for example.
These are probably more advanced features not everyone would use, but I'd certainly find it handy, rather than exporting to Excel and doing my own semi-manual analysis. It may even be something a third party could implement, if they had proper hooks to get to the data without users running manual exports, etc. Right now, I don't think those hooks exist in NT 6.5.
Comment