i have had very strange results when using the optimization engines both on nt 7 and nt 8 before. consequently, i decided to do some very general testing on the performance of the optimization engines themselves and i have some preliminary results now.
i have some suggestions to make and questions to ask of the people at nt, and would also appreciate if other users shared their experiences and advice using nt's optimization engine.
so far i have run all my tests on nymex's cl contract with end date 31-XII-2014 optimizing for largest long + short net profit. i'll explain my tests in more detail further below.
- the first situation i would like to point out is that running optimization processes in ninjatrader is unnecessarily cumbersome. one can only run one single optimization at a time and there's no way to create a set of several optimizations so that they all run one after another. in nt one has to wait until the optimization in process is finished before being able to run another one and this is inefficient, tedious and excessively time consuming. in other platforms i have used, one can run one optimization per data chart and have as many charts as one wants so that one can prepare an unlimited number of optimization processes and they will run unattended one after another for hours or days.
- the second improvement i would suggest has to do with being able to copy the optimal settings for the parameters of a strategy that has been optimized to another chart with a different symbol or a different time period. in other platforms i have used one can optimize a strategy up to a certain date on a chart leaving data out of sample and then just change the end date on the chart, and the data that had been left out of sample will be added to the chart. in this way it is easy to evaluate how strategies would have done on real live data. one can copy and paste these charts that include optimized strategies and subsequently change their symbol, start and end date, and all other properties like values for slippage, commissions, etc. this allows for great flexibility and ease when evaluating and optimizing strategies but nt completely lacks these features and everything must be done typing all the optimized settings by hand.
the following is a composite image i have created from the reports of several different strategies i evaluated. i started by evaluating the sample sma cross over strategy that is included in the nt platform for largest long + short net profit. i ran optimizations for this strategy disabling the exit on session close feature on daily, 1440 minute and 360 minute bars and seemingly all these variations worked correctly. i then evaluated the same strategy on the same bars but enabling the exit on session close feature and seemingly all worked fine as well. i then optimized the same strategy on the same bars just adding a stop loss order for every position in the strategy and results were also positive (these cases are the ones with green charts on the composite image below).
in all of these cases, it seems like the strategies functioned as expected and the optimization engine also did its job. the size of the profits is not so relevant and i don't think these strategies could be traded with real money, but it seems like the strategies and the optimization engine were working acceptably.
the first unexpected result came when i tried to optimize the sample sma cross over strategy on the same bars incorporating a trailing stop order (calculated on ticks) for every position. in this case this variation generated losses in all kind of bars i tried. the blank reports in the composite image are of these cases with trailing stops, as the platform reported that the best results were when both sma periods were identical and no trades were generated, therefore the optimal result was 0.
and then the biggest issue came when trying to optimize any kind of strategy on range bars. every single combination of range size (30, 40 or 50 ticks), strategy, exit on session close, stop orders or no stop orders i tried generated zero trades. this must be a malfunction of the platform as it was not like the case with trailing stops where the optimal parameters generated zero trades and the rest were all losing settings, in this case there are zero results. the following report illustrates this situation.
these are my results so far. i'm fairly experienced at developing automated strategies so i think these unexpected results with trailing stop orders and range bars must be malfunctions with the nt platform.
the following two screengrabs illustrate how the same sample sma cross over does work and generate trades on a cl range 50 chart and how the sma cross over with a trailing stop also works on a cl range 50 chart and should generate profits if the optimization engine worked properly.
i will now do similar tests on other bar types like renko, tick and volume and will report back as soon as i have any results. i'm not inclined to spend any time right now evaluating more complicated strategies until these initial malfunctions have been solved and i better understand the limits of the nt platform and the optimization engine.
very well, thanks, regards.
Comment