Optimizer Suggestions

[Anagoge]

I have some optimizer suggestions/comments:

- Allow optimizing for a given value as it does now (max profit factor, etc.) but also allow some constraints. For example, optimize for max profit factor but only consider runs where the max drawdown is < 20%, or the minimum win % is 75%, the cumulative profit > 80%, etc.
- Support multi-threaded optimizations. On a 2-8 core machine, there is a lot of idle horsepower while optimization runs.
- If I run several large optimization/backtest sequences in a row, the memory for each optimization run is not released, and continues to grow (see the Memory (Private Working Set) numbers in task manager). If I run enough of these sequences, eventually memory grows over 1 GB and almost always, NT then crashes with an error and genreates a trace file if I try to view the optimizer results/tabs. Note that I am optimizing large datasets (30+minute runs) and am saving 250 of the best resulting strategies, but I'd still expect NT to release the memory of the old optimization once you start a new one.
- I would like to be able to right-click the optimization parameters list in the Optiize dialog and choose "Reset to Defaults" and have it re-load the strategy default param values.
- The progress bar that shows while optimizing isn't always centered horizontally in the progress dialog. It is flush with the left window border but has spacing on the right.

I am running Windows Vista 64.

[NinjaTrader_Ben]

Hello,

Thank you for your suggestions. I will pass them on to our development department. There will be improments in these areas in version 7. We do not yet have a release date for version 7.

[Anagoge]

I have one more suggestion:

If I'm optimizing on a single parameter for a strategy that has 12 parameters, all 12 are still displayed in the "Parameters" results column in the "Optimizer" grid/tab. I'd prefer the column only document the parameters that I'm optimizing by specifying a range on them, since it makes the display more readable, and If I'm only optimizing on a single parameter, it would (roughly) allow sorting the results by the parameter value to analyze trends as the parameter value changes.

[NinjaTrader_JoshP]

Thanks for spending time and voicing your thoughts. We will forward this additional suggestion to development as well.

[Anagoge]

Other Ideas/Problems:

- If you add the "# of Trades per Day" column to the optimizer tab grid, it shows the results to 16 decimal places, which seems a little excessive, since you get numbers like this: 5.1234567890123456. I'd suggest 2-4 decimal places instead. You might also consider renaming the column to just "Trades per Day", so the column header is more readable at smaller widths.

- If you right-click any row in the Optimizer tab grid and choose "Performance Viewer", the resulting graphs in the new window that pops up and most other tabs except the Summary/Settings are blank, even though the similar tables below the grid showing the same data properly show the data.

- I know this is part of the NT design for now, but I find it distracting, so... When backtesting/optimizing using daily data, the results graph shows all of the entry/exit trades with arrows on the chart, but they are offset one day, since the trades are not simulated until the next bar starts. It would be great if the chart could then manually offset those to the arrows (move them left one bar, etc.) and then the profit/loss lines between entry/exit would also match the graph of the closing price of the instrument. In general, this offsetting issue seems to cause a lot of confusion in the forums, so maybe something could be done about in NT7.

[NinjaTrader_Ben]

Hello,

1- Thank you for your suggestion. I will forward this to our development department.

2- I cannot reproduce this behavior. Please run a test using the MACrossOver which is a default strategy.

3- This would make it look like the order was placed prior to the actual execution and on the bar that established the signal. You can't place a trade on the signal bar in backtesting. There is nothing that can be done here.

[Anagoge]

2- I cannot reproduce this behavior. Please run a test using the MACrossOver which is a default strategy.

The strategy in use does not make a difference for me. Try this in NT 6.5.1000.6:

• File, New, Strategy Analyzer
• Right click SP 500, AAPL ticker, Optimize
• Choose SampleMACrossover, Day Series, 1 value (I'm using Yahoo daily historical data)
• Leave everything else at the default, hit OK
• Optimizer tab, right click the single result row, choose Performance Viewer
• In the resulting popup dialog, executions, graphs, chart, etc. are all blank, even though the same tabs in the main strategy analyzer windows are not blank

3- This would make it look like the order was placed prior to the actual execution and on the bar that established the signal. You can't place a trade on the signal bar in backtesting. There is nothing that can be done here.

Sorry to be such a pest here, but I just don't get the hangup. I've used other similar programs that can use daily historical data to backtest and can still make the chart show the trade entry/exit markers show in the day the trade signal happened. Think about it from a user's perspective - is the trade more closely associated with the day of the signal or the next day? The next day will often never even touch the previous day's close due to after hours trading, etc. Right now, the transaction arrows and green/red dotted lines often float off above/below the next bar, which isn't very intuitive. I certainly understand you can't trade based on the close until the close happens, but I'm asking the graphs be drawn (or at least have an option to be drawn) more intuitively, not that you violate the laws of physics. Maybe the trade could optionally be timestamped the exact millisecond of the close, so it would still be reflected on the graph of the day's close, or maybe all of the trade indicators could optionally just be shifted back one bar for daily historical data. Yesterday, I demoed NT6.5 to the owner of an investment company (a client of mine), and the first thing he commented on when seeing the backtest results was that the trade arrows floated to the right of the closing bar and was not overjoyed when I told him I asked and it isn't fixable. The strategy I was demoing trades based on a guess at the close a few seconds before close in reality, but I backtest it using the actual historical closes.

I'm a software developer myself, so I certainly understand if your company decides not to implement this sort of feature because the benefit/effort tradeoff isn't there, but to say it isn't possible to make the charts and trade dates more intuitive for people in my case just can't be correct. The great thing about software is the flexibility that almost anything is possible (it is "soft").

[Anagoge]

I have one more suggestion for the optimizer. Id like to see a way to optimize on one or two parameters and have the optimizer generate a graph of those one or two parameters with respect to some optimization goal. So, for example, you could run the sample MA crossover and optimize on the days in the moving average (10-100) and then graph drawdown based on the value of the parameter. This allows you to see the effect of the parameter in an easy way, and to see the general trend of the parameter's values, while also recognizing any anomalous values that do not fit into the pattern.

I'd be perfectly happy being able to chart a single parameter and an optimization goal in 2-D, but some similar optimization tools I've used implement this using 3-D surface graphs when optimizing for 2 parameters. A Google search for "3D Optimization Chart" will show some examples of these graphs for various trading systems.

[NinjaTrader_JoshP]

Anagoge,

We cannot reproduce the optimizer issue you are having on your end. Please try a reinstall of NinjaTrader.

You cannot trade on the signal bar. Reason is because in order for you to even know that signal was valid you already have the bar close value. Because the signal is valid ONLY for that one moment in time, with the close value, you cannot trade on that bar. It is closed so the earliest trade time available to you now is the next bar. Without additional granularity there is no way NinjaTrader can reliably create results. OHLC only gives you one instance of time. You do not know anything inside that bar and as such there is no way we can fill you inside that bar.

Now if you wish to backtest with additional granularity your option is to create a multi-time frame strategy and trade inside the higher granularity time frame while taking signals from the lower granular one. This is what is available to you: http://www.ninjatrader-support.com/v...ead.php?t=6652 No changes will be made in regards to opening up and allowing trades on signal bars in backtesting because that is "cheating". Trading on the signal bar is like saying, "I know this bar closes out as a green bar so it is profitable. Let's just pretend that I had a position in this bar 30 seconds ago so I can capture this good trade".

Not sure what you mean in regards to optimizing goals. There are already optimization goals you can optimize against. Under Optimize you have the option "Optimize on..." and you can choose different goals.

[Anagoge]

I'm not sure why you can't duplicate the issue with the blank tabs, but I'll re-test with the next NT release and see if it is fixed there. I'm not keen on reinstalling programs to fix minor issues, since this bug does not really bother me.

You cannot trade on the signal bar. Reason is because in order for you to even know that signal was valid you already have the bar close value.

I'm probably not being very clear. I don't want to trade during the bar, I only want the buy/sell arrow on the chart to appear in a more logical/intuitive place. Right now the buy/sell arrows appear floating over to the right of the close line. This isn't as big a distraction with the candlestick charts, because they generally incorporate something close to the previous day's close somewhere in next candlestick. In the attached image, you can see the "floating arrows" in blue/pink that would more intuitively match up the the peak/trough of the yellow close line. I'm perfectly clear on the technical reasons why NT does this right now, but I just don't think it is the clearest way to present the data.

If I decide to use NT more after evaluating it I might try the multi-timeframe approach, though it seems like quite bit of overhead and extra data just to move the location of the arrows on the chart to a more logical place.

Not sure what you mean in regards to optimizing goals. There are already optimization goals you can optimize against. Under Optimize you have the option "Optimize on..." and you can choose different goals.

I'm aware of the different optimization goals, but I'm asking for a bar/line graph based on an optimized parameter. Imagine an SMA strategy with one of the moving averages being optimized from 10-20 days and then imagine a bar graph of cumulative profit on the Y axis and the 10-20 parameter on the X axis. This would allow visualizing the cumulative profit effect of moving from 10, 11, 12, ... 20 days and would allow you to see a trend more easily. The Google search I mentioned for "3D Optimization Chart" has several examples of the 3D version of this chart on the first page of hits, but 2D versions (bar, line) are also very useful. If you want me to send you some specific links, send me a PM/email, but I have not posted them here because many examples are from competing products.

[Anagoge]

Another optimizer/backtest results suggestion:

Especially for single-instrument backtesting/optimization, it would be very helpful for me to compare a custom long/short strategy to a buy and hold of the same instrument. For example, it would help if the optimizer's cumulative profit, drawdown, etc. graphs could optionally include a line representing a buy and hold of the same instrument (or a custom instrument). It is great to know a custom strategy would have made 10% last year, but it isn't as impressive if a buy and hold of the same instrument would have instead made 15%. There are some complications here with varying position sizing, etc. making the comparisons non-trivial in some cases, but for the basic case of a strategy using a position size of one share each entry/exit, this would be a great feature.

If someone at NT wants specific links to examples of this feature/graph or more details, let me know, but hopefully my description is clear enough.

This suggestion also hints at a more full portfolio management feature, where while you are out of the the market, your cash position could automatically accrue interest, you could dynamically allocate portfolio resources to different strategies, etc.

[NinjaTrader_JoshP]

In terms of the trade arrows, they are drawn at the location corresponding to the price of the trade. If you are submitting limit orders it is possible for you to be filled outside the range of the bar.

Thank you for your suggestions on the optimizer. I will forward them to development.

[Anagoge]

In terms of the trade arrows, they are drawn at the location corresponding to the price of the trade. If you are submitting limit orders it is possible for you to be filled outside the range of the bar.

In reference to my chart graphic, I'm apparently still not making a clear point, so I'll try one last time. I'm only trading market at the closing price on daily historical data (no limit orders). I don't want the floating arrows to move up or down on the Y axis to match the close line. I'd only like a setting to offset the arrows one bar left (a purely visual setting with no change to the trade engine), so the arrows will point directly to the close that triggered the trade. I realize that some might consider such a chart misleading since the trade at the close price happens right after the close, but what I'm trying to point out is that my suggested optional change would make the graph more intuitive for some others, since the arrow would always point to somewhere on the close line and the arrows would always point to the close that triggered the trade, instead of the next day (again, I know a trade at the close price technically happens in the next bar/day, but that isn't a limitation on how we might ideally visualize the data).

In my chart example, if the trade arrows had such a setting to offset them one place to the left (many indicators [price channels, etc.] in charting packages have such a configurable offset), the pink arrow would line up with the previous day's close (the peak of this graph) and the blue arrow would also line up with the previous day's close (the trough of this graph), rather than "floating", and I think the result would be a more intuitive chart. I've already spent quite a bit of time trying to clarify my point, and it isn't a huge issue to me, so I'll just leave things there, and be content that I've tried my best to explain things...

[NinjaTrader_JoshP]

Anagoge,

The line is not moving to match anything. The arrow is drawn where you are filled. It is not offset in any manner. In terms of visualization you can try just turning off execution plotting and draw your own arrows and paint the background to help you visualize. Try using DrawArrowDown(), BackColor, or BarColor.

Thank you for spending time and voicing your thoughts. I have forwarded your suggestion to development and it will be on the list of future considerations.

[Anagoge]

Another suggestion:
It would be a big help when analyzing backtesting/optimization results if NT allowed attaching custom data/properties/columns to trades that could show in the Trades tab grid of the backtesting results. For example, if I could call order.AddProperty('Volume', BarVolume) after placing an order, that could allow showing the volume of the bar at the time the order was placed in a new column of the Trades grid, and then when I export the Trades grid results to excel I could use that new data to compare the volume on winning/losing trades, for example, to further tweak the strategy. Some tools similar to NT allow this sort of custom columns on analysis results/reports.

In my previous posts, I wasn't trying to claim that NT moves lines or moves arrows inappropriately or erratically (only that the current behavior isn't ideal for all uses). I understand exactly what it does any why. I just want a configuration switch to set the x-axis displacement of the blue/pink trade arrows, just like you can with all of the indicators using the Displacement property.