Genetic Optimizer 2.0 for NinjaTrader 6.5

[whitmark]

For a limited time, I am making available on my website an enhanced version the Whitmark Genetic Optimizer 2.0 that I use to develop my own strategy ideas and support client work. As your strategies grow in complexity and the number of parameters increase or the values/dates expand, it is increasingly difficult to process all of the potential combinations in a day or week, much less in an hour. Even with a modest number of parameters and value/date ranges, the ability to crunch all of the iterations can take literally weeks, months, or even years using the default exhaustive optimization techniques.

As a workaround, many strategy builders attempt to severely reduce parameter ranges to chunk down the problem into multiple, bite-size optimization runs at the risk overlooking parameter combinations that could result in more robust global solution. If you have done this yourself, you know how time intensive and tedious it can be and you're still left with the uneasy feeling, "Did I just unwittingly filter out some of the best solutions just to get the optimizer to run overnight?"

If your strategy development has evolved to require numerous parameters or sizable test ranges, the chances are you have outgrown the default exhaustive search optimizer available in NT 6.5 and can significantly accelerate your system development discovery, research, and testing efforts with the use of a "smart" optimizer. Time is money, and I don't know about you, but I prefer to work smarter using today's technology, not harder.

If you are looking for a supported NinjaTrader genetic optimizer solution today, using tomorrow's enhanced genetic optimization methods, please review the information at the following link:

http://www.whitmarkdevelopment.com/products/wmGO/wmGO.htm

Whitmark

[ctrlbrk]

Fantastic! (runs to download)

An answer to all my prayers!

Mike

[whitmark]

Amen to that! Thanks for checking it out.

Whitmark

[whitmark]

Genetic Optimizer 2.0 for NinjaTrader 6.5

Thank you to those of you who have checked out the new Genetic Optimizer 2.0 for NinjaTrader 6.5 and all of the positive comments I have received. I did not anticipate the concerns regarding the uncertainty that some of you have expressed regarding your ability to port your NT 6.5 strategy to NT 7.0, so I have extended my support for the wmGO 2.0 for NT 6.5 until June 30, 2010 when I anticipate everyone will be taking advantage of NT 7. Please keep the feedback coming to [email protected].

Unless you try it for yourself, the significant productivity gains that can be realized by using the wmGO may not be apparent. For example, suppose you observe the ES market has been exhibiting some great volatility of late, and you want to see if a moving average crossover system can take advantage of the current market behavior. Say you want to explore the use of different moving average types (e.g., SMA, EMA, HMA, WMA) in combinations at different periodicities. To keep things simple, you want to test various fixed tick profit targets and stops, as well as see what trading times are most suited for this trading idea. Finally, you want to explore both trading with the crossover signal and as well as fading it.

Using the reference strategy supplied with the wmGO, wmGORef1Strategy, the expected settings to explore this idea would be:

Parameters                 min;max;incr		
p01_MA1Type                1;4;1       (1=SMA, 2=EMA, 3=HMA, 4=WMA)
p02_MA1Period              2;20;2
p03_MA2Type                1;4;1       (1=SMA, 2=EMA, 3=HMA, 4=WMA)	
p04_MA2Period              5;50;5
p05_ProfitTgtTicks         2;16;2
p06_StopLssTicks           4;20;2
p07_TimeBegBasis           90000;90000;1
p08_TimeBegOffsetMins      30;120;5    (Starting times from 9:30 to 11:30, step 5 mins)
p09_TimeDurationMins       30;270;5    (Trading periods from 30 to 270 mins, step 5 mins)
p10_TradeOrFadeEntry       1;2;1       (1=TradeSignal, 2=FadeSignal)

By all accounts, this is a fairly unsophisticated strategy with a modest number of parameters and uses increments to step through the min/max ranges. Even so, the number of iterations required to process all of the parameter-min/max/increment combinations using the default "brute force" or "exhaustive" approach is nearly 35.3 million iterations. Although I am only optimizing on two months of data and using a blazingly fast Intel i7 chip in my research PC, this would take me three weeks of continual running before the optimization would be completed. However, using the wmGO, I found numerous upward sloping equity curve solutions in just 1280 iterations in less than a minute. (See attached) Of course, I will want to optimize on more data and conduct walk forward robustness testing before even thinking about trading these results.

See for yourself how you can put your strategy research and development efforts on overdrive at the link below:

http://www.whitmarkdevelopment.com/p.../wmGO/wmGO.htm

Please email me at [email protected] if you would like the source code to the reference example to see how easy it is to make your strategy parameters numeric and compatible with the wmGO.

Whitmark

[DangerBoy]

Good to see that software like this has been created for NT, i have watched the video and am really interested in your product although i'm not quite at the stage where i can make the best use of it yet in terms of where i am with my strategy development. I intend on a future purchase but can I ask do you intend on developing a version of the wmGO for NT7?

Cheers
Ross

[whitmark]

Genetic Optimizer 2.0 for NinjaTrader 6.5

Thanks, Ross, for checking out the video and your comments. Certainly, some credit goes to the guys at NinjaTrader for using C# and creating an architecture to support this type of third-party development. Until the beta of NT7 is released and I can determine the feasibility and level of effort to update the wmGO, I cannot yet comment. Given the time intensity of using the default brute force optimizer in NT6.5 and the uncertainty of when NT7 will be out of beta and in a stable release (in particular for intensive optimization runs) I wanted to offer this product to enable system developers using NT 6.5 in the "here and now" who can realize significant and immediate benefits. In addition to being a great productivity tool, the wmGO also enables prototyping capabilities to discover new combinations of indicators, entry/exit rules, money management policies, etc to aid development research.

Thank you for your interest in the wmGO. Feel free to download the trial to test it firsthand.

Whitmark

Please send comments/feedback to [email protected].

[SuperDriveGuy]

Looks promising!

Will this work for intrabar optimization as well! I know thatn the current optimizer only calculates on bar close. Will yours check intrabar? I will illustrate my question...

I want to go long if the current ask is greater that close of previous H1 candle. Currently there is no way to optimize this as NT will always calculate on the current bar's close for the purpose of optimization. This strategy will actually work in market replay and LIVE/SIMULATION mode.

Regards

[whitmark]

Thanks SuperDriveGuy,

The wmGO 2.0 for NT 6.5 has the same limitations that any optimizer will have running on the Strategy Analyzer. You can run strategies whereby your primary signal data series is a 5 minute bar and the secondary trade management data series is a 1 minute (or 1 second or 10 ticks for that matter) to appoximate intrabar operations to enable certain entry and exit techniques. However, all evaluations are done on the last price (close) or in the case of forex where there is no last price, on the bid. I am afraid, in NT 6.5 only market replay will be able to allow you to correctly evaluate your ask logic. Perhaps NT7 will have more to offer in this regard.

Whitmark

[SuperDriveGuy]

Thanks whitmark.

Confirms my worst fears! But thanks for your suggestion of using a primary and a seconday data series to achieve near enough results.

Best of luck with the product. I was thinking of dedicating a PC just for optimization and/or wait for NT7 that would use all the cores of a QUAD core PC etc. But your product might just be the answer.

Regards

[SuperDriveGuy]

Thank you to those of you who have checked out the new Genetic Optimizer 2.0 for NinjaTrader 6.5 and all of the positive comments I have received. I did not anticipate the concerns regarding the uncertainty that some of you have expressed regarding your ability to port your NT 6.5 strategy to NT 7.0, so I have extended my support for the wmGO 2.0 for NT 6.5 until June 30, 2010 when I anticipate everyone will be taking advantage of NT 7. Please keep the feedback coming to [email protected].

Unless you try it for yourself, the significant productivity gains that can be realized by using the wmGO may not be apparent. For example, suppose you observe the ES market has been exhibiting some great volatility of late, and you want to see if a moving average crossover system can take advantage of the current market behavior. Say you want to explore the use of different moving average types (e.g., SMA, EMA, HMA, WMA) in combinations at different periodicities. To keep things simple, you want to test various fixed tick profit targets and stops, as well as see what trading times are most suited for this trading idea. Finally, you want to explore both trading with the crossover signal and as well as fading it.

Using the reference strategy supplied with the wmGO, wmGORef1Strategy, the expected settings to explore this idea would be:

Parameters                 min;max;incr        
p01_MA1Type                1;4;1       (1=SMA, 2=EMA, 3=HMA, 4=WMA)
p02_MA1Period              2;20;2
p03_MA2Type                1;4;1       (1=SMA, 2=EMA, 3=HMA, 4=WMA)    
p04_MA2Period              5;50;5
p05_ProfitTgtTicks         2;16;2
p06_StopLssTicks           4;20;2
p07_TimeBegBasis           90000;90000;1
p08_TimeBegOffsetMins      30;120;5    (Starting times from 9:30 to 11:30, step 5 mins)
p09_TimeDurationMins       30;270;5    (Trading periods from 30 to 270 mins, step 5 mins)
p10_TradeOrFadeEntry       1;2;1       (1=TradeSignal, 2=FadeSignal)

By all accounts, this is a fairly unsophisticated strategy with a modest number of parameters and uses increments to step through the min/max ranges. Even so, the number of iterations required to process all of the parameter-min/max/increment combinations using the default "brute force" or "exhaustive" approach is nearly 35.3 million iterations. Although I am only optimizing on two months of data and using a blazingly fast Intel i7 chip in my research PC, this would take me three weeks of continual running before the optimization would be completed. However, using the wmGO, I found numerous upward sloping equity curve solutions in just 1280 iterations in less than a minute. (See attached) Of course, I will want to optimize on more data and conduct walk forward robustness testing before even thinking about trading these results.

See for yourself how you can put your strategy research and development efforts on overdrive at the link below:

http://www.whitmarkdevelopment.com/p.../wmGO/wmGO.htm

Please email me at [email protected] if you would like the source code to the reference example to see how easy it is to make your strategy parameters numeric and compatible with the wmGO.

Whitmark

Hi Whitmark,
Even your strategy is very good. I have not attained that kind of results for any of my strategies. So many thanks for that. This now points me in the right direction , in terms of what to look for in a setup.

One can tell I am a newbie
Regards

[whitmark]

Genetic Optimizer 2.0 for NinjaTrader 6.5

Glad to set you in the right direction. But keep in mind that Genetic Optimizers are powerful tools that are extremely good at quickly finding parameter values to maximize the selected fitness function. Its up to you to test the results of the wmGO against unseen data to ensure robustness and mitigate the risk of curvefitting the data. You can also use the wmGO on the Strategy Analyzer Walkforward optmizer iteratively optimize and walk forward the best result on "unseen" historical data.

You may want to check out an excellent book on the subject:

The Evaluation and Optimization of Trading Strategies (Second Edition)
2008 by Robert Pardo, President, Pardo Capital Limited

Whitmark

[jdfagan]

Will you consider allowing end users to define their own fitness functions? Or at least expand your fitness functions to include UPI (Ulcer Performance Index)?

Will you consider allowing alternative optimization engines to be plugged in or selectable by user? It looks like you are using one similar to GGO (Genetic Grail Optimizer) add-on from TradeStation days. Other optimization engines that I find very powerful are:

* CMAE (Covariance Matrix Adaptation Evolutionary Strategy)
* PSO (Particle Swarm Optimization)

See AmiBroker's genetic optimization plug-in capabilities here (search for "CMAE" to find right area where he discusses in release notes about adding this optimization engine plugin):

http://www.amibroker.com/devlog/wp-content/uploads/2008/06/readme5130.html

Good luck..I'd be interested in this product if more flexibility regarding fitness function were offered. Allowing for other optimization engines would be a real nice capability bonus as well .

JD

[SuperDriveGuy]

Glad to set you in the right direction. But keep in mind that Genetic Optimizers are powerful tools that are extremely good at quickly finding parameter values to maximize the selected fitness function. Its up to you to test the results of the wmGO against unseen data to ensure robustness and mitigate the risk of curvefitting the data. You can also use the wmGO on the Strategy Analyzer Walkforward optmizer iteratively optimize and walk forward the best result on "unseen" historical data.

You may want to check out an excellent book on the subject:

The Evaluation and Optimization of Trading Strategies (Second Edition)
2008 by Robert Pardo, President, Pardo Capital Limited

Whitmark

Thanks for advice and the book recommendation.

Just one question though, I can't actually "see" your sample strategy to edit it or gain pointers from it for my own coding. What I am missing? Or is that intentional, in which case, no worries, I understand.

Regards,

[NinjaTrader_Dierk]

Guys, here is more info about the genetic algorithm in NT7:
- NT7 will come with a GA which is based on Pete's work here: http://www.ninjatrader-support2.com/...ead.php?t=5932
- NT7 optimizer framework is improved to better support custom development of enhanced optimizers

-> it likely makes sense to hold off any further development efforts until be published first beta of NT7

[whitmark]

wmGO 2.0 for NinjaTrader 6.5

Kudos to the Ray and Dierk for their vision to incorporate a GO into NT7 and retain an open architecture to enable customization including third-party plug-ins. Like everyone else, I am anxiously awaiting NT7 to see what it has to offer, including its optimization capabilities and handling of input parameters and output results.

Based on my understanding of the Pete’s work (who is no longer supporting his valuable contribution btw) the wmGO already has several additional features (e.g., multiple fitness functions, seed control, iteration de-duping, additional optimization controls ) with more planned. The Ulcer Index is on the list as well as the ability to specify your own fitness function. As far as using other algorithms, that is a large undertaking and as Dierk points out, best to wait until NT 7 Beta is released before evaluating.

Several have contacted me with concerns regarding the portability of their 6.5 systems to 7.0 (or the expense if they hired someone to develop their strategy), and were relieved to learn that a supported 6.5 GO solution exists. Of course, there are likely to be several data and memory management enhancements in NT7 that will allow expanded optimization runs, especially when using tick based bars.

Like I have mentioned previously in this thread, my motivation for releasing the wmGO is to help developers in the “here and now” who rely upon the current 6.5 production version of NT and want to take advantage of tomorrow’s technology today to expedite development and research efforts.

If you haven't already taken the wmGO for a spin, you can try it out at:

http://www.whitmarkdevelopment.com/p.../wmGO/wmGO.htm

Please email me at [email protected] if you would like the source code to the reference example to see how easy it is to make your strategy parameters numeric and compatible with the wmGO.

Whitmark