"The strategy has too many parameters to optimize"

[Seekingpinecones]

Hello,

I have received error message "The strategy has too many parameters to optimize" while using the Strategy Analyzer (for all Optimization, Walk Forward, Multi-Objective).
The custom script I have does not have any issues while Backtesting.
Would someone please share the Max amount of parameters I can have in my script to run the Optimization functions?
*I have tried several searches on this topic and haven't been able to locate the information.

For the script, there are several combinations of SMA and EMA crosses, and other parameters such as price close above/below "x" SMA/EMA. As well as other entry and stop loss rules.

I assume that the provided sample "Sample MA crossover" strategy has 2 optimization parameters - 1. Fast and 2. Slow moving averages. Is this correct?
Or do each sub-parameter count as 1 parameter? for example, "Min", "Max", "Increment" would be 3 parameters and multiplied by 2 (Fast and Slow) for a total of 6 parameters?

Your help would be very much appreciated.

[NinjaTrader_Jesse]

Hello Seekingpinecones,

Thank you for the post.

I am not aware of what the limit would be to bring up that message, we would likely need to put that question in for development to answer or document at a later time. We could further explore the strategy you created to see why you are hitting that though.

Are you currently using a very large number or properties or setting a large number of optimization settings/ranges?

If you have a sample strategy you can provide that would be helpful here. I would not need to see a complete working strategy, it would just need to be an empty strategy that contains your public properties I could test that on my end using the settings that you had to see if I also get that message.

The backtest would not have this message because there are no parameters being optimized, it would be a single instance of the strategy being executed. When you run an optimization it will create many iterations of the strategy instance for the variety of optimization properties set. If you set a large number of properties to optimize at once that can lead to tests which would not complete in any reasonable timeframe. The amount of permutations can get extremely large if you are trying to optimize many parameters over a large range at once.

The included SampleMA does have two properties which can be optimized, the min/max/increment are how the optimziation will work and not parameters. If you set a min of 12 and a max of 13 that would be 2 tests.

I look forward to being of further assistance.

[dave416]

NinjaTrader_Jesse I have the same issue as the OP, and this is really impacting my trading operations that use strategy optimization. This issue didn't occur with earlier builds of NT8, but this message started appearing about a year ago. While it's understandable NT is trying to give the user a better experience here, there is no reasonable cap on the number of parameters that can be executed in a Genetic Optimization given the nature of the search algorithm and the varying convexity of the search space, so having this limitation in place doesn't make much sense.

Attached is an example strategy to reproduce the issue in NT 8.0.25.0 64-bit. It's a simple strategy that checks the close of a bar is above/below the close of a previous bar, and uses 4 of these checks with different parameters and offsets to evaluate an entry or exit. NTtooManyParametersStrategy.cs
I've also attached a set of parameters in a template that were used to trigger the error message Default.zip:

Parameters are..
Data series: EURUSD, daily bars
Start Date: 2019-01-01
End Date: 2022-01-19

All parameters P1 to P4 are: 1;365;1
All parameters P1office to P4 offset are: 0;365;1

I really hope this is addressed in future builds of NT so I can take full advantage of the lifetime license I just acquired. Thank you for taking the time to investigate this!

[NinjaTrader_Jesse]

Hello dave416,

I took a look at the script however the ranges you are using for optimization are fairly large so this type of situation can be expected. When you use very large ranges over many properties that will generate a huge number of permutations. When that happens the length of time which the test would take could be years so its an invalid test.

When using the optimization tools you would generally need to first test the script you made and find the initial values you wanted to start with for the given instrument. Once you know the general starting point you could set smaller ranges around those values to create a test with less permutations.

[dave416]

Hello NinjaTrader_Jesse , thank you for the response.

When you use very large ranges over many properties that will generate a huge number of permutations. When that happens the length of time which the test would take could be years so its an invalid test.

This would be a correct statement for the Default optimizer in Ninjatrader, however this is certainly not true for more efficient optimizers such as the Genetic Optimizer in NT, or other non-grid search algorithms such as Particle Swarm which can very quickly find the most optimal solution over a very large range of parameters. This picture taken from this library shows it well

When using the optimization tools you would generally need to first test the script you made and find the initial values you wanted to start with for the given instrument. Once you know the general starting point you could set smaller ranges around those values to create a test with less permutations.

I would argue that in algorithmic trading, there are no best practices - there is only what works. In science and in my trading strategies, a wide parameter approach works when using non-grid search algorithms, and for this reason I wouldn't expect NT to put any hard limits on this. If you test the attached Strategy in earlier versions of NT8 from 2 years ago before this limit was introduced, it works well.

I would ask Ninjatrader to at the very least build this hard limit into the @DefaultOptimizer.cs code rather than in the core/kernel. For @GeneticOptimizer.cs either leave out the hard limit check since it doesn't apply, or insert soft limits on the number of parameters that can be bypassed. For now I have a number of strategies that are un-usable now which I am porting over to Python.

Thanks for the consideration.

[dave416]

Hello NinjaTrader_Jesse , Could we please have a Feature Request tracking number for this issue? I believe a solid case was made to implement, and we would also need a reference for any lobbying efforts performed to get this resolved.

Thank you

[NinjaTrader_Jesse]

Hello dave416,

Can you clarify what specific feature you are requesting to be added? If you have a specific feature we could submit that with the details you provide, nothing has been submitted based on your last posts.

The hard limit error you previously hit would be expected based on the use case that was provided. The message is being presented based on the analysis of the parameters and the result falling outside what the analyzer is capable of testing. You can find a post which details the iteration limitation, the parameter limitation would also be similar situation. If the given parameters generate too many permutations that may fall outside of what the platform can physically handle and the message is generated.. https://ninjatrader.com/support/foru...tion-variables

The only real solutions here would be to A). refine the parameters so you can use the NinjaTrader tools B). use external tools like the items you mentioned to get a better idea of a starting range of parameters and then use those values in NinjaTrader. Using a huge range will not be able to work in the analyzer a feature request to remove that message is not likely going to be helpful as we are hitting a type/numeric limitation of C# as noted in the other post.

[dave416]

Hello dave416,

Can you clarify what specific feature you are requesting to be added? If you have a specific feature we could submit that with the details you provide, nothing has been submitted based on your last posts.

The hard limit error you previously hit would be expected based on the use case that was provided. The message is being presented based on the analysis of the parameters and the result falling outside what the analyzer is capable of testing. You can find a post which details the iteration limitation, the parameter limitation would also be similar situation. If the given parameters generate too many permutations that may fall outside of what the platform can physically handle and the message is generated.. https://ninjatrader.com/support/foru...tion-variables

The only real solutions here would be to A). refine the parameters so you can use the NinjaTrader tools B). use external tools like the items you mentioned to get a better idea of a starting range of parameters and then use those values in NinjaTrader. Using a huge range will not be able to work in the analyzer a feature request to remove that message is not likely going to be helpful as we are hitting a type/numeric limitation of C# as noted in the other post.

Hi NinjaTrader_Jesse, I've drafted up the problem/solution statement below based on the posts in this thtread - feel free to copy it into the feature request or modify as required. Copying NinjaTrader_Jim who was managing the other thread of the same nature

---

Problem:

During optimization, customers receive "The strategy has too many parameters to optimize". There is no indication of what the limit is. The error appears regardless which Optimization method (Default or Genetic) is selected. This is limiting some legitimate parameter use cases as well as impacting User Experience.

Solution:

The ideal solution would be to remove the hard limit for non-Default (brute force) optimizations.
The optimal solution would be to migrate this hard limit into the open code for the 'brute force' Default optimizer, as this hard limit would only apply to this algorithm, whereas the Genetic optimizer should have no hard limit enforced.

Background:

As demonstrated in this thread, https://ninjatrader.com/support/foru...rs-to-optimize
customer d416 has uploaded a legitimate use case for a strategy with a large number of parameters, however in optimizing the strategy with Strategy Optimizer using the Genetic optimization algorithm, customer receives the above error. Customer did not experience this hard limit in earlier versions of NT8 or at all in NT7.

Why this is an issue:

1) The hard limit is implemented in Ninjatrader 8 to ensure that strategy optimizations are completed within a reasonable amount of time, however this works under the assumption that the user would use the Default optimizer which traverses the entire grid of permutations of strategy parameters. With advance optimization algorithms such as Ninjatrader's Genetic algorithm, or others such as Particle Swarm Optimization, the amount of time to traverse the grid is significantly reduced, thus making a hard limit a hindrance when working on legitimate strategies.

2) As documented in the above thread, the hard limit assumes that no strategy ever should have more parameters than the hard limit as this is "not best practice", however in reality there are many use cases within trading and in science (AI is a good example) which make no assumptions of a baseline set of optimal parameters within a given search space.

[NinjaTrader_Jesse]

Hello dave416,

I will put in a feature request with the details you provided, If there is a specific tracking ID assigned I will update the post at a later time.

[dave416]

Thank you NinjaTrader_Jesse. coming up to the 2 week anniversary of this request, are there any updates? Looking forward to receiving the Feature Request ID

[NinjaTrader_Jesse]

Hello dave416,

At this point I have not received a tracking id, if I do I will update the post at a later time. There are no ETA's on feature requests or if they will be accepted/implemented. If an ID is generated for a feature request I will update the post and then you can use that in the change log to track features with new updates. We are not able to check the status of feature requests.

[BarzTrading]

I would like to add a vote for this feature request. The genetic optimizer iterations are defined by the number of generations times the size of the generation. You could put a limit on this if you wanted to because this is the actual number of iterations that will run.

But it would be really nice to allow a larger total parameter space for the GO. There are valid use cases for this, particularly when optimizing strategy rules rather than, say, just optimizing five indicator lengths from 1 to 1000 in steps of 1 ... which would be rather pointless.

Any update on whether this feature request was ever accepted?

[NinjaTrader_Gaby]

Hello BarzTrading,

Thank you for your suggestion.

I will send this in to our feature request team and they will follow up with me with the number to track this request. I'll get back to you with the number as soon as possible.

[NinjaTrader_Gaby]

Hello BarzTrading,

We are tracking interest for this feature in an existing internal feature request ticket and I have added your vote. This request is being tracked under the number SFT-1801.


As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted on the Release Notes page of the Help Guide.


Release Notes —https://ninjatrader.com/support/help...ease_notes.htm