Strategy uses a lot of indicators on each iteration optimization, how to save memory?

[mbcmbc]

Hi,

I have coded an strategy that uses many indicators instances on each iteration in the optimization. For example, one iteration can use Stochastics with many K parameters or SMA with many different periods. They are dynamically choosen on each iteration and they can be more than 15 on each indicator. Each iteration will have his own combinations that may be the same or not as other iterations.

In the strategy I use IsInstantiatedOnEachOptimizationIteration = false and I nullify any old reference to indicators in OnLoadedData as documentation suggests, in order to start a fresh iteration.

The optimization works fine with a small time window (many months), but it run outs of memory if I increase it more.

After some debug I found two problems:

1- All used indicators remains in memory until all iterations are finished. It would be nice release that memory at the end of each iteration.
2- The main amount of memory is consumed by Ninjatrader.NinjaScript.Series<T> that comes from indicators. His MaximumBarsLookBack is Infinite because they are normally Plot values. This size increases as soon I increase time window.


I would like to confirm:
1 - If there is a way to release Ninjatrader.NinjaScript.Series on each iteration (may be releasing Indicators cache system).

2 - I do not need all Indicators data available for strategy calculation, for me would be enough MaximumBarsLookBack as TwoHundredFiftySix (even less). But Values are Infinite due: https://ninjatrader.com/support/help...rslookback.htm.
But Is there a way to force MaximumBarsLookBack as TwoHundredFiftySix on Values (in indicators like Stochastic or SMA)?
Or is possible to code a version of those indicators without Plots and use Custom Series instead of Values? It would work?

3 - Any other suggestion it would be wellcommed.

Thank you for your help.

Regards

[NinjaTrader_Jesse]

Hello mbcmbc,

1) There is not a way to release the series for each iteration in an optimization, that information is needed to produce the results at the end.

2) for plot and price series there is not a way to make that less, for indicators you have the source for you can edit them to use 256 in their code. For system indicators you would need to duplicate the indicator to modify that. I am unsure if this would provide much of a performance gain, it would likely be more advisable to perform more targeted optimizations to avoid larger tests.

3) really all I could suggest here would be to make more targeted optimization. If you have removed any unnecessary indicators or usages of series that would basically be all you can do to increase performance in this aspect. The IsInstantiatedOnEachOptimizationIteration is not really intended to save performance for series based data but for objects you create and store as class variables. The indicators and series based data are still being re created for each test as shown in the sample you linked to.



I look forward to being of further assistance.

[mbcmbc]

Hi Jesse,

Thank you for your response,

As I only need last Value of each indicator I'll code indicators versions without series and plot values, just LastValue calculated in OnUpdateBar. Something like (EMA example):
protected override void OnBarUpdate()
{
if (CurrentBar > 0)
{
LastValue = Input[0] * constant1 + constant2 * LastValue;
}
else
{
LastValue = Input[0];
}
}
public double CurrentValue
{
get { Update(); return LastValue; }
}

If there are not performance gain with this versions of indicators I'll try more targered optimizations

Regards