Optimizations fitness stops when getting the first result.

[guillembm]

Hello,

I developed an optimization fitness that worked fine in NT7 but when translated to NT8 it has an unexpected behaviour.

As a simplified example of what I am doing I set this code:


protected override void OnStateChange()
{

if (State == State.SetDefaults)
{

Description = @"MyCustomOpt";
Name = "MyCustomOpt";

}

}

protected override void OnCalculatePerformanceValue(StrategyBase strategy)
{

if (strategy.SystemPerformance.AllTrades.Count < 2

|| strategy.SystemPerformance.AllTrades.TradesPerform ance.ProfitFactor < 80)

Value = 0;

else

Value = strategy.SystemPerformance.AllTrades.Count;

}

This code structure in NT7 gets all the results with 2 trades or more that have a profit factor bigger than 80. I it gets these conditions the GO displays the result. If not the result is zero. I am interested in the results that offer more trades.

The point is that I only get one result and all the others are zero. Thus, I suppose that when it finds the first result different to zero it stops.

Can I avoid it?

[NinjaTrader_Jim]

Hello guillembm,

Thanks for your post.

In your Optimization Fitness Metric, we have two possible outcomes:

1. Overall Profit Factor for backtest instance is less than 80, result is 0
2. Overall Profit Factor for backtest instance is greater than 80, result is AllTrades.Count, or how many trades that strategy has made. (These are not profitable trades only.)

Profit Factor is dependent on the Gross Profit and Gross Loss. You could consider comparing AllTrades.TradesPerformance.ProfitFactor with some PnL values of each trade which would involve looping through each trade, but I am not entirely sure on your end goal.

Looping through trades:

foreach(Trade t in strategy.SystemPerformance.AllTrades)
{
    Print(t.ProfitCurrency);
}

Could you clarify on the statistic you want to calculate for your Optimization Fitness Metric so we can share additional insight?

I look forward to assisting.

[guillembm]

I will try to explain differently.

I want the optimizer to display first, in descending order, the results with an overall profit pactor less than 80 and with more than 2 trades. I do not care if those trades are profitable or not.

So, what you replied is corrrect except for this. I consider all trades, even if they are not profitable.

Thanks

[NinjaTrader_Jim]

Hello guillembm,

I think you want to try something like the following then. I have a print added so you can see the ProfitFactors that are calculated with each strategy so you can see what is getting filtered in the Optimization Fitness Metric. I picked 0.97 for the ProfitFactor instead of .80 so the filtering can be observed easily with a test optimization of the SampleMACrossover strategy. See attached screenshot.

protected override void OnCalculatePerformanceValue(StrategyBase strategy)
{
    if (strategy.SystemPerformance.AllTrades.TradesPerformance.ProfitFactor < .97 && strategy.SystemPerformance.AllTrades.Count > 2)
    {
        Print(strategy.SystemPerformance.AllTrades.TradesPerformance.ProfitFactor.ToString());
        Value = strategy.SystemPerformance.AllTrades.TradesPerformance.ProfitFactor;
    }
    else
        Value = 0;
}

We look forward to assisting.

[guillembm]

It is more or less the same but Profit Factor, if I am not wrong, must not be expressed in a per unit basis and the result I want to display in Value is the number of trades (
strategy.SystemPerformance.AllTrades.Count). A higher number of repetitions means that the patterrns found are more reliable.

Code would be more like this in NT8:

if (strategy.SystemPerformance.AllTrades.TradesPerfor mance.ProfitFactor > 8
&& strategy.SystemPerformance.AllTrades.Count > 2)
{
Value = strategy.SystemPerformance.AllTrades.Count;
}
else
Value = 0;

But, If I make the optimization in NT7 with the same paramters I get completely different results.
The code in the optimization fitness in NT7 is:

public override double GetPerformanceValue(SystemPerformance systemPerformance)
{
if (systemPerformance.AllTrades.TradesPerformance.Pro fitFactor > 8
&& systemPerformance.AllTrades.Count > 2)
return systemPerformance.AllTrades.Count;
else
return 0;
}

I attach the results obtained in both versions.

[NinjaTrader_Jim]

Hello guillembm,

I am getting results as I would expect when I compare a NinjaTrader 7 Optimization Fitness Metric to NinjaTrader 8. I have attached CS files for the fitness metrics I have tested, and I have also provided Strategy Analyzer screenshots showing my settings and results. In the screenshots, I am focusing on the Number of trades that are being entered in the Performance column. My tests were with Kinetick data and my time zone is Mountain Standard Time.

I suggest making sure your backtest is set up to give you apples to apples tests between NinjaTrader 7 and NinjaTrader 8. Both platforms use different fill algorithms so it is best to focus on the a market order only strategy like the SampleMACrossover strategy to set up a controlled experiment. When the backtests align, you can more confidently work on the optimization fitness metric and expect consistency.

CustomMetric is a NinjaTrader 7 optimization fitness metric while MyCustomOptimizationFitness is the NinjaTrader 8 fitness metric.

To ensure you have the same data on both platform, please ensure the platforms are up to data and that you have opened a chart with the data series parameters you would like to backtest, and then right click on the chart and select Reload All Historical Data. This will ensure that incase you have opened a chart from January to March and then again from June until August, any gap from March until June will be filled in.

We look forward to assisting.

[guillembm]

Thanks Jim,

I made several tests with the metrics you attached and they work properly.

I realized that the problem came when optimizing, specially with large periods of data. If I optimize using any strategy I obtain significantly different results between NT7 and NT8. Usually, I obtain better results in NT7.

As we discussed with one of your colleagues in the thread (https://ninjatrader.com/support/foru...t8#post1127840) data in NT8 has been migrated from NT7 in a fresh install. He asked me to load data directly using NT8 and then compare the results. As you can read in the attached thread results throw irrelevant inconsistencies.

But, when optimizing large periods of data, default optimizer gives quite different results. And, when using the genetic optimizer, NT8 sometimes shows no results at all as I show in the attached screeshot.

[NinjaTrader_Jim]

Hello guillembm,

I am using Default 24/7 for Session template/Trading Hours template, and I have Break at EOD unchecked. You will want to ensure Break at EOD is unchecked to have the data align closely to NinjaTrader 7.

Keep in mind, NinjaTrader 7 and NinjaTrader 8 have different historical fill algorithms. You could see some variances with how limit orders are handled if you test other strategies. However, you should be able to achieve the same consistency in our screenshots as long as Trading Hours, Break at EOD and other settings are minded in addition to using verbatim data. Please ensure your PC is set to the same time zone that is configured in NinjaTrader 8, and please also make sure both platforms have MergePolicy set to Merge back adjusted.

Merge Policy can be set in NinjaTrader 7 under Tools > Options > Data, and can be set in NinjaTrader 8 under Tools > Options > Market Data.

If you are still stuck, please reach out to me at platformsupport [at] ninjatrader [dot] com and we can schedule a remote support session to set up a backtest of the SampleMaCrossover strategy that matches. Please include the text "Attn Jim 2835719" and include a link to this thread if you would like to do so.

We look forward to assisting.