Sortino Ratio always 0

[Zardo]

Hello,

The Sortino Ratio on the Strategy Performance window is always 0 or very near, despite the Sharpe Ratio near 1, which doesn't make any sense imo. See attachment.

Although we can make custom performance metrics, the NT default metrics of the strategy analyser are far behind the competition. This is an area that you should improve (also lack the portfolio analyser feature).

NT 8.0.6.0 64-bit (Standard)

[NinjaTrader_PatrickH]

Hello Zardo,

Thank you for your post.

Do you have a definition or formula for Sortino Ratio that differs from ours in the Help Guide?

• Sortino Ratio formula in Help Guide: http://ninjatrader.com/support/helpG...ngSortinoRatio

Are you stating the Sortino Ratio you are seeing is incorrect based on the formula in the Help Guide or is it incorrect based on another formula?

I look forward to your response.

[Zardo]

Hi Patrick,

The formula in the help guide is correct. The problem should be in the implementation. Maybe in the way you are calculating the standard deviation of monthly drawdown. For the ratio to be around zero, this value must be much bigger than the Profit per Month, which is not the case.

Here one paper about this subject

[NinjaTrader_PatrickH]

Hello Zardo,

Thank you for your response.

Based on your inquiry do you always see the value incorrect to the formula? If you were to run the SampleMaCrossover do you see the same behavior? If so, could you provide the details on the instrument, period type, value, From and To date tested, etc. so we may recreate on our end?

I look forward to your response.

[Zardo]

Patrick,

I personally don't remember seeing any print screen of the strategy analyzer on the web with the Sortino ratio different from 0, 0.1 or 0.2.

I've attached the simulation you asked.

[ShatteredX]

I'm also having this problem. Sortino ratio is always between 0 and 0.1. I believe NT8 is calculating Sortino incorrectly.

NT8 help guide defines Sortino as:
(Profit per Month –risk free Rate of Return) / standard deviation of monthly drawdown.


Since risk free is set to 0 in the source code for Sortino, Sortino should simply be Monthly profit / Std dev of monthly drawdown.


See the following example:


From a random backtest of one of my strategies:
Monthly profit: $15,031.64
Std dev. of monthly drawdown: $8281.32


I calculated these numbers in Excel alongside the exported trade statistics.


So Sortino should be 15031.64/8281.32 = 1.82.
But NT8 says Sortino = 0.05.


P.S. The definition of Sortino ratio in the help guide is not quite correct. The denominator of the Sortino calcuation should really be:






In practice, the resulting Sortino numbers are close:

Sortino (as defined in the OP's link): 1.70
Sortino (as defined by NT8 guide ): 1.82

To be honest I'm fine with how the NT8 guide defines Sortino, but it's definitely not implemented correctly in the current NT8 version.

Someone please correct me if any of the above is incorrect.

EDIT: The definition for Max Sortino Ratio in the NT8 guide uses the correct definition of Sortino as defined in the PDF the OP linked. It says the denominator of Sortino is the Ulcer Index which I believe is the same thing as that pictured equation above. I think the problem is the Ulcer Index is not being calculated correctly in NT, like it's missing a SQRT() somewhere.

[NinjaTrader_PatrickH]

Hello,

Thank you for your posts ont his matter.

If you print out the values used in the Sortino do you see an incorrect Ulcer or any other value?

				Print("Monthly Ulcer: " + SystemPerformance.AllTrades.TradesPerformance.MonthlyUlcer);
				Print("Monthly StDev: " + SystemPerformance.AllTrades.TradesPerformance.MonthlyStdDev);
				Print("Profit per month: " + SystemPerformance.AllTrades.TradesPerformance.Percent.ProfitPerMonth);
				Print("Sortino: " + (SystemPerformance.AllTrades.TradesPerformance.Percent.ProfitPerMonth / SystemPerformance.AllTrades.TradesPerformance.MonthlyUlcer));
				Print("Sharpe: " + (SystemPerformance.AllTrades.TradesPerformance.Percent.ProfitPerMonth / SystemPerformance.AllTrades.TradesPerformance.MonthlyStdDev));

If you do see incorrect values can you provide what you are comparing those values to on your end?

I look forward to your response.

[ShatteredX]

Thank you for replying, NinjaTrader_PatrickH!

I have added those lines to SampleMACrossOver in NT8 to test a strategy.

Strategy set-up:
Fast: 16
Slow: 45
Instrument: CL 08-17
30 minute bars
07/01/2016 to 06/23/2017
Include commission: No
Bars required: 45
Slippage: 1
Exit on close: No
Default Quantity: 1







Here is the final output:
Monthly Ulcer: 1.07551898644733
Monthly StDev: 0.0628252769928223
Profit per month: 0.0468054362431505
Sortino: 0.0435189307050348
Sharpe: 0.745009628027552

Monthly StDev is correct, profit per month is correct, Sharpe is correct.

Monthly Ulcer and Sortino are not correct.

I calculated Monthly Ulcer and Sortino in Excel using the exported data:

Monthly Ulcer:
SQRT(Summation((cumulative currency profit - maximumrealized currency profit) ^2 )/ 12 months) = 0.024


Sortino = Profit per month / Monthly Ulcer = 0.0468 / 0.024 = 1.95

It appears that the main problem is that Monthly Ulcer is not being calculated correctly, which causes Sortino to be incorrect as well.

Thanks for your help!

[ShatteredX]

Any updates on this?

As I stated in the previous post, the problem seems to be that Monthly Ulcer is not being calculated correctly, which results in Sortino being incorrect as well.

[NinjaTrader_PatrickH]

Hello ShatteredX,

Thank you for your note.

There is no update at this time. Development is looking into this and I will update when we have further details.

[NinjaTrader_PatrickH]

Hello ShatteredX and Zardo,

Thank you for your patience.

Can you confirm the following?

1. NinjaTrader will use 30.5 days to represent a month between the first entry trade and the last exit trade on a performance result set.
2. Risk Free Rate of Return will be 0 unless the strategy sets this exclusively.
3. There must be at least 12 months of data before the Sharpe and Sortino ratio will present any meaningful results.

Do these items change your calculations?

I look forward to your response.

[ShatteredX]

Hi NinjaTrader_PatrickH,

Thanks for your response.

Those items do not affect my calculations. I was aware of #1 and #2 already and #3 does not affect my results since my backtest was a year long.

The problem is MonthlyUlcer is not being calculated correctly. This then makes the Sortino incorrect because NT8 calculates Sortino using Monthly Ulcer.

Also, the Ulcer Index isn't even being displayed properly in the Strategy Analyzer Summary. It should be in terms of: Currency, Percent, Ticks, etc. not just a static decimal value as noted in the NT8 guide: https://ninjatrader.com/support/help...efinitions.htm

Thanks!

[ShatteredX]

Hi,

FYI, this problem is still present in NT8 8.0.9.

[NinjaTrader_PatrickH]

Hello ShatteredX,

Thank you for your response.

Can you provide a screenshot from the performance that was ran with the following items in mind?

1. NinjaTrader will use 30.5 days to represent a month between the first entry trade and the last exit trade on a performance result set.
2. Risk Free Rate of Return will be 0 unless the strategy sets this exclusively.
3. There must be at least 12 months of data before the Sharpe and Sortino ratio will present any meaningful results.

I look forward to your response.

[ShatteredX]

NinjaTrader_PatrickH,

Thanks for the reply.

The Ulcer index and Sortino ratio are wrong.

Also, the Ulcer index should be in terms of the units selected: Currency, Percent, etc. It has units just like standard deviation does.