Is default ATR precise?

[Zapzap]

I am making my own ATR with some changes, and I realized, that default NT ATR returns illogical values.

I tried a period 2 ATR on YM (which has a tick size of 1) on a shallow period, where true ranges were 1,2,3.
Logically only values of full integers (1,2,3) or halfs (1.5, 2.5) should show, but I get 2.88, 2.49, etc, which is impossible.

period 1 ATRs are okay.

What am I thinking wrong?

[koganam]

I am making my own ATR with some changes, and I realized, that default NT ATR returns illogical values.

I tried a period 2 ATR on YM (which has a tick size of 1) on a shallow period, where true ranges were 1,2,3.
Logically only values of full integers (1,2,3) or halfs (1.5, 2.5) should show, but I get 2.88, 2.49, etc, which is impossible.

period 1 ATRs are okay.

What am I thinking wrong?

Looks like double storage rounding errors, which will be exacerbated on such a small time frame.

[Zapzap]

Well, if in any systems (3+3) / 2 = 2.88 because of rounding errors, then space expeditions must be fake.

[koganam]

Well, if in any systems (3+3) / 2 = 2.88 because of rounding errors, then space expeditions must be fake.

Take a look at the code, and you will see why errors can be so easily magnified on a small time frame. You will see that the calculations, because of the way that they are done, carry the effects of all preceding round-off errors. What you are seeing are the results of roundoff errors for the entire chart.

Try a bigger time frame/lookback, and see if the deviations are not smaller.

Writing your own version, starting with a copy of the existing version, and making sure to round the Plot value to TickSize would also probably cut down the total error.

[Harry]

I am making my own ATR with some changes, and I realized, that default NT ATR returns illogical values.

I tried a period 2 ATR on YM (which has a tick size of 1) on a shallow period, where true ranges were 1,2,3.
Logically only values of full integers (1,2,3) or halfs (1.5, 2.5) should show, but I get 2.88, 2.49, etc, which is impossible.

period 1 ATRs are okay.

What am I thinking wrong?

The average true range is not a simple moving average of the true ranges but an exponential moving average. NinjaTrader uses the original formula developed by Welles Wilder. The NinjaTrader formula is correct.

Also there are no significant rounding errors.

The only problem that I can see with the average true range is that it returns too small values when you use it with "CalculateOnBarClose = False". The reaons here is that the true range of the developing bar is usually smaller while it develops. For this reason the calculation of the ATR would be skewed to the downside. I therefore recommend not to use the ATR with "CalculateOnBarClose = False".

For indicators that use the average true range to calculate channels or a stop loss, the problem can be handled by using the average true range one bar ago. For example, my version of the SuperTrend is based on the average true range on bar ago, as this way it cannot be skewed to the downside.

[koganam]

The average true range is not a simple moving average of the true ranges but an exponential moving average. NinjaTrader uses the original formula developed by Welles Wilder. The NinjaTrader formula is correct.

Also there are no significant rounding errors.

The only problem that I can see with the average true range is that it returns too small values when you use it with "CalculateOnBarClose = False". The reaons here is that the true range of the developing bar is usually smaller while it develops. For this reason the calculation of the ATR would be skewed to the downside. I therefore recommend not to use the ATR with "CalculateOnBarClose = False".

For indicators that use the average true range to calculate channels or a stop loss, the problem can be handled by using the average true range one bar ago. For example, my version of the SuperTrend is based on the average true range on bar ago, as this way it cannot be skewed to the downside.

Take another look. Here is the critical line:

Value.Set(((Math.Min(CurrentBar + 1, Period) - 1 ) * Value[1] + trueRange) / Math.Min(CurrentBar + 1, Period));

We multiply the previous value by the (period - 1), which in a world with no storage rounding error should result in the corrected proportion of the total sum of the TrueRange for the calculation period on the previous bar, to which is added the TrueRange of the CurrentBar, then divided by the period. The net result is to end up creating Wilder's Average, and a consequent infinite impulse response filter. Every IIR, by definition, propagates errors, from any source. The real issue is the first part of the process, which assumes that the previous values were evenly distributed, and so can be proportioned to get a simple measure of the total of the previous (period - 1) value. It does not help either that in Wilder's time in question, these calculations were done by hand, and this provided a quick and dirty method of calculation.

That having been said, I agree with you that my first response was not quite correct, and I stand corrected. (I am rolling eyes at myself, not you. )

[arbuthnot]

It's been fascinating to read this thread.

John von Neumann and Alan Turing, great mathematicians and computer pioneers both, recognised the problem of the accumulation of rounding errors in the very early days of computing in the 40s and 50s, so the problem's been around a long time!

Harry, you pointed out that Wilder's ATR is of an exponential moving average type. It's the recursive nature of these formulae (sorry: I'm English - 'formulas' if you like) that produces the problem.

It shouldn't be difficult at all to rejig the formula to a WMA type: EMA has, as it were, an infinite tail but WMA can be calculated with a 'finite' loop.

Koganam, you pointed out that in

Wilder's time ... these calculations were done by hand, and this provided a quick and dirty method of calculation.

We have the advantage of having fast computers.

It's very late here now but I'll try and give this problem some thought during the week. Even my muddled thinking leads me to believe the rounding problem will disappear.

Someone better qualified than me may well beat me to it and do a better job (hint, hint).

P.S. As it happens, I grew up on the very street (Castellain Road in London) where Alan Turing was born. Does anyone think this would qualify me for a free lifetime Ninja licence? I doubt it, somehow

[sledge]

It's been fascinating to read this thread.

John von Neumann and Alan Turing, great mathematicians and computer pioneers both, recognised the problem of the accumulation of rounding errors in the very early days of computing in the 40s and 50s, so the problem's been around a long time!


Someone better qualified than me may well beat me to it and do a better job (hint, hint).

I won't say I'm better qualified - but I deal with matching 2 systems with of data with rounding errors.

It's kind of complicated but out of scope here. Suffice to say, we use a fudge factor in System B to handle the rounding, both intra-year and in year totals to make all the #s match.

System A is racked and stacked one way.
System B is another formula using a % rate in total. So total amount = rate * allocated amt + factor;

[Harry]

Harry, you pointed out that Wilder's ATR is of an exponential moving average type. It's the recursive nature of these formulae (sorry: I'm English - 'formulas' if you like) that produces the problem.

The original problem was not linked to rounding errors.

I have come across rounding errors at several occasions. I remember that both the NinjaTrader Stochastics and the PSAR were heavily impacted.

[arbuthnot]

Well, if in any systems (3+3) / 2 = 2.88 because of rounding errors, then space expeditions must be fake.

I've now coded up a WMA-type ATR. (Please see post #7 for motivation.)

If public bool TrueForNotWeighted = true, then the result is not weighted. Set to false, the result is weighted as in WMA.

I've used a Mult variable (instead of TickSize). I work with Forex, typical value around 1 with 4 decimal points. Setting Mult to 10000 will make this an integer inc. the significant digits. For the ES, etc., set this to 1.

Anyway, I've done what I set out to do, and if no one else will, I'll be using it!

I think having the weighted and unweighted versions is quite helpful.

I've attached the indicator as well.

OnBarUpdate:

if ( CurrentBar < Period )
return;

int i = 0;
double RunningATR = 0;
int p = Period * ( Period + 1 ) / 2;

if (!TrueForNotWeighted)

{
while ( i < Period ) // WEIGHTED VERSION

{
RunningATR = RunningATR + ( Period - i ) * ( Math.Max( Math.Abs( Low[ i ] - Close[ i + 1 ] ), // continued next line
Math.Max( High[ i ] - Low[ i ], Math.Abs(High[ i ] - Close[i + 1] ) ) ) );
i++;
}

Value.Set( RunningATR * Mult / p );
}

else
{
while ( i < Period ) // UNWEIGHTED VERSION

{
RunningATR = RunningATR + ( Math.Max( Math.Abs( Low[ i ] - Close[ i + 1 ] ), // continued next line
Math.Max( High[ i ] - Low[ i ], Math.Abs(High[ i ] - Close[i + 1] ) ) ) );
i++;
}

Value.Set( RunningATR * Mult / Period );

[Harry]

I suggest that you clean up your code to improve CPU performance.

Imagine that you run your indicator in CalculateOnBarClose = false. Then your code runs an iteration over N bars with every incoming tick. Each of those iterations performs Math.Max and Math.Abs calculations.This is not necessary.

(1) Do not calculate the true range for each of the last N bars with every incoming tick. It is sufficient to calculate the true range for the current bar only. This reduces the CPU load by a factor N.

(2) The formula for the true range can be simplified. You only need one MAX and one MIN calculation instead of 2 MAX and 2 ABS calculations. No such calculations are required within the iteration, if you store the true range in a DataSeries object.

(3) Calculation of p can be done in OnStartUp().

Below is a simple variation of your code that should run much faster and achieve the same results. The iteration is only calculated once per bar, and the true range is calculated in a simpler way. The code can be further improved, the lines below are only meant to serve as an example for clean code.

protected override void OnStartUp()
{
    if(weightedMean)
        divisor = period * (period + 1) / 2;
    else
        divisor = period;
}    

protected override void OnBarUpdate()
{
    if (CurrentBar == 0)
    {    
        trueRange.Set(High[0] - Low[0]);
        Value.Set(trueRange[0]);
    }
    else if (CurrentBar < period - 1)
    {    
        trueRange.Set(Math.Max(High[0], Close[1]) - Math.Min(Low[0], Close[1]));
        sum = 0.0;
        for (int i = 0; i <= CurrentBar; i++)
                sum = sum + trueRange[CurrentBar - i];
        Value.Set(sum /(CurrentBar + 1));
    }
    else if(weightedMean)
    {
        if(FirstTickOfBar)
        {    
            sum = 0.0;
            for (int i = 0; i < period-1 ; i++)
                sum = sum + (i + 1) * trueRange[period - 1 - i];
        }    
        trueRange.Set(Math.Max(High[0], Close[1]) - Math.Min(Low[0], Close[1]));
        Value.Set((sum + period * trueRange[0])/divisor); 
    }
    else
    {    
        if(FirstTickOfBar)
        {    
            sum = 0.0;
            for (int i = 0; i < period - 1; i++)
                sum = sum + trueRange[period - 1 - i];
        }    
        trueRange.Set(Math.Max(High[0], Close[1]) - Math.Min(Low[0], Close[1]));
        Value.Set((sum + trueRange[0])/divisor);
    }    
}

Not tested.

[arbuthnot]

This is just a quick reply for the moment before I implement the changes but thanks so much, Harry, for the time you've taken with this.

I undertook the exercise just to see if I could convert ATR from EMA-type to WMA-type in a short space of time. It was worth my taking the time because I've learnt and will learn (when I go through the code in detail) so much from your post.

Just to give one example, I would never have considered putting the calculation of p in OnStartUp(), a method I've rarely used but I've known is there. I'll know to do this sort of thing in future.

Your basic message is: make code as CPU efficient as possible and, after reading your post, I and other readers of this thread will find this a far easier task to achieve.