SMA calculation depends on time frame chosen

[Alex_W]

Hi,

I have a strategy with the following logic:

* Calculate Instrument 1 divided by Instrument 2 ("quotient")
* Calculate two SMAs based on this quotient ("SMA1", "SMA2")
* When SMA1 crosses SMA 2 buy or sell Instrument 2

The SMA calculation is based on a Series:

SMA1 = SMA(myDoubleSeries, SMADailyPeriod1);
SMA2 = SMA(myDoubleSeries, SMADailyPeriod2);

SMADailyPeriodx is the period in days for the respective SMA.

The values for myDoubleSeries are calculated as follows:
private Series<double> myDoubleSeries;
myDoubleSeries[0] = Closes[1][0]/Closes[2][0];

My problem is that the calculated SMAs - especially for short SMA periods like 5 or 10 days - differs significantly based on the time frame chosen.

An example: If I start the backtest in 2009 NT gives me the following values in the output window for 17th Mar 2021 (this is a random date - the problem occurs for the whole time period):
(SMA 1 = 10 days / SMA 2 = 50 days)

17.03.2021 22:00:00 SMA 1: 0,0170115111988172
17.03.2021 22:00:00 SMA 2: 0,0151562639356313

17.03.2021 22:00:00 Ratio: 0,016321433076826

The exact same strategy with an identical setup, but with start date 1st Jan 2021 comes up with these results:
17.03.2021 22:00:00 SMA 1: 0,016741616961223
17.03.2021 22:00:00 SMA 2: 0,0151022850881123

17.03.2021 22:00:00 Ratio: 0,016321433076826

The code for the output window is:
myDoubleSeries[0] = Closes[1][0]/Closes[2][0];

SMAOBV1value=myDoubleSeries[0];

print ("SMA 1: " + SMA1[0].ToString());
print ("SMA 2: " + SMA2[0].ToString());

print ("Ratio: " + SMAOBV1value);

As you can see, especially the SMA 1 differs quite significantly (the earlier the start of the backtest the higher the SMAs) and I have no idea why. The only difference between these two calculations is the start date - everything else (end date, market data, setup) is identical.

I calculated the SMAs by hand and I can reconcile the figures that NT calculates when I start the backtest in 2021 (0.0167416... for SMA 1). I have absolutely no idea why NT comes up with different SMAs depending on the start date of my backtest.

I am using back adjusted data from Kinetick.

Do you need any further information?

Can you help me, please?

[NinjaTrader_Kate]

Hello Alex_W,

Thank you for your note.

The SMA begins its calculations from the first bar of data loaded with the first price available there and does not reset when calculating the SMA for the next bar.

I've added some prints to better illustrate this to the SMA indicator that is built into the platform.

https://ninjatrader.com/support/helpGuides/nt8/moving_average_-_simple_sma.htm

Because starting from a different date would yield a different starting price, we'd expect these to be different.

Please let us know if we may be of further assistance to you.

[Alex_W]

Hello Kate,

thank you for your quick reply. I am aware that different starting points would result in different SMAs, but only as long as the start date is included in the SMA period. This not the case in my case.

To make this more concrete I can give you the following example:
I have an SMA 5 days and calculate this SMA as of The underlying values for these five days in the example are:

12th Mar 2021: 0.0167058673955883
15th Mar 2021: 0.0165426640560854
16th Mar 2021: 0.0164174573055028
17th Mar 2021: 0.016321433076826
18th Mar 2021: 0.0153917050691244

I would expect to see the following 5 day SMA as of 18th Mar 2021:
(0.0167058673955883 + 0.0165426640560854 + 0.0164174573055028 + 0.016321433076826 +0.0153917050691244) / 5 = 0.0162758253806254

What I see, when I start my calculation on 1st Jan 2021 is exactly this figure I am expecting.

If I start the calculation on 1st Jan 2009 NT calculates an SMA of 0.0168156138558141 instead which differs significantly from what is correct.

Since only the period between 12th Mar 2021 and 18th Mar 2021 is of relevance for the 5 day SMA as of 18th Mar it is completely irrelevant whether the backtest starts in 2009 or in 2021 for this calculation. But for reasons I don't understand NT gives me two different SMAs. Especially for shorter SMAs (5 days or 10 days) the difference is materially, for longer SMAs (50 days) there is still a difference, but it is much smaller.

However, for SMAs with a calculation period that starts and ends in 2021 (as it is the case in the given example) it shouldn't make any difference whether the backtesting calculation period starts in 2009 or in 2021. This also results in different trades in the backtesting.

[NinjaTrader_Kate]

Hello Alex_W,

Thank you for your reply.

Sorry about that, you're correct in that if you have two SMAs and the start date is not included in the current period, you should theoretically see those match - clearly I need some more coffee today.

I'm curious as to whether you're running into an issue with double variables given the decimals. What instrument are you testing with to get these? A forex instrument? Which?

Thanks in advance; I look forward to assisting you further.

[Alex_W]

Hello Kate,

thanks for your answer. I am using CL divided by ES. I made two observations:

* The shorter the SMA the larger the error
* The longer the backtesting period (i.e. the earlier the start date of the period) the larger the error

To give you an example, I calculated the 2 day SMA for CL/HS starting in 2009. The result for Jul 2020 to Jun 2021 is the curve shown below:

The blue line is the 2 day SMA, the green line is the quotient (CL/ES) itself. It's obvious that this does not make any sense.

Interestingly there are also periods for which the SMA seems to be correct, as for example in 2015/2016:


When I start my backtest in 2021 I NT calculates correct SMAs, compared to the screenshot above:

Please note that this chart shows the period between Feb to Jun 2021 which is also shown in the first screenshot (which also shows the time period from Jul 2020 to Feb 2021). So, in both screenshots there is the same time period, but the values of the blue line differ significantly while the green line (the quotient itself) has identical values.

This behaviour is quite disturbing and it seems to make my backtesting questionable as the results are not reliable - or hopefully you can figure out something else in my approach that I might have missed, which would be the best outcome.

Thanks!

[NinjaTrader_Kate]

Hello Alex_W,

Thank you for your reply.

I'd be happy to look further into what may be occurring. So that I can ensure we're testing the same calculations, please provide an example script that replicates this behavior.

Thanks in advance; I look forward to assisting you further.

[Alex_W]

Hi Kate,

ok. This means you need the C# code? If this is the case I would try to make the C# code as slim as possible so that all the parts that are not necessary for the SMA calculation are removed in order to make it easier to replicate the calculation.

Shall I send it as an email or what would be the best way to provide this?

Best regards
Alex

[NinjaTrader_Jim]

Hello Alex_W,

This is Jim responding on behalf of Kate who is out of the office at this time.

You may prepare a small example demonstrating what you re running into, and then export the source code so you can attach it here for our review.

Exporting as source code - https://ninjatrader.com/support/help...tAsSourceFiles

Another colleague will be able to review, or Kate may be able to return to this thread once she returns.

[Alex_W]

Hi Jim,

thanks for the information. The SMA is part of a strategy that I wanted to backtest. I therefore have to extract the relevant source code to an indicator. This might take some time, but I hope that I can provide you the source code by tomorrow.

[ntbone]

I suspect NinjaTrader SMA suffers from floating point drift. Look at the source code for the the SMA in @SMA.cs.

After N bars have passed it updates the sum by adding the next value and subtracting the Nth value. Over time this is going to cause accumulated error due to double precision. This might be what you are seeing, especially since you are adding significant historical data to your test by comparing something using historical data as far back as 2009 vs something that is using historical data from January of 2021.

https://stackoverflow.com/questions/...viding-doubles

As listed in the thread, there's a whole field of math that deals with computing and precision that covers not only how to avoid precision loss when computing but also how to analyze how much error is accumulated when you do.

If you want to see how much of a difference this makes try replacing OnBarUpdate in SMA.cs (or clone it) with

        protected void AcurateOnBarUpdate()
        {
            if(BarsArray[0].BarsType.IsRemoveLastBarSupported)
            {
                if(CurrentBar == 0)
                    Value[0] = Input[0];
                else
                {
                    double last = Value[1] * Math.Min(CurrentBar, Period);

                    if(CurrentBar >= Period)
                        Value[0] = (last + Input[0] - Input[Period]) / Math.Min(CurrentBar, Period);
                    else
                        Value[0] = ((last + Input[0]) / (Math.Min(CurrentBar, Period) + 1));
                }
            }
            else
            {
                double accurateSum = 0;

                for(int i = 0; i <= Math.Min(CurrentBar, Period - 1); i++)
                {
                    accurateSum += Input[i];
                }

                Value[0] = accurateSum / (CurrentBar < Period ? CurrentBar + 1 : Period);
            }
        }

This version is computationally more expensive, especially for large periods. You could potentially calculate both methods and compare. Over time I would expect the values of the accurate one to be different from the one provided by NinjaTrader.

More efficient algorithms could be created to reduce the # of calculates required at the cost of a bit more memory usage.

[ntbone]

I just tested the code on a 20 period moving average for a 1 minute chart as well as a 5 period moving average and the error is extremely small so my suspicions appear to be incorrect.

[Alex_W]

Hi NTBone,

thank you for your support. I was also thinking that there might be a double rounding issue. I calculated two kind of SMAs: The "NT way" and "your way". I used the: 3.12 / S&P index in order to create very small figures (as I had in my example). But I couldn't reproduce my problem. I have to look into this a bit further.

Best regards
Alex

[Alex_W]

Hi,

so, I think I was able to reproduce the problem. Based on ntbone's script I created two indicators:

* SMA_NT: This one is based on NinjaTrader's logic
* SMA_Accu: This one is the correct calculation of an SMA as indicated by ntbone

I had to write some code around in order to reproduce my original problem. The indicator is based on the instrument chosen ("SMA instrument") and another second instrument as a parameter. The SMA is based on the ratio "second instrument divided by SMA instrument".

My setting was:
* SMA instrument = ES ##-##
* Second instrument = HG ##-##
* SMA period = 2 days (I chose this SMA period as this is the shortest SMA period possible so that many recalculations are necessary)

Although the input values are the same for SMA_NT and SMA_Accu, the results are different:
* The longer the time period the larger the differences
* For the first days of the underlying period the SMA's are identical for both alternatives, but after only 100 days there's already a difference in the SMAs
* This means: If you run SMA_NT for 1,000 days you will get different SMA's for let's say 11th June as when you run the SMA_NT for 100 days

The differences are small in absolute terms, e.g. 0.0010615 compared to 0.0010658, but in relative terms the differences are quite significant and leads to problems with trading strategies.

As far as I can see it the problem is neglectable/irrelevant for "normal SMA's" for instruments with prices well above 1, but is relevant for instruments/ratios with small figures.

Hopefully my coding is correct as I am not an expert in NT and C# and hopefully it is not too difficult to understand my logic. If you have any questions please let me know.

[NinjaTrader_Kate]

Hello Alex_W,

Thank you for your reply.

We appreciate you sharing your findings as this may be helpful to other users in the future who are working with very small decimal prices.

Please let us know if we may be of further assistance to you.

[ntbone]

I was wondering how things behaved for smaller prices (like currencies). I tried my SMA test on AAPL so accumulation of error would be tougher to find.