Use of SUM on custom Series Exception

[saltminer]

Hi,
when using SUM on a custom Series<double>, an Exception occurs on State change to realtime. (After successfully calculating hundreds of historical bars).
"Exception has been thrown by the target of an invocation".

Calculate was set to Calculate.OnBarClose but changing it made no difference.

Example code (an MFI calculation):

Value[0] = (SUM(negative, Period)[0] == 0 ? 50 : 100.0 - (100.0 / (1 + SUM(positive, Period)[0] / SUM(negative, Period)[0]))); // causes exception in transition to Realtime

I had to program MySum method to replace SUM to get the code operational.

Cheers,
saltminer

[NinjaTrader_JessicaP]

Hello saltminer, and thank you for your report. Would you be able to provide a stripped down copy of your NinjaScript which contains just the custom Series<double> logic and call to the SUM indicator?

[saltminer]

Hi,
in order to address this and another issue in this post:

http://ninjatrader.com/support/forum/showthread.php?t=93618

I will send the entire code via email to support.

Cheers,
saltminer

[saltminer]

I've found a reason for the wierd behaviour. The indicator thinks it is Realtime right from bar zero.

This is when it is instansiated from a strategy at bar zero from inside OnBarUpdate.

if (!bInit)
			{
				// do-once
				ClearOutputWindow();
					
				d3 = cbD3SpotterV2(scanWidth,method,pType,period,smooth,fast,slow,periodD,periodK,priceDiffLimit,indicatorDiffLimit,
									divergenceColor,divergenceDashStyle,divergenceLineWidth,markerDistanceFactor,myAlert1,myAlert2,
									range,emaperiod1,emaperiod2,smiemaperiod,false,
									longwavfilename,shortwavfilename,writealerttext,showpaintbars,showarrows,
									offset,smooth1,smooth2,smooth3,gamma,smoothing,trigger);
			
				bInit = true;
			}

To test I used this:

Print("bar "+CurrentBar+" of "+Count+" State="+State);

produces this result;

bar 0 of 5087 State=Realtime
bar 1 of 5087 State=Realtime
bar 2 of 5087 State=Realtime
bar 3 of 5087 State=Realtime
bar 4 of 5087 State=Realtime
bar 5 of 5087 State=Realtime

When put on the chart as an indicator it works correctly.


UPDATE: When instansiating from within OnStateChange [Configure] the problem remains.

Cheers,
saltminer

[NinjaTrader_JessicaP]

Hello Saltminer,

We weren't quite able to see the same thing you were on our end using the latest version of NinjaTrader; the strategy we were sent runs without error. Please re-run your test with the next release of NinjaTrader. If this behavior then persists, please let us know so we can assist further.

[saltminer]

Hello Jessica,
did you do as I requested in the email and remove my "fix" and actually use the SUM? You must uncomment line 839 and comment line 840. This is for the indicator cbD3SpotterV2 which is used by the strategy.
Also you need to look in the Output Window and see if Ninja is reporting it is realtime on historical bars. There is a Print() in the cbD3SpotterV2 indicator to output this info.

For the SUM problem set the indicator to use the MFI (enum dropdown), and in the code uncomment line 839 and comment my substitution of it at line 840.


//Value[0] = (SUM(negative, Period)[0] == 0 ? 50 : 100.0 - (100.0 / (1 + SUM(positive, Period)[0] / SUM(negative, Period)[0]))); // causes exception in transition to Realtime
double negSum = MySum(negative,Period,0), posSum = MySum(positive,Period,0);

Cheers,
saltminer

[NinjaTrader_JessicaP]

To clarify our methods, I brought the commented out lines under test in isolation, with an environment I specified. I ensured that they were run under historical, transition, and realtime states.

I do need to mention that sub-indicators loading all their data during State == State.Realtime is a known and expected condition, and I used pre-populated data with unit tests based on your code designed with the purpose of detecting flaws in NinjaTrader itself. We were not testing for correct functionality of your code on our end. Based on your replies it sounds as though your sub-indicator may use its own data series. A strategy is unable to use a sub-indicator's secondary data series, and any data not processed when BarsInProgress == 0 is inaccessible to sub-indicators. This is documented in this section of the help guide,

http://ninjatrader.com/support/helpGuides/nt8/en-us/multi-time_frame__instruments.htm

See the "True Event Driven OnBarUpdate() Method" section's notes section.

The data that I used to seed the test :

        private Series<double>        nom; 
        private Series<double>        den; 
        private    Series<double>        negative; 
        private    Series<double>        positive;
        private int periodD = 3;
        private int period = 4;
        private int smooth = 5;

...
 
            den            = new Series<double>(this); 
            nom            = new Series<double>(this); 
            negative            = new Series<double>(this); 
            positive            = new Series<double>(this); 

 

...


        [NinjaScriptProperty] 
        [Display(ResourceType = typeof(Custom.Resource), Description = "Number of bars for smoothing", GroupName = "Parameters")] 
        public int Smooth 
        { 
            get { return smooth; } 
            set { smooth = Math.Max(1, value); } 
        }
  
        [NinjaScriptProperty] 
        [Display(ResourceType = typeof(Custom.Resource), Description = "Numbers of bars used for moving average over D values", GroupName = "Parameters")] 
        public int PeriodD 
        { 
            get { return periodD; } 
            set { periodD = Math.Max(1, value); } 
        }
  
        [NinjaScriptProperty] 
        [Display(ResourceType = typeof(Custom.Resource), Description = "Numbers of bars used for calculations", GroupName = "Parameters")] 
        public int Period 
        { 
            get { return period; } 
            set { period = Math.Max(2, value); } 
        }

Each series was filled with random values during OnBarUpdate. The 6 tests run :

SUM(nom, Smooth)[0]
(SUM(den, Smooth)[0]
// Smooth  = SlowKperiod     // causes exception at realtime
  
SUM(nom, PeriodD)[0]
(SUM(den, PeriodD)[0]
// causes exception at realtime
  
(SUM(negative, Period)[0]
SUM(positive, Period)[0]
// causes exception in transition to Realtime

[saltminer]

Hello Jessica,
thank you for the full explanation, much appreciated.

I tried calling your test indicator from a strategy and it did not experience the same issue that I was reporting. So ultimately there is still the mystery left for me on this one.

Could you please elaborate on "sub-indicators loading all their data during State == State.Realtime is a known and expected condition"? I don't see how processing bar 1 of n hundred Count can be a realtime state.

Thanks,
saltminer

[NinjaTrader_JessicaP]

I would be happy to elaborate. I am providing a section of the help guide which goes into detail on this subject.

State.Historical :

• This state is called once when running an object in real-time
• Initialize any class level variables (including custom Series<T> objects)

State.Transition :

• Prepare realtime related resources

The only state that is being tracked and reported by Ninja at the top level is the Strategy's state; the Indicator has a separate state that is tracked inside it. While the Strategy's own logic is set up during State.Configure, the Indicator is first prepared during State.Historical. There is no indicator processing during State.Transition. The earliest processing is done on the indicator is during the Strategy's State.Realtime processing. Even though the indicator is being run over historical bars, because this processing happens during the Strategy's realtime state, this is the state that is returned to you. You can print MyIndicatorInstance.State alongside State to get a more complete picture.