Best practice for structuring computationally intensive Indicators

[Shansen]

NT Team,

Please advise best practice for structuring computationally intensive Indicators hosted by a multi-time frame, single Instrument Strategy (e.g. ES 09-22).
Hosted Indicators are set to Calculate = Calculate.OnPriceChange, which allows current (intra-bar) Indicator exposed values to be accessed on all time frames. The Strategy evaluates all BarsArrays just before the close of a 1min bar. So, Indicator exposed values are only required at this time, the point of evaluation.

As the hosted Indicators calculate OnPriceChange, they are computationally intensive when calculating exposed values for Bar[0] on each PriceChange. It is sufficient to calculate and display Indicator exposed values for Bar[1] on IsFirstTickOfBar.

As the hosed Indicators only require update when 1) referenced by the Strategy and 2) on IsFirstTickOfBar, is it advisable to:

1. Set all OnBarUpdate calculations to occur on IsFirstTickOfBar evaluating Bar[1], and
2. Set a public method (EvaluateCurrentBar()) to be called by the hosting Strategy to evaluate Bar[0] after Update().

[FONT=Arial]protected override void OnBarUpdate()[/FONT]
[FONT=Arial]{[/FONT]
[FONT=Arial] if (IsFirstTickOfBar)[/FONT]
[FONT=Arial] {[/FONT]
[FONT=Arial]  const int barsAgo = 1;[/FONT]
[FONT=Arial]  EvaluateBar(barsAgo);[/FONT]
[FONT=Arial] }[/FONT]
[FONT=Arial]}[/FONT]

[FONT=Arial]public void EvaluateCurrentBar()[/FONT]
[FONT=Arial]{[/FONT]
[FONT=Arial] Update();[/FONT]
[FONT=Arial] const int barsAgo = 0;[/FONT]
[FONT=Arial] EvaluateBar(barsAgo);[/FONT]
[FONT=Arial]}[/FONT]

[FONT=Arial]private void EvaluateBar(int barsAgo)[/FONT]
[FONT=Arial]{[/FONT]
[FONT=Arial] // Do stuff[/FONT]
[FONT=Arial]}[/FONT]

Any comment or alternative approach is welcome.

As always, thanks
Shannon

[NinjaTrader_ChelseaB]

Hello Shansen,

Any property exposed that is not a series that gets it value set from OnBarUpdate() should use Update().

But this appears a bit redundant. Calling Update from EvaluateCurrentBar(), will trigger OnBarUpdate() which will already call EvaluateBar(), then it would be called a second time as EvaluateCurrentBar() continues evaluating.

The EvaluateBar() called from EvaluateCurrentBar() may not be necessary, unless you are specifically trying to call this when IsFirstTickOfBar is false as well.

[Shansen]

Chelsea,

Thank for your response. If I understand your response correctly, this should work.

My goal is for exposed properties (no shown in the pseudo code), to be updated
1) on IsFirstTickOfBar to calculate for Bar[1], so exposed properties for Bar[1] (not Bar[0]) will be updated, and
2) on a call to the method EvaluateCurrentBar(), so exposed properties for Bar[0] will be updated.

The Indicator is necessarily set to Calculate.OnPriceChange. I am trying to limit the number of re-calculations to
1) once when Bar[1] is complete, and
2) once when required by the hosting Strategy (via EvaluateCurrentBar).


Shannon

[NinjaTrader_ChelseaB]

Hello Shansen,

The Calculate setting of both the host and the hosted indicator should be the same.

Are you getting an error with this code?

[bltdavid]

I think what you're asking is:
How to properly handle the 'Calculate' setting when you
need access to the most recently closed bar?

I use a global variable, which I always call 'B0' -- because
it represents BarsAgo index '0' -- well, really because it's
super brief and easy to type -- my head reads 'B0' as 'bar
zero' but my mind knows it's really 'bars ago zero'.

private int B0 = 0;

Anyways, 'B0' is quick to type, and doesn't mess with code
readability too much -- using some long ass name will play
hell with your fingers, and it will hurt your eyes and sting
your senses when reading code. Just use 'B0', it melts
into the code well.

Next, when you need to access to the most recently closed
bar, never access Close[0] or High[0] directly, but instead,
always use the 'B0' index -- for ex, Close[B0] or High[B0].

The point is, the 'B0' index always represents the correct
BarsAgo index of the most recently closed bar.

This value is usually '0', but sometimes needs to be '1' instead.
How to know when to set it to '0' or '1'?
Ah, good question.

I first define a property, like this,

// return first BarsAgo index for most recently closed bar
protected virtual int FirstBarsAgo {
    get { return (State == State.Historical || Calculate == Calculate.OnBarClose) ? 0 : 1; }
}

and then set 'B0' one time at start of OnBarUpdate,

private int B0 = 0;
protected override void OnBarUpdate()
{
    B0 = FirstBarsAgo;
    if (BarsInProgress == 0)
    {
        if (IsFirstTickOfBar)
        {
            Print("CurrentBar="+CurrentBar);
            Print("Most recent closed bar Close price " + Close[B0]);
            Print("Next recent closed bar Close price " + Close[B0+1]);
            ....
        }
        else
        {
            Print("Most recent closed bar Close price " + Close[B0]);
            Print("Next recent closed bar Close price " + Close[B0+1]);
            ....
        }
    }
}

Always reference the most recently closed bar via B0, B0+1, B0+2, etc.

But, be careful,
You might still use Close[0] to access the price of the most recent close,
which may still be needed in some algorithm or calculation, such as an
inline EMA or SMA calculation.

The point is:
You really need to understand your code well to know when you need
the most recently closed bar (use Close[B0]) vs the most recent close
price (use Close[0]).

This technique has worked well to make my indicators and strategies
independent of the Calculate setting.

Just my 2˘.

[bltdavid]

Some older reading here,
but I think the ideas/problems are still applicable to NT8.

[Shansen]

Chelsea,

The Calculate setting of both the host Strategy and hosted Indicator is the same, both OnPriceChange.

The Strategy uses a custom timer and TriggerCustomEvent (https://ninjatrader.com/support/help...ustomevent.htm) to evaluate for trade entry just before bar close. The Strategy calls the Indicator method EvaluateCurrentBar() to (hopefully) update all Indicator properties (i.e. Series[0]) for reference by the Strategy to evaluate for trade. (not shown in the pseudo code) The Indicator has public Series (e.g. <bool>, etc) to access and expose internal Series (e.g. <bool>, etc) (I understand in line with Exposing Indicator Values (https://ninjatrader.com/support/help...alues_that.htm).

Indicator properties (i.e. Series[1]) are updated at IsFirstTickOfBar to display the Indicator on a Chart. I accept there will be no Indicator display for Bar[0].

I am not getting a error with this code… the code worked using Market Playback. Annoyingly the code does not appear to work using live market connection (the only difference between Market Playback and Live is Market Playback executes 3 secs before close of 1min bar using a more granular 1sec BarsArray, while the Live Connection uses a timer and TriggerCustomEvent()).

Should the pseudo code work? That is, update correctly:
1) on IsFirstTickOfBar to calculate for Bar[1], so exposed properties for Bar[1] (i.e. not Bar[0]) will be updated, and
2) on a call to the method EvaluateCurrentBar(), so exposed properties for Bar[0] will be updated.

_ _ _ _ _ _ _ _

bltdavid, thank you for your input and approach to reference the most recently closed bar.

[NinjaTrader_ChelseaB]

Hello Shansen,

IsFirstTickOfBar applies to all series added with AddDataSeries() that update OnBarUpdate().
Whichever series is currently updating OnBarUpdate is the series IsFirstTickOfBar is pointing to.

The Update() method will trigger OnBarUpdate() to update so calculations in OnBarUpdate are made, this is used to get properties set in OnBarUpdate() up-to-date before the property is returned to the host calling script. OnBarUpdate() will update in the typical fashion with the Calculate setting and IsFirstTickOfBar. Any methods called in OnBarUpdate() would also be run.

Calling EvaluateCurrentBar() would call Update() which would update OnBarUpdate() which would run the EvaluateBar() method if the update is the first tick of the bar, then EvaluateCurrentBar() would call EvaluateBar() a second time, regardless of OnBarUpdate() updated with the first tick of bar or not. The EvaluateBar() called from OnBarUpdate() is redundant and isn't needed, because EvaluateBar() is going to run a second time anyway from EvaluateCurrentBar() no matter what.

[Shansen]

Chelsea,

Thanks again for your clarity in the several NT processes and their behaviour.

Regarding your paragraph on IsFirstTickOfBar, the OnBarUpdate() method is limited to a single BarsArray. For example, "if (BarsInProgress != 0) return;" (or similar) (updated in the below pseudo code). I believe this clarification addresses your valid point.

The method OnBarUpdate() with all code within the IsFirstTickOfBar block is designed to update all Bar[1] properties on IsFirstTick of Bar[0] (i.e. once Bar[1] is complete).

The method EvaluateCurrentBar() is called to evaluate Bar[0], Update() is called to ensure all pointers are update-to-date before evaluating Bar[0]. I am not confident the call to Update() is required here. For clarification, the host calling script is unlikely to call EvaluateCurrentBar() on IsFirstTick, meaning the evaluation of Bar[1] is unlikely to be called.

The reason for this approach is to remove or limit the number of calculations when Calculation must be set to Calculate.OnPriceChange.

protected override void OnBarUpdate()
{
 // Update public properties for Bar[1]. Occurs once per bar based on complete Bar[1] (i.e. at first tick of Bar[0]).
 // For clarity, code is contained entirely in IsFirstTickOfBar.
 if (BarsInProgress != 0) return; 
 if (IsFirstTickOfBar)
 {
  const int barsAgo = 1;
  EvaluateBar(barsAgo);
 }
}

public void EvaluateCurrentBar()
{
 // Update public properties for Bar[0], only called on EvaluateCurrentBar from hosting Strategy. 
 // EvaluateCurrentBar is called the minimum of times and unlikely to ever occur on IsFirstTickOfBar. 
 Update();
 const int barsAgo = 0;
 EvaluateBar(barsAgo);
}

private void EvaluateBar(int barsAgo)
{
 // Do stuff
}

1) With this (hopeful) clarification, does this approach make sense?
2) Is the call to Update() required in EvaluateCurrentBar()?

Again, thanks
Shannon

[NinjaTrader_ChelseaB]

Hello Shansen,

OnBarUpdate() is going to run normally for all BarsInProgress when it updates.

If a host script needs to return a value from an indicator that is specifically not a Series, then Update() should be called in the getter for that property.

Where you have inquiried:
"Is the call to Update() required in EvaluateCurrentBar()?"

It does not appear anything is being returned to the host script. Not enough information is known about what is being evalued in EvaluateBar to know if Update() needs to be run first or why this needs to be run from the host.. But if is working with Series then OnBarUpdate() does need to update first.

That said, EvaluateBar() does not need to be called from EvalateCurrentBar() as this already calls Update() which already calls EvaluateBar().

[Shansen]

Chelsea,

I believe I've misunderstood how hosted Indicators behave.

Where:

• A Strategy primary BarsArray is 1min
• The Strategy (and all child Indicators) are set to Calculate.OnPriceChange
• The Strategy hosts an Indicator
• The Indicator has public Series<>
• The Strategy BarsArray is active (i.e. price changing multiple times per second)
• The Strategy evaluates once three seconds before the close of the bar (i.e. at 57sec of 1min bar)
• The Strategy references the Indicator Series for the first time 57sec into 1min bar.

As the Indicator Series is referenced for the first time 57sec into the bar, is IsFirstTickOfBar() true?
For clarity, the Strategy has no reference to the Indicator Series in OnBarUpdate()

Thanks again.

[NinjaTrader_ChelseaB]

Hello Shansen,

IsFirstTickOfBar will be true in OnBarUpdate() when the indicator processes the first tick of a new bar.

The first indicator reference will cause the indicator to process all historical data.

[jeronymite]

It's a very old thread, but this post references a document by Richard Todd that was useful for NT7: https://ninjatrader.com/support/foru...ategy-analyzer

Perhaps NT Support could review and provide any relevant up-to-date advice appropriate to NT8?

Thanks.

[Shansen]

Thank you for the link (granted to an old thread).

Happy with most the points raised and generally in line with NinjaScript Best Practices | Referencing indicator methods (https://ninjatrader.com/support/help..._practices.htm).

A comment of interest is (NinjaTrader_JoshP) "In general though, it is best practices to only plot what is absolutely necessary. If you do not want to visualize it, do not plot it." (https://ninjatrader.com/support/foru...622#post415622).

In my code a Strategy references an Indicator with 30+ Plots. Specifically, transparent plots to display calculated values in the chart data box (when applied to a Chart) (https://ninjatrader.com/support/help...sindatabox.htm).

1) Could these plots be causing an issue where Calculate = Calculate.OnPriceChange?
2) Where a Strategy hosts the Indicator, the Plots will not be displayed. Should (these) plots be programmatically disabled where a Strategy hosts such an Indicator?

[Shansen]

1. Please confirm my understanding of the OnBarUpdate event between Strategy and Indicator (below)
2. Please advise the best approach to separate intensive calculations (on IsFirstTickOfBar focusing on Bar[1]) and evaluation of Bar[0] (in the closing seconds of a bar). I accept the attached Strategy and Indicator have no such intensive calculations.

If my understanding is correct…
When an Indicator is applied to a Chart,

1. OnBarUpdate occurs with every bar update (regardless of whether the Indicator has a Plot or Public Series Property)
2. IsFirstTickOfBar occurs on the first tick of a bar.

When a Strategy is enabled, similar to above, within the Strategy OnBarUpdate occurs with every bar update and IsFirstTickOfBar occurs on the first tick of a bar.

When an Indicator is hosted by a Strategy, within the Indicator

1. OnBarUpdate does not occur with every bar update. OnBarUpdate occurs when the Indicator Property (Series<> or Variable with Update()) is called by the hosting Strategy. While the Indicator Property could be called by the Strategy with every bar update, this may not be the case.
2. IsFirstTickOfBar occurs on the first tick of a bar when OnBarUpdate is triggered. Again, this could be on the first tick of a bar, but may not be the case. It depends when the Indicator Property is called by the hosting Strategy.
3. When OnBarUpdate occurs, all unprocessed bars are processed sequentially.

Please find attached a Strategy TestIsFirstTickOfBarOccurance and associated Indicator TestIsFirstTickOfBar. The Strategy:

• Calculates OnPriceChange
• Has three BarsArrays
  • BA[0] = 1min,
  • BA[1] = 1sec – to gauge Bar IsClosingThreeSeconds, and
  • BA[2] = 1min – to process after BA[0] & BA[1] are updated (AKA _processBarsArray).
• Hosts Indicator TestIsFirstTickOfBar (with ISeries = BA[0] 1min)
• Calls the hosted Indicator Property in the closing three seconds of a bar.

The below Output Window excerpt appears to confirm my understanding.
The output was generated via a Playback Connection (Markets were closed).
Column 1 shows BA[1] (1sec) time for Strategy Prints and BA[0] (1min) for Indicator Prints.

[FONT=Arial]3:29:01 AM (6:43:25 PM) 2576 Strategy IsFirstTickOfBar[/FONT]
[FONT=Arial]3:29:57 AM (6:43:27 PM) 2576 Strategy IsClosingThreeSeconds[/FONT]
[FONT=Arial]3:29:00 AM (6:43:27 PM) 2575 Indicator OnBarUpdate[/FONT]
[FONT=Arial]3:30:00 AM (6:43:27 PM) 2576 Indicator OnBarUpdate[/FONT]
[FONT=Arial]3:30:00 AM (6:43:27 PM) 2576 Indicator IsFirstTickOfBar[/FONT]
[FONT=Arial]3:30:01 AM (6:43:28 PM) 2577 Strategy IsFirstTickOfBar[/FONT]
[FONT=Arial]3:30:57 AM (6:43:30 PM) 2577 Strategy IsClosingThreeSeconds[/FONT]
[FONT=Arial]3:30:00 AM (6:43:30 PM) 2576 Indicator OnBarUpdate[/FONT]
[FONT=Arial]3:31:00 AM (6:43:30 PM) 2577 Indicator OnBarUpdate[/FONT]
[FONT=Arial]3:31:00 AM (6:43:30 PM) 2577 Indicator IsFirstTickOfBar[/FONT]
[FONT=Arial]3:31:01 AM (7:24:43 PM) 2578 Strategy IsFirstTickOfBar[/FONT]

At 3:29:01 - Strategy IsFirstTickOfBar of Bar 2576 – As expected (no Indicator Prints)
At 3:29:57 - Strategy IsClosingThreeSeconds of Bar 2576 – As expected
- Indicator OnBarUpdate of Bar 2575 – As expected (processing remaining 3sec of ticks)
- Indicator OnBarUpdate of Bar 2576 – As expected (processing 57sec of ticks)
- Indicator IsFirstTickOfBar of Bar 2576 – As expected
At 3:30:01 - Strategy IsFirstTickOfBar of Bar 2577 – As expected (no Indicator Prints)
At 3:30:57 - Strategy IsClosingThreeSeconds of Bar 2577 – As expected
- Indicator OnBarUpdate of Bar 2575 – As expected (processing remaining 3sec of ticks)
- Indicator OnBarUpdate of Bar 2576 – As expected (processing 57sec of ticks)
- Indicator IsFirstTickOfBar of Bar 2576 – As expected