Behavior on delayed bar update

[cliffhu]

Hi,

My strategy and the indicator included are based on multi instruments. One symbol is actually an index which doesn't change as quick as others. Its bar close about 5 seconds later. To make sure all bars are updated I have the following in both the indicator and strategy,

protected override void OnBarUpdate()
{

if (Times[0][0] != Times[1][0])
return;
...

}

If I run just the indicator in real time, it works properly. But when running in the strategy, it is not triggered on many bars. The strategy itself is calculating on each bar, though obviously with partial input from the indicator.

Does anyone know what is the problem and what would be the best way to sync these instruments.

Thanks!

[NinjaTrader_Jim]

Hello cliffhu,

Thanks for your post.

I have moved the thread to our Strategy Development forum as it sounds like your main issue involves NinjaScript Strategy Development and reading indicators.

Indicator plots are going to be synchronized to the primary data series.

Bar closures are signaled by the first tick of a new bar, and it can be difficult to attain synchronization between multiple instruments, because we cannot know when their ticks will arrive.

Could you elaborate on your goal, and can you provide an export of a strategy and simple test indicator that demonstrate what you are doing?

Exporting as source code - https://ninjatrader.com/support/help...tAsSourceFiles

With a small example we can review, and a description of your intentions and challenges, we may be able to prove a suggestion.

[cliffhu]

Hi Jim,

Thanks for your quick response.

I understand bar closure is signaled by the first tick of next bar. So my strategy and indicator are waiting for the close of bars for each symbol. If there is delay, we have to accept.

What I do not understand is why the indicator works separately out of the strategy. But not inside of it. It is not just not plotted. From its output, I can see it is not triggered. Actually I have 4 symbols in similar indicators, the other ones with less delay works. Only this index doesn't.

I have attached a simplified example. Please add both the strategy and indicator separately to a 5 min bar chart with live data, use VXX and ^VIX for example.
CliffHu.zip

[NinjaTrader_Jim]

Hello cliffhu,

I am on my way out for the day, but I did finish drafting a script that would use 2 public Series<double>'s that are synchronized to the primary data series and the secondary data series.

In your script, you are using plots which would be synchronized to the primary data series. This may be the issue

Could you have a look at this approach and let me know if it helps? I can spend more time looking into this with you tomorrow if additional help is needed.

[cliffhu]

Hi Jim,

Thanks for the script.

Do you mean in strategies it is plot that triggers indicator update, instead of bar close within the indicator? What if I don't plot it?
Or more precisely, if it is triggered only by the primary bar close from strategy, then "if (BarsInProgress == 1)" in your indicator will not work. This is what we have seen from my script. If this is true, then NT has some issues here.
I think the key thing is to understand how indicator is updated within a strategy.

Assuming the secondary symbol is the late one, the Primary[0] in your script is calculated with Closes[0][0] and "Closes[1][1]" as "Closes[1][0]" is not available yet. This is what I have been trying to avoid.

Anyway, I will test them when market is open.

[cliffhu]

Hi Jim,

I am testing your script with live data. Surprisingly, the bar close of secondary symbol does not trigger either the indicator within strategy or the strategy itself. It seems NT ignores it when delay is too long. Please investigate.

It does trigger the indicator when running out of strategy.

[NinjaTrader_Jim]

Hello cliffhu,

My example does not add plots and instead uses Series<double>'s that are synchronized to the primary and secondary data series. The indicator assigns them in the appropriate BarsInProgress, and the strategy checks them in the appropriate BarsInProgress.

I had to correct a CurrentBar check in the script that I overlooked as I was heading out the door. I re-uploaded the example, and I added some prints in the strategy to print out the values of each Series in their appropriate BarsInProgress index. I see unique prints from both the primary and secondary, which tells me the strategy is reading the signals when they are found in the indicator when it processes BarsInProgress 0, or BarsInProgress 1. (Attached screenshot shows my realtime results.)

If I modify the indicator to add prints, I see the same behavior where I see both Primary and Secondary triggered. Primary and Secondary can come first or second, we can't predict the order because we cannot know if we will get a tick for secondary before a tick for primary, or vice-versa.

Example prints to add in indicator:

protected override void OnBarUpdate()
{
    if (CurrentBars[0] < 1 || CurrentBars[1] < 1)
        return;

    if (Times[0][0] != Times[1][0])
        return;

    if (BarsInProgress == 0)
    {
        primary[0] = Closes[0][0] - Closes[1][0];
        Print("Primary: " + primary[0]);
    }
    else if (BarsInProgress == 1)
    {
        secondary[0] = Closes[0][0] - Closes[1][0];
        Print("Secondary: " + secondary[0]);
    }
}

[cliffhu]

Hi Jim,

I added plot and print to your script before testing. Please see the result and script.

I was wrong about secondary bar close being just ignored. But you can see the result is very messy. Your print is not following a clear order either.

[NinjaTrader_Jim]

Hello cliffhu,

Plots will be synchronized to the primary data series. My suggestion is to instead use Series<double>'s that are synchronized to each data series and to: Update the Series<double> within the BarsInProgress that the Series<double> is synchronized to, and to read the Series<double> from the strategy on BarsInProgress that the Series<double> is synchronized to.

There will not be an expected order of OnBarUpdate events with realtime data, because the bars will get ticks at different times and will close at different times.

The best that can be done is check for matching bar timestamps when one of the bars closes. This could either be the primary or secondary.

If you modify the indicator prints so they look like the following, you may see that the the Actual Time (DateTime.Now) is always going to be in order, showing that the realtime updates for the data series you are comparing do not come in a determined order.

[cliffhu]

Hi Jim,

I understand your approach better now. Still some further questions.

First, I guess it is impossible to combine the two Series<double>? Is it still possible to plot them together / overlaid? In your strategy, AddChartIndicator(IndicatorSecondaryPrimaryCheck1) doesn't draw.

Second, if I do not want to wait for the close of secondary bars, rather use latest available price from past, for example last close of its 1 min bar. So I now have 10 min primary bar and 1 min secondary bar, then plot the calculation. With this print

if (BarsInProgress != 0)
return;
...
Print(Times[0][0] + "; " + Times[1][0] + "; " + DateTime.Now + "; "+ Values[0][0].ToString("N3"));

I got this

28/10/2021 15:00:00; 28/10/2021 14:51:00; 28/10/2021 15:00:04; 0.035
28/10/2021 15:10:00; 28/10/2021 15:01:00; 28/10/2021 15:10:01; -0.387
28/10/2021 15:20:00; 28/10/2021 15:10:00; 28/10/2021 15:20:09; -0.187
28/10/2021 15:30:00; 28/10/2021 15:21:00; 28/10/2021 15:30:02; -0.156
28/10/2021 15:40:00; 28/10/2021 15:30:00; 28/10/2021 15:40:02; -0.565
28/10/2021 15:50:00; 28/10/2021 15:40:00; 28/10/2021 15:50:01; -0.309

It seems Times[0][0] is matched with Times[1][9] or Times[1][10]. Could you tell me where is the problem?

[NinjaTrader_Jim]

Hello cliffhu,

AddChartIndicator was just left as I used your code as the base for what I demonstrated here. It may be removed and ignored.

You could combine the data in a plot to have it visually show a value that is calculated from the secondary series, but that value would be associated with the primary bars. Since we don't know which data series will update first in realtime, or that there will be a bar closure at a certain time, I would recommend using public Series<double>'s synced to each data series so you can properly capture your signals in a strategy.

With regards to the prints, your code is processing only on BarsInProgress == 0. So all of our prints would be 10 minutes apart. Times[1][0] would reflect the timestamp of the secondary bar, and if we did not have a secondary bar closure when that primary bar is processed, we could see timestamps like :51, :01, :10, because a secondary bar would not be formed at that time.

[cliffhu]

Hi Jim,

OK. I will use Series<double>'s to sync them.

About Times[1][0] at the close of primary bar, shouldn't it be 1 minute earlier than Times[0][0]? instead of 9-10 minutes earlier. I don't understand what you mean with "if we did not have a secondary bar closure when that primary bar is processed". I think there are 10 secondary bars closed within 1 primary bar?

[NinjaTrader_Jim]

Hello cliffhu,

If we do not have any data in a given minute, there will be no minute bar.

28/10/2021 15:00:00; 28/10/2021 14:51:00; 28/10/2021 15:00:04; 0.035

With the print above, the primary bar for 15:00:00 had closed after we got the first tick of the next primary bar, which is at 15:00:04 with your PC clock time.

The Secondary bar timestamp for Times[1][0] (the most recently closed bar for the secondary series) is 14:51:00. We see this print because there was not any other bar formations for this data series after that bar.

28/10/2021 15:10:00; 28/10/2021 15:01:00; 28/10/2021 15:10:01; -0.387

With the print above, the primary bar for 15:10:00 had closed after we got the first tick of the next primary bar, which is at 15:10:01 with your PC clock time.

The Secondary bar timestamp for Times[1][0] (the most recently closed bar for the secondary series) is 15:01:00. We see this print because there was not any other bar formations for this data series after that bar.

Let me know if you have any additional questions.

[cliffhu]

Hi Jim,

I am still confused. Between 15:01:00 and 15:10:01 there should be 9 secondary bars closed, which are 1 minute bars. Right?

[NinjaTrader_Jim]

Hello cliffhu,

We only have bars when there is data at that time.

If you reference the screenshot attached, we can see that there are spaces when there are no bars. When there is no bar, there will be no OnBarUpdate iteration. Referencing Times[1][0] will always get the timestamp of the last closed secondary bar. Or in the case of Calculate.OnEachTick/OnPriceChange, the timestamp of the secondary bar that is still developing (not yet closed by a tick of a new bar.)