Multi timeframe strategy - Order of BarUpdates

[Shansen]

NT Team,

Apart from Calculate.OnEachTick and Calculate.OnBarClose, what else can change the order in which BarsArray updates trigger where both BarsArrays are the same Instrument and different time-frames? Both of the below sequences were generated in Playback(Historical).

Please find below the sequence of expected behaviour (in line with Working With Multi-Time Frame Objects), with Calculate.OnEachTick and IsFirstTickOfBar is used to identify the close of the 5min BarsArray (BA[0][1]) and the concurrent developing 15min BA[1][0], and in turn the close of the 15min BarsArray (BA[0][1]), and:
- BIP (BarsInProgress) is shown as BarsInProgress, CurrentBars[0], & Times[BarsInProgress][0]
- BA[0] 5min is shown as target barsAgo0 & Times[0][barsAgo0], and
- BA[1] 15min is shown as target barsAgo1 & Times[1][barsAgo1]

        [B]BIP[/B]	                [B]BA[0] 5min[/B]	[B]BA[1] 15min[/B]	
0 72 (4:05 PM) --> 	[1](4:00 PM),	[0](4:15 PM)
1 72 (4:15 PM) --> 	[1](4:00 PM),	[1](4:00 PM)
0 73 (4:10 PM) --> 	[1](4:05 PM),	[0](4:15 PM)
0 74 (4:15 PM) --> 	[1](4:10 PM),	[0](4:15 PM)
0 75 (4:20 PM) --> 	[1](4:15 PM),	[0](4:30 PM)
1 75 (4:30 PM) --> 	[1](4:15 PM),	[1](4:15 PM)
0 76 (4:25 PM) --> 	[1](4:20 PM),	[0](4:30 PM)
0 77 (4:30 PM) --> 	[1](4:25 PM),	[0](4:30 PM)
0 78 (4:35 PM) --> 	[1](4:30 PM),	[0](4:45 PM)
1 78 (4:45 PM) --> 	[1](4:30 PM),	[1](4:30 PM)
0 79 (4:40 PM) --> 	[1](4:35 PM),	[0](4:45 PM)
0 80 (4:45 PM) --> 	[1](4:40 PM),	[0](4:45 PM)
0 81 (4:50 PM) --> 	[1](4:45 PM),	[0](5:00 PM)
1 81 (5:00 PM) --> 	[1](4:45 PM),	[1](4:45 PM)

What are the possible causes for the below behaviour, where behaviour that diverges from the above expected behaviour is in bold?

        [B]BIP[/B]	                [B]BA[0] 5min[/B]	[B]BA[1] 15min[/B]	
0 72 (4:05 PM) --> 	[1](4:00 PM),	[0](4:15 PM)
1 72 (4:15 PM) --> 	[1](4:00 PM),	[1](4:00 PM)
[B]0[/B] 73 (4:10 PM) --> 	[1](4:05 PM),	[0](4:15 PM) - note 5 consecutive BA[0] updates
[B]0[/B] 74 (4:15 PM) --> 	[1](4:10 PM),	[0](4:15 PM)
[B]0[/B] 75 (4:20 PM) --> 	[1](4:15 PM),	[0][B](4:15 PM)[/B]
[B]0[/B] 76 (4:25 PM) --> 	[1](4:20 PM),	[0][B](4:15 PM)[/B]
[B]0[/B] 77 (4:30 PM) --> 	[1](4:25 PM),	[0](4:30 PM)
1 77 (4:30 PM) --> 	[1](4:25 PM),	[1](4:15 PM) - expected to occur after [0][75]
0 78 (4:35 PM) --> 	[1](4:30 PM),	[0][B](4:30 PM)[/B]
0 79 (4:40 PM) --> 	[1](4:35 PM),	[0][B](4:30 PM)[/B]
0 80 (4:45 PM) --> 	[1](4:40 PM),	[0](4:45 PM)
1 80 (4:45 PM) --> 	[1](4:40 PM),	[1](4:30 PM) - expected to occur after [0][78]
0 81 (4:50 PM) --> 	[1](4:45 PM),	[0][B](4:45 PM)[/B]
0 82 (4:55 PM) --> 	[1](4:50 PM),	[0][B](4:45 PM)[/B]
0 83 (5:00 PM) --> 	[1](4:55 PM),	[0](5:00 PM)
1 83 (5:00 PM) --> 	[1](4:55 PM),	[1](4:45 PM) - expected to occur after [0][81]
0 84 (5:05 PM) --> 	[1](5:00 PM),	[0](5:00 PM)

It is as though BA[1] BarUpdate is triggering at the end of the bar? Again noting Calculate is set as OnEachTick.

I am sure it is obvious... but I can not crack it.
Any ideas for further investigation are appreciated.

[Shansen]

The issue of BarUpdate events diverging from expectation in a multi time-frame Strategy appears to be caused by adding a BarsArray based on the Primary Instrument with specified Trading Hours. For example:

AddDataSeries(instrumentFullName, new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, tradingHoursName);

Please find attached :

• an example Strategy, StratA,
• a picture of the expected sequence of bar updates and resultant errant sequence (as noted in the prior post BIP (BarsInProgress) is shown as BarsInProgress, CurrentBars[0], & Times[BarsInProgress][0]. In the video I misspoke, Times[BarsInProgress][0] occurs at the first tick of Bar[0]), and
• an explanatory video (https://www.screencast.com/t/OTOQoTTL). While I concluded the video saying I'd talk further, I believe the concept was fully explained.

Please note, all BarArrays are based on the Primary Instrument (i.e. 6A 03-18).

Please assist

As always, thanks
Shannon

[NinjaTrader_Jesse]

Hello,

Thank you for the post.

I will review what has been provided and reply back once I have further details.

I look forward to being of further assistance.

[NinjaTrader_Jesse]

Hello,

I just wanted to collect some more details on this concern.

I have so far been unable to reproduce the difference you showed in the video but I did note one thing in the video.
At the 4 minute mark, you copy the first set of results and make a change to the script. I see that you then re-run the test but I do not see where you have removed and re-added a new instance of the strategy between tests. Making changes to OnStateChange would generally require that you remove and re-add it. Could you try doing this and see if it makes a difference in your test?

Also if you have not yet tried to re-download the data, that may be another item to test. I tried both the date/instrument you used and the ES 03-18 for my test but have been unable to reproduce this.

I look forward to being of further assistance.

[Shansen]

Jesse, thanks for your help.

As requested, differences occur using a different Instrument (i.e. 6B 03-18), please find attached :

• a simplified example Strategy, StratB,
• a revised picture of the sequence of bar updates under AddDataSeries and AddDataSeries (with TradingHours), and
• an explanatory video (https://www.screencast.com/t/Mc7GdkYd6e).

Looking closer at the picture, on the left table, AddDataSeries, I've circled two rows BA[0][13] and BA[1][13] (again the "[13]" is always CurrentBars[0]), and on the right table I've circled the equivalent BarUpdate events (as evidenced by the "FirstTick" and "Completed" columns).

Firstly, focusing on the left table, please confirm my understanding of Working With Multi-Time Frame Objects. Where all BarArrays share the same exact timestamps (e.g. when using the same Instrument), BA[0] will always be processed first, followed by BA[1], etc.

• At row BA[0][13], the first tick of the 5min bar ended 4:20pm occurs (e.g. 4:15:01pm).
• At row BA[1][13], the same tick is processed for the 15min bar ended 4:30pm.
• Restated, at row BA[0][13], BA[1] has not processed the tick.
• So, why at row BA[0][13] has BA[1] progressed from the 15min bar ended 4:15pm to the bar ended 4:30pm, this will not occur until row BA[1][13].

Secondly, focusing on the right table:

• At row BA[0][13], the first tick of the 5min bar ended 4:20pm occurs, as BA[1] is yet to process the tick, BA[1][0] still shows as the 15min bar ended 4:15pm. This is in line with my understanding (as contrast to the left table).
• At row BA[1][15], this is purporting to be the FirstTick of the 15min bar ended 4:30pm (e.g. 4:15:01pm). However, as shown by the preceding rows BA[0][14] & BA[0][15], the 5min bars ended 4:20pm & 4:25pm have concluded (e.g. at 4:20:01pm & 4:25:01pm respectively). It is as though the FirstTick of BA[1] is just under 15mins late.

Thanks again.

[NinjaTrader_Jesse]

Hello,

Thank you for the post.

I will review what has been provided and reply back once I have further details.

I look forward to being of further assistance.

[Shansen]

Jesse, thanks for your help.

Testing with live data (Interactive Brokers), shows the same behaviour for AddDataSeries as shown using Market Replay (Historical) in prior posts. The question remains whether my understanding of Working With Multi-Time Frame Objects is correct.

I was originally unable to test AddDataSeries (with TradingHours) in a live environment, it is as though no BarUpdate events are triggered. Hence my inability to test. Please refer video at https://www.screencast.com/t/yaBuReuEIZLv.

This behaviour is similar to that resolved by PatrickH in NT forum post - Multiple AddDataSeries under Playback Connection. That issue revolved around adding multiple DataSeries (with TradingHours) where the added DataSeries TradingHours did not overlap. The Strategy we are working with here, StratB, differs in that we are adding a single DataSeries (with TradingHours) not the multiple DataSeries resolved by PatrickH.

After applying the same workaround (i.e. adding another DataSeries with an unused Instrument and "Default 24 x 7" TradingHours), tests on the DataSeries (with TradingHours) yielded the same behaviour as AddDataSeries (without TradingHours) as shown using Market Replay (Historical).

Any insight is welcome. Again, thanks for your help.

[Shansen]

Please provide an update on this query.
Is there anything further you require from my side?

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

At this time I don't have anything additional I could provide and do not need any further details. Once I have further information from testing what you have provided I will reply back here.

I look forward to being of further assistance.

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your patience.

I have created a video to detail this behavior further and explain why Real-Time data and Playback look different then what you would expect on Historical processing. You can view this video at the following link: https://patrickh-ninjatrader.tinytak...I3OV83MTgxMTQ4

To be clear the "Working with Multi-Time Frame Objects" documentation in the Help Guide demonstrates how Real-Time processing works on Tick based bars. Tick based bars have a time stamp of the current time where Minute based bars have the time stamp of when they close.

The code I used in my video is attached.

Please let me know if you have any questions.

[Shansen]

Patrick,

Thanks for weighing in on this issue.

There are two questions, both of which relate to Playback (Historical). For clarity, neither question relates to BarUpdate events under State.Historical. I believe the questions were best detailed in post #5 (02-03-2018, 02:34 PM). To summarise:

Question 1 pertains to both cases AddDataSeres and AddDataSeries (with TradingHours). When BA[0] is processing the tick (i.e. FirstTickOfBar) that will soon kick off processing BA[1] (i.e. FirstTickOfBar), the reference points for BA[1] appear to have progressed before BA[1] is processed.

Question 2 pertains to differing behaviour between AddDataSeries and AddDataSeries (with TradingHours).

Please find another explanatory video at https://www.screencast.com/t/qEdUQWuPdH4 and Indicator attached. Apologies, I was not able to demonstrate question 2 in this format. Please refer to post #5.

Thanks again.

[NinjaTrader_PatrickH]

Hello Shasen,

Thank you for your response.

As the Realtime or Playback data is coming in the same tick updates both bars arrays belonging to the same instrument. However the bar update for the primary series calls OnBarUpdate() first even though the secondary series has already updated as well. once the primary series bar is updated then the secondary series processes for OnBarUpdate and this continues for each tick.

On the last part of your video it seems you wish to explain something about the sequence with the trading hours template but I am not sure I understand as the updates are still seen in the same sequence without the trading hours template. Please elaborate here if you can.

I look forward to assisting you further.

[Shansen]

Regarding Question 1, I understand:

• As BA[0] and BA[1] are the same Instrument, in Playback (or Realtime) the same tick updates both BarArrays, and
• BA[0] is updated then BA[1] is updated. For clarity, at the smallest scale (e.g. one CPU cycle), BA[0] is updated first then BA[1]. At no time is it possible for BA[0] and BA[1] to be updated at the same time (i.e. concurrently). At no time is it possible for BA[1] to be updated first then BA[0].

These points are not in question.

In the scripts thus far, Strategy StratB and Indicator ExampleOrderOfUpdatesMultiTimeFrame, as part of the BA[0] BarUpdate, either:

• BA[1] is reported to have been already processed (as detailed above, this is not possible), or
• BA[1] is pointing to a barsAgo different to that detailed in Working With Multi-Time Frame Objects (i.e. Bar[1][-1] as opposed to Bar[1][0]).

For clarity, we are looking at BA[0] FirstTickOfBar in isolation, which is processed before BA[1] FirstTickOfBar. We are not looking at the immediately following BA[1] FirstTickOfBar.

If the NT8 HelpGuide is correct, and BA[0] is updated then BA[1], then at BA[0] FirstTickOfBar the just completed BA[0] is barsAgo[1] (i.e. BA[0][1]), the equivalent BA[1] is always barsAgo[0] (i.e. BA[1][0]). That is, BA[1][0] is still developing, it is yet to update. One CPU cycle later, at BA[1] FirstTickOfBar the equivalent, now just completed, bar for BA[1] will be BA[1][1].

The scripts thus far show at BA[0] FirstTickOfBar where BA[1] will subsequently become FirstTickOfBar, the just completed bar is BA[0][1] and equivalent bar is BA[1][1]. For all other BA[0] FirstTickOfBar, the just complete bar is BA[0][1] and equivalent bar is BA[1][0].

Again, please refer to post #5 for Question 2; the different behaviour between AddDataSeries and AddDataSeries (with TradingHours).

[Shansen]

Please find below and attached an example Strategy, StratC, which demonstrates the differing behaviour between:
- AddDataSeries,
- AddDataSeries (with TradingHours), and
- AddDataSeries (with TradingHours) plus AddDataSeries (with "Default" TradingHours and non-primary Instrument).

StratC differs to StratB. StratB evaluates all BarArrays (BAs) on the FirstTick of BA[0]. StratC evaluates all BAs on the final BA (i.e. BA[2]).

Please find an explanatory video at https://www.screencast.com/t/LAZBqA7qD

Again, thanks

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your patience.

I am looking into the Trading Hours template matter further with development. I also noticed a difference in bar updates between the Historical and Market Replay Playback options.

I will follow up when I have any additional information.