Issue - Multiple AddDataSeries under Playback Connection

[Shansen]

NT Team,

The (updated) attached Strategy, TestAddDataSeries, shows an issue where the Playback Connection (historical) breaks where DataSeries are added with different TradingHoursNames.

TestAddDataSeries has two tests:
- TestOne - two DataSeries are added with TradingHours of the same time zone, "US Equities RTH" and "Nymex Energy RTH" share same time zone "(UTC-5:00) Eastern Time (US & Canada)". Functions as expected.
- TestTwo - two DataSeries are added with TradingHours of different time zones "US Equities RTH" (UTC-5:00) and "Snfe RTH" (UTC+10:00). Does not function as expected.

Please view a brief video at https://www.screencast.com/t/XAffpe4JW8j

Any advice would be greatly appreciated.

Thanks
Shannon

[NinjaTrader_ChelseaB]

Hello Shannon,

The script has:
Instrument.FullName called in State.Configure.

The instrument will not have a value when optimized when State.Configure is called.

From the help guide:
"Arguments supplied to AddDataSeries() should be hardcoded and NOT dependent on run-time variables which cannot be reliably obtained during State.Configure (e.g., Instrument, Bars, or user input). Attempting to add a data series dynamically is NOT guaranteed and therefore should be avoided. Trying to load bars dynamically may result in an error similar to: Unable to load bars series. Your NinjaScript may be trying to use an additional data series dynamically in an unsupported manner."

Below is a publicly available link to the help guide.

https://ninjatrader.com/support/helpGuides/nt8/en-us/adddataseries.htm

Are you able to reproduce this behavior without using the Instrument object in State.Configure?

[Shansen]

I confirm the issue is present without using the Instrument object in State.Configure.
i.e. "instrumentFullName" was hard coded to "6A 02-18" (or similar).

[NinjaTrader_ChelseaB]

Hello Shannon,

When hard coding "6A 03-18" and ensuring I have both minute data and Market Replay data for the entire period for both instruments I am not able to reproduce this behavior using 8.0.11.1.

Below is a link to a video that demonstrates the test.

https://drive.google.com/file/d/1Boih-afHc0U3n3o2--UC3ghV2Yu7KyFK/view

May I confirm I have taken the correct actions to test?


This strategy cannot be optimized, there are no inputs to optimize over.

Are you certain the script you posted is the script you are optimizing in the Strategy Analyzer?

[Shansen]

Thank you for testing. It appeared as though you ran the test on Playback (Market Replay). My test ran on Playback (Historical). The DataSeries was based on imported tick data (Second Granularity) (Data type = Last).

You are correct the strategy cannot be optimized. I am not attempting to optimize the script. The script is purely to explain unexpected behaviour I am seeing.

Please confirm the test you ran was using Playback (Historical)

[NinjaTrader_ChelseaB]

Hello Shannon,

Thank you for checking that and letting me know I need to re-test using historical tick data.

I've retested but I am not able to reproduce. Both methods produce the same output for the primary series.

https://drive.google.com/file/d/1PDzO8mHLGAGIt2Yg7jm_M2g2VazcGJyg/view

Do you have tick data for the entire period you are testing over for all series?

[Shansen]

Thanks for your ongoing help.

To (hopefully) simplify the issue, I've run the tests using Playback (Market Replay), similar to your first tests. The issue presents using Playback (Market Replay) as it does in Playback (Historical).

For the cleanest test;
- The instrument (i.e. 6A 03-18) had no prior data
- Market Replay data was downloaded from NT servers
- The Strategy TestAddDataSeries was run using Playback (Market Replay)
- The Primary DataSeries and added DataSeries all use the same instrument (i.e. "6A 03-18)

As the issue presents in Playback (Market Replay), I hope a few possible causes have been knocked out of the equation. Grasping at straws, could my Local PC time and NT timezone settings cause the issue?

Please refer to the video at:
https://www.screencast.com/t/0Q7bZYqP

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your patience.

The reason it worked for Chelsea is that he was testing the ES 03-18 on the chart yet in code was calling the 6A 03-18. The reason it did not work for you is you are using the same instrument on the chart as in code.

There is a limitation in AddDataSeries() that after the first call that shares the same instrument as the next call the same Bars To Load and Trading Hours Template is used.

If, for example, the first call is the Snfe RTH Trading Hours Template then this behavior is not seen.

Please let me know if you have any questions.

[Shansen]

Thank you for investigating further.

Please help me better understand this limitation. Please find below two videos.

The first shows an indicator "TradingHours" using Historical Data. The indicator appears to successfully use the same instrument for BA[0], BA[1] & BA[2] each with different Trading Hours. (https://www.screencast.com/t/m5tgvb3tX)

The second shows the strategy "TestAddDataSeries" using Market Reply. The strategy behaves very differently under four scenarios, where primary DataSeries is always 6A (CME FX Futures ETH) and:

1. Same Instrument, BA[1] 6A(US) & BA[2] 6A(Nymex) - Successful, BA[1] & [2] start at 2am, though BA[0], [1] & [2] conclude at 6am (expected BA[0] 4pm, BA[1] 8am, & BA[2] 6am). If BA[2] was added before BA[0] & [1], this supports the thesis "next call the same Bars To Load and Trading Hours Template is used".
2. Same Instrument, BA[1] 6A(US) & BA[2] 6A(Snfe) - No Output, as though OnBarUpdate is never triggered for BA[0], BA[1], or BA[2].
3. Different Instrument, BA[1] 6B(US) & BA[2] 6B(Nymex) - Successful, BA[1] & [2] start at 2am, BA[1] concludes at 8am, BA[2] concludes at 6am, and BA[0] concludes at 4pm (as expected in test scenario 1).
4. Different Instrument, BA[1] 6B(US) & BA[2] 6B(Snfe) - Successful, BA[2] starts after 10am.

(https://www.screencast.com/t/TJdDpQ5ffII)

As the indicator TradingHours appears to work as expected, it does not appear to be affected by the limitation "after the first call that shares the same instrument as the next call the same Bars To Load and Trading Hours Template is used". Have I missed something?

If I understand correctly, the limitation should apply "when adding multiple Data Series of the same instrument and the same Bar Type..." (AddDataSeries)

The current syntax used is:

AddDataSeries(instrumentName, new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, tradingHoursName);

Where the BarsPeriod differs to all other DataSeries is the issue avoided? For example, where the Value (1) is altered to differ to all other DataSeries?

Again, thank you for your help.

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your patience.

I was incorrect in stating this behavior is due to the AddDataSeries() limitation. These are unrelated.

This appears to be related to the Time Zone of the Trading Hours Template and not your PC as I can reproduce the same behavior on my PC in the MST Time Zone.

I will follow up with you when I have further details.

[Shansen]

Please provide an update on this issue.

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your post.

I will provide an update on this matter when I have further information.

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your patience.

This is a limitation when the Sessions do not overlap. You can work around this by adding an unused Bar Series to the script that uses the 'Default 24/7' Trading Hours template.

Please let me know if you have any questions.

[Shansen]

Patrick,

I may have misunderstood your response. Please find below my understanding of the reworked AddDataSeries, including an "unused" Instrument with "Default 24 x 7" Trading Hours.

else if (State == State.Configure)
{
  var instrumentFullName = "6A 03-18";
  var tradingHoursName1 = "US Equities RTH";
  var tradingHoursName2 = "ASX Equities RTH";
  AddDataSeries(instrumentFullName, new BarsPeriod {BarsPeriodType = BarsPeriodType.Minute, Value = 1,}, tradingHoursName1);
  AddDataSeries(instrumentFullName, new BarsPeriod {BarsPeriodType = BarsPeriodType.Minute, Value = 1,}, tradingHoursName2);

  var unusedInstrument = "6B 03-18";
  var defaultTradingHours = "Default 24 x 7";
  AddDataSeries(unusedInstrument, new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1, }, defaultTradingHours);
}

If I have implemented correctly, the work around does not appear to work (as shown in https://www.screencast.com/t/enQQe9APwl)

My sole purpose of adding DataSeries with specified TradingHours, is to gauge when the specified markets are open.

For example, the Instrument 6A 03-18 (Australian Dollar Future) is most liquid during US cash market hours (i.e. "US Equities RTH") and Australian cash market hours (i.e. "ASX Equities RTH"). To gauge when the US and ASX markets are open, DataSeries are added with appropriate TradingHours. Two DataSeries are required to account for changes in daylight savings locally in the US and in Australia.

How can this goal be achieved?

Again, thanks

[NinjaTrader_PatrickH]

Hello Shansen,

Thank you for your response.

The work around I posted will not work. One of the series you use must intersect and therefore even adding a 'Default 24/7' Trading Hours Template to a series that is not used will not help. One of the series used must have it's Start or End Time intersect with the other series Start and End Time for it's Trading Hours Template.

As far as what is meant by "not intersecting",

Snfe RTH runs weekdays between 09:50 and 16:30 AEST/AEDT (Australian Eastern Standard / Daylight Time)

• 09:50 - 16:30 in AEDT is 22:50 - 05:30 UTC (Coordinated Universal Time)
• 09:50 - 16:30 in AEST is 21:50 - 04:30 UTC

CME US Index Futures RTH runs weekdays between 08:30 and 15:15 CST/CDT (Central United States Standard / Daylight Time)

• 08:30 - 15:15 in CST is 14:30 - 21:15 UTC
• 08:30 - 15:15 in CDT is 15:30 - 22:15 UTC

There is no overlap when Australia is in Daylight Savings and the US is not (October - March), because 2:30 PM - 9:15 PM and 10:50 PM - 5:30 AM UTC include no times in common

• However when the US is in Daylight Savings and Australia is not (April - September), 3:30 PM - 10:15 PM and 9:15 PM - 4:30 AM overlap between 9:15 PM and 10:15 PM

Please let me know if you have any questions.