AddDataSeries and OnBarUpdate

[Wayne.Daigle]

Today I updated from an earlier Beta version to "8.0.0.13 64-bit (Standard) - Release Candidate 1" and my strategy stopped working when running in Market Replay (I don't know about live yet). I narrowed down the issue and wrote a test program.

So, first, with this code...

		protected override void OnBarUpdate()
		{
			Print("BarsInProgress " + BarsInProgress + " " + Time[0] + " " + State.ToString());
		}

on a 5-min chart, I get this:

BarsInProgress 0 8/17/2016 9:05:00 AM Historical
BarsInProgress 0 8/17/2016 9:10:00 AM Historical
Enabling NinjaScript strategy 'Testing/86292581' : On starting a real-time strategy - StartBehavior=WaitUntilFlat EntryHandling=All entries EntriesPerDirection=1 StopTargetHandling=Per entry execution ErrorHandling=Stop strategy, cancel orders, close positions ExitOnSessionClose=True / triggering 30 seconds before close SetOrderQuantityBy=Strategy ConnectionLossHandling=Keep running DisconnectDelaySeconds=10 CancelEntriesOnStrategyDisable=False CancelExitsOnStrategyDisable=True Calculate=On bar close IsUnmanaged=False MaxRestarts=4 in 5 minutes
BarsInProgress 0 8/17/2016 9:15:00 AM Realtime
BarsInProgress 0 8/17/2016 9:20:00 AM Realtime

Perfect. That's exactly as expected.

HOWEVER, if, in anticipation of using an indicator that requires a daily chart, I add ...

			else if (State == State.Configure)
			{
				AddDataSeries(BarsPeriodType.Day, 1);
			}

... to the OnStateChange, I get ...

BarsInProgress 0 8/17/2016 9:15:00 AM Historical
BarsInProgress 0 8/17/2016 9:20:00 AM Historical
Enabling NinjaScript strategy 'Testing/86292581' : On starting a real-time strategy - StartBehavior=WaitUntilFlat EntryHandling=All entries EntriesPerDirection=1 StopTargetHandling=Per entry execution ErrorHandling=Stop strategy, cancel orders, close positions ExitOnSessionClose=True / triggering 30 seconds before close SetOrderQuantityBy=Strategy ConnectionLossHandling=Keep running DisconnectDelaySeconds=10 CancelEntriesOnStrategyDisable=False CancelExitsOnStrategyDisable=True Calculate=On bar close IsUnmanaged=False MaxRestarts=4 in 5 minutes

So, the result is that OnBarUpdate is not being called for the 5-min chart when it goes to Realtime if I add in an AddDataSeries for a daily chart so that I can use an indicator that my strategy relies on. This worked fine in an earlier version of NT8, but not the latest. I have not been able to find a workaround.

Is there something I'm missing? Has anyone else seen this? Can anyone replicate this? Is this a known issue?

[Wayne.Daigle]

I discovered a workaround today.

If I have the main chart at 5 minutes and add a data series in the strategy of daily, as in my OP, it calls OnBarsUpdate once for BarsInProgress 0 and once for BarsInProgress 1 every day. Yes, the main 5-min chart is called once per day.

If I add a third data series to the strategy (specifically a 5-min series to reflect what is already in the chart itself), it calls BarsInProgress 0 AND BarsInProgress 2 every 5 minutes. And still BarsInProgress 1 every day.

In conclusion, I am finding that, in the latest version, the main chart is NOT driving OnBarUpdate in realtime unless it's the only one in the strategy. If there are other data series it will only "piggy-back".

[NinjaTrader_JessicaP]

Hello Wayne,

Thank you for your report. I am glad you were able to find a work-around that will help you. Would it be possible to provide us with two complete code samples that demonstrate this behavior? I will investigate this further and, should a bug with NinjaTrader be found that I can reproduce, I will inform the product management teams.

Thank you for using NinjaTrader 8

[hemlock]

I am finding that this is occurring for me as well. The lower timeframe OnBarUpdate are only coming in (if at all) when the OnBarUpdate is fired for the higher timeframe.

The output below comes from a YM 1 minute primary series and a YM 3 minute secondary series added via AddDataSeries.

The color groupings all come in together...

9/28/2016 3:22:28 PM kpTest1: STATE=SetDefaults
9/28/2016 3:22:28 PM kpTest1: STATE=Terminated
9/28/2016 3:22:28 PM kpTest1: STATE=Configure
9/28/2016 3:22:28 PM kpTest1: STATE=DataLoaded
9/28/2016 3:22:28 PM kpTest1: STATE=Historical
9/28/2016 3:22:28 PM kpTest1: STATE=Transition
9/28/2016 3:22:28 PM kpTest1: STATE=Realtime
9/28/2016 3:24:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:23:00 PM State=Realtime CurrentBar=5169 BarsInProgress=0
9/28/2016 3:24:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:24:00 PM State=Realtime CurrentBar=1786 BarsInProgress=1
9/28/2016 3:27:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:24:00 PM State=Realtime CurrentBar=5170 BarsInProgress=0
9/28/2016 3:27:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:25:00 PM State=Realtime CurrentBar=5171 BarsInProgress=0
9/28/2016 3:27:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:26:00 PM State=Realtime CurrentBar=5172 BarsInProgress=0
9/28/2016 3:27:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:27:00 PM State=Realtime CurrentBar=5173 BarsInProgress=0
9/28/2016 3:27:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:27:00 PM State=Realtime CurrentBar=1787 BarsInProgress=1
9/28/2016 3:30:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:28:00 PM State=Realtime CurrentBar=5174 BarsInProgress=0
9/28/2016 3:30:01 PM kpTest1: OnBarUpdate Time=9/28/2016 3:30:00 PM State=Realtime CurrentBar=1788 BarsInProgress=1
9/28/2016 3:33:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:29:00 PM State=Realtime CurrentBar=5175 BarsInProgress=0
9/28/2016 3:33:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:30:00 PM State=Realtime CurrentBar=5176 BarsInProgress=0
9/28/2016 3:33:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:31:00 PM State=Realtime CurrentBar=5177 BarsInProgress=0
9/28/2016 3:33:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:32:00 PM State=Realtime CurrentBar=5178 BarsInProgress=0
9/28/2016 3:33:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:33:00 PM State=Realtime CurrentBar=5179 BarsInProgress=0
9/28/2016 3:33:02 PM kpTest1: OnBarUpdate Time=9/28/2016 3:33:00 PM State=Realtime CurrentBar=1789 BarsInProgress=1

[COLOR=#0000ff]protected[/COLOR] [COLOR=#0000ff]override[/COLOR] [COLOR=#0000ff]void[/COLOR] [COLOR=#080808]OnStateChange[/COLOR]()
{
[COLOR=#0000ff]if[/COLOR] ([COLOR=#080808]State[/COLOR] == [COLOR=#080808]State[/COLOR].[COLOR=#080808]SetDefaults[/COLOR])
{
[COLOR=#080808]Description[/COLOR] = [COLOR=#b22222]@"Enter the description for your new custom Indicator here."[/COLOR];
[COLOR=#080808]Name[/COLOR] = [COLOR=#b22222]"kpTest1"[/COLOR];
[COLOR=#080808]Calculate[/COLOR] = [COLOR=#080808]Calculate[/COLOR].[COLOR=#080808]OnBarClose[/COLOR];
[COLOR=#080808]IsOverlay[/COLOR] = [COLOR=#0000ff]true[/COLOR];
[COLOR=#080808]DisplayInDataBox[/COLOR] = [COLOR=#0000ff]true[/COLOR];
[COLOR=#080808]DrawOnPricePanel[/COLOR] = [COLOR=#0000ff]true[/COLOR];
[COLOR=#080808]DrawHorizontalGridLines[/COLOR] = [COLOR=#0000ff]true[/COLOR];
[COLOR=#080808]DrawVerticalGridLines[/COLOR] = [COLOR=#0000ff]true[/COLOR];
[COLOR=#080808]PaintPriceMarkers[/COLOR] = [COLOR=#0000ff]true[/COLOR];
[COLOR=#080808]ScaleJustification[/COLOR] = [COLOR=#080808]NinjaTrader[/COLOR].[COLOR=#080808]Gui[/COLOR].[COLOR=#080808]Chart[/COLOR].[COLOR=#080808]ScaleJustification[/COLOR].[COLOR=#080808]Right[/COLOR];
[COLOR=#006400]//Disable this property if your indicator requires custom values that cumulate with each new market data event. [/COLOR]
[COLOR=#006400]//See Help Guide for additional information.[/COLOR]
[COLOR=#080808]IsSuspendedWhileInactive[/COLOR] = [COLOR=#0000ff]true[/COLOR];
}
[COLOR=#0000ff]else[/COLOR] [COLOR=#0000ff]if[/COLOR] ([COLOR=#080808]State[/COLOR] == [COLOR=#080808]State[/COLOR].[COLOR=#080808]Configure[/COLOR])
{
[COLOR=#006400]//AddDataSeries(Data.BarsPeriodType.Day, 1);[/COLOR]
[COLOR=#080808]AddDataSeries[/COLOR]([COLOR=#080808]Data[/COLOR].[COLOR=#080808]BarsPeriodType[/COLOR].[COLOR=#080808]Minute[/COLOR], [COLOR=#ff8c00]3[/COLOR]); 
}
[COLOR=#080808]Print1[/COLOR]([COLOR=#0000ff]string[/COLOR].[COLOR=#080808]Format[/COLOR]([COLOR=#b22222]"STATE={0}"[/COLOR], [COLOR=#080808]State[/COLOR]));
}
[COLOR=#0000ff]private[/COLOR] [COLOR=#0000ff]void[/COLOR] [COLOR=#080808]Print1[/COLOR]([COLOR=#0000ff]string[/COLOR] [COLOR=#080808]msg[/COLOR])
{
[COLOR=#080808]PrintTo[/COLOR] = [COLOR=#080808]PrintTo[/COLOR].[COLOR=#080808]OutputTab1[/COLOR]; 
[COLOR=#080808]Print[/COLOR]([COLOR=#0000ff]string[/COLOR].[COLOR=#080808]Format[/COLOR]([COLOR=#b22222]"{0} {1}: {2}"[/COLOR], [COLOR=#080808]DateTime[/COLOR].[COLOR=#080808]Now[/COLOR], [COLOR=#0000ff]this[/COLOR].[COLOR=#080808]Name[/COLOR], [COLOR=#080808]msg[/COLOR]));
} 
 
[COLOR=#0000ff]protected[/COLOR] [COLOR=#0000ff]override[/COLOR] [COLOR=#0000ff]void[/COLOR] [COLOR=#080808]OnBarUpdate[/COLOR]()
{
[COLOR=#0000ff]if[/COLOR] ([COLOR=#080808]State[/COLOR] == [COLOR=#080808]State[/COLOR].[COLOR=#080808]Realtime[/COLOR]) [COLOR=#080808]Print1[/COLOR]([COLOR=#0000ff]string[/COLOR].[COLOR=#080808]Format[/COLOR]([COLOR=#b22222]"OnBarUpdate Time={0} State={1} CurrentBar={2} BarsInProgress={3}"[/COLOR], [COLOR=#080808]Time[/COLOR][[COLOR=#ff8c00]0[/COLOR]], [COLOR=#080808]State[/COLOR], [COLOR=#080808]CurrentBar[/COLOR], [COLOR=#080808]BarsInProgress[/COLOR])); 
}

[NinjaTrader_JessicaP]

Thank you both for your reports.

The attached script will not demonstrate the behavior you have observed on my system. Could I ask that you test this on your systems? If it generates the same behavior you are reporting, no further action will be needed, and you can expect this to be resolved with the next release of NinjaTrader.

If, however, it also does not demonstrate the behavior you are observing on your end, would it be possible for you to modify it and return it to me so that it does?

When this script is run with a 1 tick data series on my system, it will print out once per tick, rather than collecting all the output until the end of the minute and printing it all out at once.

[Wayne.Daigle]

Looks like you fixed it on your system.

On mine, with a 1-tick market replay it produces this after many ticks go by:

 
BarsInProgress: 0
CurrentBar: 452272
Time: 8/19/2016 8:15:00 AM
BarsInProgress: 1
CurrentBar: 5825
Time: 8/19/2016 8:16:00 AM

It sits there doing nothing and then produces this 1 minute later:

 
...
BarsInProgress: 0
CurrentBar: 452410
Time: 8/19/2016 8:16:53 AM
BarsInProgress: 0
CurrentBar: 452411
Time: 8/19/2016 8:16:53 AM
BarsInProgress: 0
CurrentBar: 452412
Time: 8/19/2016 8:16:58 AM
BarsInProgress: 1
CurrentBar: 5826
Time: 8/19/2016 8:17:00 AM

Note: The "..." at the top represents a ton of entries that scrolled off the top of the output buffer, one for each tick.

Also, thank you to hemlock for his contribution.

[hemlock]

Jessica,

Thanks for sending out the script. I can confirm as well that the script you supplied demonstrates the same behavior as previously reported.

What is the timeframe for pushing out this update?

Thanks again.

[NinjaTrader_JessicaP]

Thank you for running that test on your end hemlock. While we do not have an exact release timetable we can keep an eye on the NinjaTrader 8 Release Notes page for updates and bugfixes.

http://ninjatrader.com/support/helpGuides/nt8/en-us/release_notes.htm

Thank you all for using NinjaTrader 8 and for all of your feedback.

[hemlock]

OK. To me this seems like a significant issue for anyone developing multi-timeframe strategies and should be pushed out as soon as you can.

Although it does appear if the primary series is the higher timeframe and the secondary series is a lower timeframe the OnBarUpdate work correctly.

Also can you tell me if this issue is related to another that I posted here: http://ninjatrader.com/support/forum...ad.php?t=89976

Thanks.

[NinjaTrader_JessicaP]

I believe they may be related. I will add a link to this thread from that thread. I'm posting the output I received after running the script attached to this thread, so that the community can see how the most recent version of Ninja is performing. BarsInProgress == 0 is on a 1 tick series and BarsInProgress == 1 is on a 1 minute series.

CurrentBar: 1353031
Time: 29-Sep-16 07:42:58
BarsInProgress: 0
CurrentBar: 1353032
Time: 29-Sep-16 07:42:58
BarsInProgress: 0
CurrentBar: 1353033
Time: 29-Sep-16 07:42:58
BarsInProgress: 0
CurrentBar: 1353034
Time: 29-Sep-16 07:42:59
BarsInProgress: 1
CurrentBar: 4544
Time: 29-Sep-16 07:43:00
BarsInProgress: 0
CurrentBar: 1353035
Time: 29-Sep-16 07:43:00
BarsInProgress: 0
CurrentBar: 1353036
Time: 29-Sep-16 07:43:00
BarsInProgress: 0
CurrentBar: 1353037
Time: 29-Sep-16 07:43:01
BarsInProgress: 0

[Assiduous]

Different Daily Highs

In NT8 RC1,

Using instrument CL 11-16, on a 15 minute chart, with days to load at 20, and trading hours set to <Use instrument settings>, I added a custom indicator with the following code.....

if (State == State.Configure)
	{
		AddDataSeries(Instrument.FullName, new BarsPeriod {BarsPeriodType = BarsPeriodType.Day, Value = 1},"Nymex Energy RTH");
		AddDataSeries(Instrument.FullName, new BarsPeriod {BarsPeriodType = BarsPeriodType.Minute, Value = 330},"Nymex Energy RTH");
	}
		
protected override void OnBarUpdate()
{
     if (BarsInProgress == 1)
	{
        Print(TradingHours.GetPreviousTradingDayEnd(Time[0]));	
	Print(Highs[1][0]);	
	}

     if (BarsInProgress == 2)
	{
	Print(TradingHours.GetPreviousTradingDayEnd(Time[0]));	
	Print(Highs[2][0]);	
	}			
}

I get two identical sets of PreviousTradingDayEnd Times. So, in theory, I should get two identical sets of daily highs but for some reason I do not. The tertiary data series (i.e. BarsArray[2]) prints the correct session highs. Whereas the secondary data series prints a few correct session highs and other highs from random times. Not sure what to think. Any ideas?

PS
My data provider is IQFeed if that makes any difference.

[NinjaTrader_JessicaP]

Hello Assiduous, and thank you for your question.

NinjaTrader takes in two kinds of data : data that is updated on price changes, and daily data. Since Minute data is more granular, Ninja has enough to work with to filter the data it displays, so that you can look at OHLC data between a specified start and stop time. Traditionally the term for this open and close time set is "session template".

In your case, if you would like to compare daily data to daily data, I would recommend creating a 1440 minute chart, as there are 1440 minutes in a day, rather than a 330 minute chart as you have done. You can experiment to verify that the session template you choose for your daily bars does nothing, so this argument can be omitted. Finally, you will want to experiment with different session templates for your 1440 minute bars, until you find the one that matches IQFeed's daily bars for your primary instrument.

Please do note that in the documentation for Instrument, it does suggest the following

Warning: The properties in this class should NOT be accessed within the OnStateChange() method before the State has reached State.DataLoaded

With this in mind I would recommend switching the overload you are using so that you are omitting the first argument to AddDataSeries. I am including the available overloads from the help guide as well.

The following syntax will add another Bars object for the primary instrument of the script.
AddDataSeries(BarsPeriodbarsPeriod)
AddDataSeries(BarsPeriodTypeperiodType,intperiod)


The following syntax allows you to add another Bars object for a different instrument to the script:
AddDataSeries(stringinstrumentName,BarsPeriodTypeperiodType,intperiod)
AddDataSeries(stringinstrumentName,BarsPeriodTypeperiodType,intperiod,MarketDataTypemarketDataType)
AddDataSeries(stringinstrumentName,BarsPeriodbarsPeriod)
AddDataSeries(stringinstrumentName,BarsPeriodbarsPeriod,stringtradingHoursName)
AddDataSeries(stringinstrumentName,BarsPeriodbarsPeriod,stringtradingHoursName,bool?isResetOnNewTradingDay)
AddDataSeries(stringinstrumentName,BarsPeriodbarsPeriod,intbarsToLoad,stringtradingHoursName,bool?isResetOnNewTradingDay)

While I believe you were using a longer overload in order to be able to specify the tradingHoursName, please note that if you are instantiating this from a chart, you can set the trading hours (or "session template") from the data series window, which you can bring up with ctrl + f .

[Assiduous]

Hi Jessica, and thank you for the prompt reply.

While I understand your explanation (I appreciate the thoroughness BTW), I think we may be talking about two different things.

My goal is to create virtual daily bars where the hours of the Nymex Energy RTH (0900 - 1430 Eastern Time) equals 1 day. This is why I added the data series that creates 330min bars. This data series correctly provides me with the highs that occurred during that timeframe.

When adding the dataseries with the BarsPeriodType.Day I assumed that it would behave the same way when I specify the trading hours within the overload. However, I get the highs from the default session (1800 - 1700 Eastern Time). These same highs print no matter what the trading hours are set to within the chart. I also tried setting the isResetOnNewTradingDay to true but I still get the same highs from the default trading hours.

Initially, this was all confusing to me since Print(TradingHours.GetPreviousTradingDayEnd(Time[0])); printed the correct time of 14:30 Eastern Time. However, at this point, I have simply resigned to the fact that BarsPeriodType.Day does not behave the way I assumed.

[NinjaTrader_JessicaP]

Thank you for the additional explanation. We are indeed unable to apply trading hours to daily bars. Please let us know if there are any other ways we can help.