Bug in MultiDataSeries Logic

[Harry]

NinjaTrader 7.0.1000.2, Local Timezone CET

NinjaTrader does not correctly add a daily dataseries to an intraday series, when used with a multi-session template. Please have a look at the following simple indicator, which adds a daily data series. Now when used with my default session template for ES, this daily data series is called at the wrong moment. Actually today's close is already inserted at the beginning of today's session. Predicting the future with hindsight? Session template, indicator and chart are attached. Chart uses this session template as per instrument settings.

Indicator Code:

protected override void Initialize()
{
              Add(new Plot(Color.Orange, PlotStyle.Line, "Yesterday's Close"));
	Add(PeriodType.Day,1);           
	Overlay = true;
}

protected override void OnBarUpdate()
{
	if(BarsInProgress == 0)
	{
		if (!init)
			Value.Set(Close[CurrentBar]);
		else
			Value.Set(close);
	}
	if (BarsInProgress == 1)
	{
		init = true;
		close = Close[0];
	}
}

[Harry]

Master Data Problem

I think that this problem is related to a conceptual weakness of the master data structure used for the session template.

If you use a session template with several sub-sessions, there is no field to indicate, which of the sub-sessions is the first one. So NinjaTrader probably uses a heuristic to determine the likeliness that the trading day starts with the session X.

However, I have seen a number of failures of this heuristic.

-> For some session templates, NinjaTrader starts the trading day at a false session break
-> As shown below NinjaTrader cannot select the right session break to add daily data

This can only be cured by a modification of the structure of the master data for the session template. Your competitors have done this, and you could for example have a look at the solution adapted by MultiCharts.

Of course I can code a workaround (filter) to avoid that the prior day's close is displayed early, but this does not solve the underlying problem.

[NinjaTrader_RyanM1]

Thank you for the report, Harry. Will look into and update you tomorrow, Thursday.

[NinjaTrader_RyanM1]

Development is currently looking into this and we will update this thread as soon as we have more information.

[Harry]

Thank you for the feedback, Ryan.

Development is currently looking into this and we will update this thread as soon as we have more information.

[NinjaTrader_JoshP]

Harry,

After looking into it, basically this is unfortunately a limitation when applying a different session template onto the daily bars than the session template it was made from (CME US Index ETH). The issue is sometimes seen when you use a session template like yours with multiple segments for a single trading day.

[Harry]

False day break selected with ES session template

Hi Josh,

I am fully aware of the problem. In my opinion it is not a limitation, but a flaw in the design of session templates, which needs to reviewed.

For example, if I select the natural segmented session template for ES, with the following intraday sessions

night session prior to break -> 3:30 PM to 4:30 PM CT
night session after break -> 5:00 PM to 8:30 AM CT
day session (floor session) -> 8:30 AM to 3:15 PM

NinjaTrader methods will put the start of the trading day right in the middle of the night session. This is a major flaw, as it means that NinjaTrader 7.0 cannot be used reliably with segmented session templates.

I have attached the template and the chart that shows how NT selects the false start of the trading day.

Now the same design flaw causes NinjaTrader to add daily data in the middle of the day and not at the day break when it should do.

I am not really satisfied with the answer that this is a limitation. Because this limitation means that Ninjatrader does not correctly work with multiple intraday sessions.

It is not a limitation, but rather a bug.

[NinjaTrader_JoshP]

Harry,

This is a limitation for many under the hood and technical reasons. I will leave it at that as it is not addressable at this point in time, but is on our list for future versions to remove the limitation.

[Harry]

Thanks for the reply Josh. That leaves some hope that there will be a better future - and I am not saying that things are bad at present.

Harry,

This is a limitation for many under the hood and technical reasons. I will leave it at that as it is not addressable at this point in time, but is on our list for future versions to remove the limitation.

[Harry]

It is a heavy bug and much worse than assumed.

This NinjaTrader bug is much worse than I have ever assumed.

(1) All daily bars have false time stamps.

(2) When adding a daily data series to an intraday data series, you will always get false results.

(3) You cannot code any multitimeframe indicators using intraday and daily data.

(4) You cannot code any strategies using intraday and daily data.

This significantly reduces the usability of NinjaTrader for multi time series charting.

I will now explain the above four points.

[Harry]

Example

The chart below shows a 60 min and daily data series of CL 04-11.

(1) The daily bars are time-stamped 9:30 and appear in the middle of the day. This does not make sense. DailyBars should use the template as per instrument settings and should be inserted at the end of the trading day. This requires that per master data it is possible to determine the end of the trading day.

(2) If I apply a daily SMA to the 60 min chart, this produces nonsense. The SMA is set to CalcOnBarClose = false , so it should display something for the current day, and I have set it to Square on purpose to show where the data points are inserted.

This does not look good.

-> Data points are inserted in the middle of the day due to false time stamps for daily bar, they should be inserted at the end of the ETH session - note that I have used a simple session template here, not my habitual 3-session template

-> Values are displayed with hindsight knowledge: the daily SMA(2) is set to CalcOnBarClose= false to display all values, also I use Square style to show the insertion of data points. The SMA(2) is already displayed before the close i s known, as its value is inserted too early.

-> No value is displayed for the current day where you need it most.

[Harry]

Intraday bar series can be added, but not daily bar series

I do not want to show it in detail here, but for the same reasons, if I code a multi time frame indicator that uses several dataseries, this works well as long as I add

- range bars
- tick bars
- volume bars
- minute bars
- second bars

but it is an absolute failure if I try to add daily bars in a synchronous way.

This is a problem as the asynchronous loading which I use for pivot and fibonacci indicators does not allow to expose any values to transfer them to other indicators

-> asynchronous loading of daily data on intraday charts does not work as expected
-> synchronous loading of daily data produces false values

Currently there is no way out of this.

[NinjaTrader_RyanM1]

Thank you for the reply, Harry. We are currently looking into matters here.

[zweistein]

@Josh,

basically the problem reduces to the globex futures session templates.
At this point you could just hard code a fix in NT7 that sets the session start for the day bars at the floor session.
Just put these 10-30 symbols from cmegroup.com in a list and act accordingly in your NT7 software. Takes 2 hours for someone proficient in C#.

regards
Andreas

Hi Josh,

I am fully aware of the problem. In my opinion it is not a limitation, but a flaw in the design of session templates, which needs to reviewed.

For example, if I select the natural segmented session template for ES, with the following intraday sessions

night session prior to break -> 3:30 PM to 4:30 PM CT
night session after break -> 5:00 PM to 8:30 AM CT
day session (floor session) -> 8:30 AM to 3:15 PM

NinjaTrader methods will put the start of the trading day right in the middle of the night session. This is a major flaw, as it means that NinjaTrader 7.0 cannot be used reliably with segmented session templates.

I have attached the template and the chart that shows how NT selects the false start of the trading day.

Now the same design flaw causes NinjaTrader to add daily data in the middle of the day and not at the day break when it should do.

I am not really satisfied with the answer that this is a limitation. Because this limitation means that Ninjatrader does not correctly work with multiple intraday sessions.

It is not a limitation, but rather a bug.

[Harry]

The problem is structural. Hard coding any symbols would aggravate the problem.

Andreas, I think that you have not understood the session logic at all. Please review.

You need to differentiate between the trading day and the sessions. You cannot hard code any session because it should be user selectable. The trading day can start with any of the sessions. This needs to be entered into the session template, where it currently cannot be entered.

Next the session template should also be applied to dailybars. The appropriate session template for daily bars depends on the data that is used. I used Kinetick data. The daily Kinetick bars cover different periods depending on the instrument. For CME traded index futures the daily bars cover the session from 15:30 CT (exception Sunday 17:00 CT) to 15:15 CT next day. Therefore the Kinetick daily bars require a time-stamp of 15:15 CT, which allows to insert them correctly at the end of the session.

Problem is that CL would require a time stamp of 17:15 EST, FDAX requires a timestamp of 22:00 CET and so on.

If you have a data provider who supplies RTH data, you would require a time-stamp at the end of the RTH session.

The whole thing has a logic, and if you do not violate it, it will produce consistent results.

@Josh,

basically the problem reduces to the globex futures session templates.
At this point you could just hard code a fix in NT7 that sets the session start for the day bars at the floor session.
Just put these 10-30 symbols from cmegroup.com in a list and act accordingly in your NT7 software. Takes 2 hours for someone proficient in C#.

regards
Andreas