Multi-Series Indicators and Data Loaded

[ntbone]

If you have a indicator with multiple data series, all the data for that series will be loaded regardless of the data loaded for the primary data series.

Consider the following.

AddDataSeries(BarsPeriodType.Second, 30);
AddDataSeries(BarsPeriodType.Minute, 1);
AddDataSeries(BarsPeriodType.Day, 1);

Now suppose for AAPL the data available is

• 1 month of tick data
• 2 months of minute data
• 1 year of daily data

If a one minute chart is loaded using a date range of 4 months

• The 30 second series will get 1 month of data
• The 1 minute series will get 2 months of data
• The 1 day series will get 4 months of data.

OnBarUpdate will be called first for daily data since its the oldest, then it will be called for minute data, and finally for the tick data.

[NinjaTrader_ChelseaB]

Hello ntbone,

I am not able to reproduce this behavior.

Attached is a simple script that loads, tested on a much smaller amount of data, so that it will load quickly for a test.

A second, minute, and day series is added.

These all start on the same day. With the 30 second ending 30 seconds after the start of the session, the 1 minute ending 1 minute after the start of the session, and the day ending 1 day after the start of the session.

https://drive.google.com/file/d/1lMSpSLMcTpIOUWu749C-tuj1vtZyatmR/view?usp=drivesdk

[ntbone]

When I tested it, I cleared my data, made sure to download

• 4 months of daily data
• 2 months of minute data
• 1 month of tick data.

After the data was downloaded I disconnected the data source to make sure there was no way to fill in the missing data. I then set the date range for my chart to be 4 months. I wanted to simulate what would happen if I specified a date range longer then what my data connection provides to see how the data is handled.

I can see in your video you are connected to a data provider, so unless you go far enough back to exceed the first bar of both the tick and minute data, you would not be able to see this particular issue. You would also need to clear the respective folders in the Db or create a fresh copy of the NinjaTrader folder in documents to simulate an account with no existing data.

Here is the code I used to test it

protected override void OnBarUpdate()
{
    if(CurrentBars[BarsInProgress] == 0)
    {
        Print(string.Format("The first bar for {0} is {1}", BarsInProgress, Times[BarsInProgress][0]));
    }
}

which makes it very easy to see what the time stamps for the first bars loaded.

[NinjaTrader_ChelseaB]

Hello ntbone,

I want to be clear what the issue is. Are you inquiring about data that showing as available is in the Tools -> Historical Data window that is not loading within the range of the Data Series of the chart?

My historical data was empty before creating the video.

To confirm, there is no issue over 1 week. Exactly 4 months data has to be loaded and no more and no less, is this correct? The behavior cannot be reproduced with 1 month of data, or 3 months and 3 weeks of data?

Further, the test I have provided has the same results on your end, is this correct?

[ntbone]

I was never reporting an issue. I was sharing information with others about how data is loaded when there is more data on larger time frames available then on the smaller time frames.

If I open a chart and set the time range to exceed the available data for the primary data series, it will still load the data for the additional data series, up to the time range specified. In my example I had only had only 2 months of minute data and 1 month of tick data. I made a chart requesting 4 months of data. Each data series attempted to load 4 months of data, but since there is no minute data past 2 months, the primary data series only gets 2 months of data, but the daily data series will get 4 months of data. Since there is only 1 month of data available for tick data, the 15 second data only had one month of data.

I wanted to understand the behavior because I was concerned that since there was only 2 months of minute data for the primary data series, then it would only load 2 months of daily data despite the chart range being 4 months.

My take away is, regardless of whether its the primary data series, or the additional data series, NinjaTrader will load as much data as is available in the requested range for the chart. This may also mean that many bars may be processed on additional data series before a bar is processed on the primary data series.

[NinjaTrader_ChelseaB]

Hello ntbone,

Yes, NinjaTrader will load the data for the number of days set in the Data Series window for each series.

[ntbone]

Thanks. In this case I am not using the Data Series Window to add more then one Data Series. I am using the AddDataSeries call for the indicator.

This brings up a separate question though. If I had 2 data series through the Data Series window, can indicators see both data series or only the one specified via the "Input Series" parameter?

[ntbone]

Further testing reveals something rather interesting about AddDataSeries and the "Data Series" window.

1st, regardless of how many Data Series you add in the Data Series window, the indicator will only see its primary data series unless it explicitly adds additional ones in code.
2nd, you can control the range of additional data series for that indicator by adding additional data series in the data series window.

Using my example from above if I have an indicator with the following code in its State.Configure

AddDataSeries(BarsPeriodType.Minute, 1);
AddDataSeries(BarsPeriodType.Minute, 5);
AddDataSeries(BarsPeriodType.Day, 1);

I then create a new chart.

I add an Instrument 3 times to the chart through the Data Series window.

1. First data series is daily. I specify February 8th, 2021.
2. Second Data series is 1 minute. I specify February 15, 2021
3. Third Data Series is 5 minute. I Specify Feburary 22, 2021.

All 3 data series in the window use the same time series.

When I check the first bar found in the indicator I get

BarsInProgress[1] => February 8th, 2021
BarsInProgress[2] => February 15th, 2021
BarsInProgress[3] => February 22nd, 2021.

It would seem that for each chart, the underlying objects for that data series is only created once for that chart and reused by all indicators (and I assume strategies) added to that chart that request the same data series.

[ntbone]

Another test reveals a potential problem.

Consider the following. An indicator with the following code in State.Configure.

AddDataSeries(null, new BarsPeriod() { BarsPeriodType = BarsPeriodType.Minute, Value = 5 }, 100, null, null);

Then consider a chart with

• First data series is 1 minute with 1000 bars to load.
• Second data series is a 5 minute with 50 bars to load

If you check the last bar loaded for historical data before switching to real time you will find that

Primary data series gets the 1000 bars you requested.
The secondary data series only gets 50 bars ignoring the request by the indicator to load 100 bars.

In this case, the indicator is explicitly requesting a # of bars to load and since the # of bars requested is less then the # of bars the indicator needs it would seem that the AddDataSeries should override the request by the user and load 100 bars?

[NinjaTrader_ChelseaB]

Hello ntbone,

I'm not certain I'm fully understanding.

You are saying you do not want the BarsToLoad overload for AddDataSeries() to have any effect, is this correct?

You are wanting the BarsToLoad overload to be ignored?

[ntbone]

What I am saying is the BarsToLoad is being ignored, which is fine if the data series in in the Data Window specifies a # bigger then what was requested by the indicator.

If the indicator requestes 100 bars in AddDataSeries and the user specifies 50 bars in the Data Window (for additional data series)100 bars should be loaded, not 50 as it does right now. The data should load whichever data series requests more bars.

[NinjaTrader_Jim]

Hello ntbone,

In a previous thread we submitted a feature request so that BarsToLoad would reflect loading the data from the starting point of the script instead of from the current time on the chart.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/indicator-development/1139609-adding-sufficient-daily-data-series-to-indicator-on-1-minute-chart?p=1140829#post1140829

I understand that this feature request would meet your requirement. Do you want BarsToLoad to start loading bars from the start point of the script, or do you want the data to go back for all data series to the point that is furthest back?

[ntbone]

Well there are 2 separate, but intertwined issues going on.

1. The feature request is to have BarsToLoad to start from the first primary bar series data point loaded instead of the last primary bar series data point (before Realtime starts).
2. The issue above has to do with BarsToLoad being completely ignored if the user adds the same data series to the "Data Series" window.

With regards to #1, if I am using an indicator on the data series that requires N # of bars in order to perform its calculation, that I will be have the required # of bars before the first primary series data bar is processed. For example, if the primary data series is a one minute chart and the additional data series is a 5 minute period and I am using a 20 period moving average on the 5 minute data series, I will want at least 20 bars loaded on the 5 minute series before processing bar 0 of the 1 minute series.

With regards to #2, if I create a chart with both a 1 minute data series and a 5 minute data series in the Data Series Window, and I set the 5 minute data series to only load 5 bars, but my indicator requests 20 bars to load on that data series, it will only get 5 bars because the Data Series Window is taking precedence. The 5 minute data series should load the max of what the user specifies in the Data Series window and what the indicators request when the call AddDataSeries. In this case, since the indicator is requesting 20, it should load 20.

[NinjaTrader_Jim]

Hello ntbone,

Thanks for clarifying.

NinjaTrader using the existing data series on the chart over the series that is added is expected. Documentation noting this can be found below in the second note.

https://ninjatrader.com/support/helpGuides/nt8/index.html?multi-time_frame__instruments.htm#TrueEventDrivenOnbarupdateMethod

As I understand, the behavior you are wanting is so that when BarsToLoad is specified, this optimization is overridden and you get the same number of bars in the added data series that you requested?

If that is the case, please let me know and we can file another feature/enhancement/change request.

[ntbone]

Sorry. I completely forgot to get back to this.

Yes, if the code specifies BarsToLoad with a specified # of bars, this should override what the user as added as an additional data series but only if the script requests a bigger # then the user.