Add Data Series Method Issue in Strategy Analyzer

[RandanAL]

I've been trying to add several data series into a timeframe using "AddDataSeries" and I also want to pull in historical bars so that I can calculate indicators off of them.

In strategy analyzer, one can simply start the analysis a month into the past and let it run, but this creates an obvious issue when transitioning to live trading.

So, I altered my code using the longer "AddDataSeries" syntax which allows barsToLoad to be included. However, this doesn't seem to be pulling through when using strategy analyzer.

When I debug using print I would expect there to be bars of the loaded historical data available from the start (Output1).
At a minimum I'd think that a few months into the analysis the data would start being pulled in (Output2).

Are there known issues with this method?

Thanks,
David


Note that the instrument names are placeholders.

AddDataSeries Code:

else if (State == State.Configure)
{
AddDataSeries("A", new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, 1170, "US Equities RTH", true);
AddDataSeries("A", new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 }, 30, "US Equities RTH", true);
AddDataSeries("B - Note this is Default Instrument", new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 }, 3, "US Equities RTH", true);
AddDataSeries("

B - Note this is Default Instrument

", new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, 1169, "US Equities RTH", true);
}

Code to Debug:
Print("Instrument Bars" + CurrentBars[0]);
Print("A Minute Bars" + CurrentBars[1]);
Print("A Day Bars" + CurrentBars[2]);
Print("Instrument Day Bars" + CurrentBars[3]);
Print("Instrument Minute Bars" + CurrentBars[4]);
Print(" "); //So that there is a space and easier to follow in the output



Output at Start:

Instrument Bars6
A Minute Bars-1
A Day Bars-1
Instrument Day Bars-1
Instrument Minute Bars6

Instrument Bars7
A Minute Bars-1
A Day Bars-1
Instrument Day Bars-1
Instrument Minute Bars7

Instrument Bars8
A Minute Bars-1
A Day Bars-1
Instrument Day Bars-1
Instrument Minute Bars8

Instrument Bars9
A Minute Bars-1
A Day Bars-1
Instrument Day Bars-1
Instrument Minute Bars9


Output Several Months into the Analysis:

Instrument Bars29618
A Minute Bars-1
A Day Bars-1
Instrument Day Bars-1
Instrument Minute Bars29618

[NinjaTrader_Jesse]

Hello RandanAL,

I believe part of what you are seeing here is normal, not all bars are going to be available on the first bars which call OnBarUpdate. When processing begins, all of the counts will begin with -1 and increment from there. The series such as daily will need to elapse a bar before the -1 turns into a 1, you will see quite a few updates to the minute series before that happens.

This is also generally why you see/need this logic in multi series scripts:

if(CurrentBars[0] < 1 || CurrentBars[1] < 1 || CurrentBars[2] < 1) return;

This makes sure you have 1 bar for all series before starting your logic.

Once the other series elapse a bar they would increment. This brings up the next question, do you have data for the added series for the timeframe requested? Can you open a chart and view the full dataset for the daily bars? I would expect that the daily bar starts to increment after a few days of minute data had passed assuming there is daily data there.

For example, here is a slightly more simple test using a single added daily series and a primary minute series, this is the output or what I would expect:

Instrument Bars-1
A Minute Bars0
Instrument Bars-1
A Minute Bars1
Instrument Bars-1
....
A Minute Bars1166
Instrument Bars1

I look forward to being of further assistance.

[RandanAL]

Yep, I have the data....

And yes I get that the other series would elapse a bar which is why I showed that they weren't working several months into the analysis... It honestly seems like the AddDataSeries syntax which allows barsToLoad is just glitchy/broken.

Previous output at instrument bar 29618:

"Output Several Months into the Analysis:

Instrument Bars29618

A Minute Bars-1

A Day Bars-1

Instrument Day Bars-1

Instrument Minute Bars29618"

If I use the following code I get a different output at bar 29618, which seems to be functional... the big drawback is that I need to load this into a chart with all of the relevant data in order to be able to trade so it just makes the whole process of switching between Strategy Analyzer and live trading janky vs. being able to load historical bars which is a "cleaner" solution:

AddDataSeries("A", BarsPeriodType.Minute, 1);
AddDataSeries("A", BarsPeriodType.Day, 1);
AddDataSeries("B", BarsPeriodType.Day, 1);
AddDataSeries("B", BarsPeriodType.Minute, 1);



Instrument Bars390
A Minute Bars388
A Day Bars0
Instrument Day Bars0
Instrument Minute Bars390

…….

Instrument Bars29618
A Minute Bars29542
A Day Bars74
Instrument Day Bars74
Instrument Minute Bars29618

[NinjaTrader_Jesse]

Hello RandanAL,

Thank you for the reply.

It seems that you have used a different overload set in what you mentioned is working. As you were specifying the trading hours, this could relate to the problem as that is no longer being used. If you have a specific script example that exhibits a problem and shows the non working syntax being used and the working syntax commented out I could look further into that. If you can provide the specific steps you used to see the problem that would help. Such as the instruments begin tested, time frame selected in the analyzer and other relevant settings to the test.



I look forward to being of further assistance.

[RandanAL]

Appreciate the follow up... for reference the pertinent info is below but honestly from my end it is easier to just load the strategy via a chart and not worry about loading the historical bars which bypasses the bug rather than spending time trying to work it out.

The only thing changed between the runs is the overload set for AddDataSeries()

Both simulations are the same dates (1/1/19-Present)
Entire strategy is running on RTH
Strategy default instrument is "B" minute bars
All data is historical data from IB and did not change between tests

To see the issue, I simply printed the bars in progress with "protected override void OnBarUpdate()" and added a return set to 100,000 after the prints so that the strategy analyzer wouldn't halt when hitting entry conditions and I could see that the bars weren't updating.

Print(CurrentBars[0]);

Print(CurrentBars[1]);

Print(CurrentBars[2]);

Print(CurrentBars[3]);

Print(CurrentBars[4]);
if (BarsInProgress<100000) return;

Relevant changes to the code:

AddDataSeries("A", new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, 1170, "US Equities RTH", true);

AddDataSeries("A", new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 }, 30, "US Equities RTH", true);

AddDataSeries("B", new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 }, 3, "US Equities RTH", true);

AddDataSeries("B", new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, 1169, "US Equities RTH", true);

AddDataSeries("A", BarsPeriodType.Minute, 1);

AddDataSeries("A", BarsPeriodType.Day, 1);

AddDataSeries("B", BarsPeriodType.Day, 1);

AddDataSeries("B", BarsPeriodType.Minute, 1);

[NinjaTrader_Jesse]

Hello RandanAL,

Thank you for the followup.

If you have found a solution we can go ahead and close the post. I will review the case further, however if this code is no longer being used it may not be worth the effort to look into without a more specific test case. If you did want to continue to troubleshoot syntax you were using, we can dive further into that I would just need an export of that script to review it further.

Please let me know if I may be of further assistance.

[affilife]

Appreciate the follow up... for reference the pertinent info is below but honestly from my end it is easier to just load the strategy via a chart and not worry about loading the historical bars which bypasses the bug rather than spending time trying to work it out.

The only thing changed between the runs is the overload set for AddDataSeries()

Both simulations are the same dates (1/1/19-Present)
Entire strategy is running on RTH
Strategy default instrument is "B" minute bars
All data is historical data from IB and did not change between tests

To see the issue, I simply printed the bars in progress with "protected override void OnBarUpdate()" and added a return set to 100,000 after the prints so that the strategy analyzer wouldn't halt when hitting entry conditions and I could see that the bars weren't updating.

Print(CurrentBars[0]);

Print(CurrentBars[1]);

Print(CurrentBars[2]);

Print(CurrentBars[3]);

Print(CurrentBars[4]);
if (BarsInProgress<100000) return;

Relevant changes to the code:

AddDataSeries("A", new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, 1170, "US Equities RTH", true);

AddDataSeries("A", new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 }, 30, "US Equities RTH", true);

AddDataSeries("B", new BarsPeriod { BarsPeriodType = BarsPeriodType.Day, Value = 1 }, 3, "US Equities RTH", true);

AddDataSeries("B", new BarsPeriod { BarsPeriodType = BarsPeriodType.Minute, Value = 1 }, 1169, "US Equities RTH", true);

AddDataSeries("A", BarsPeriodType.Minute, 1);

AddDataSeries("A", BarsPeriodType.Day, 1);

AddDataSeries("B", BarsPeriodType.Day, 1);

AddDataSeries("B", BarsPeriodType.Minute, 1);

I have the same problem. Looks like support can't see what's going on.

To speed up the strategy startup time, I need to specify the barsToLoad in AddDataSeries because I am using 1 tick dataseries. I cannot have daysToLoad as 20 days and wait for the strategy to load the data, which is unnecessary because not all data is for indicators. I added 1 tick data series to ensure I have the most accurate fill as possible. Keep in mind that this is a multi time frame strategy. That means I need to load 1 tick data to get accurate fill in strategy analyzer.

so for live I use the overload AddDataSeries to specify barsToLoad to minimize all data needed for historical runs. Then when switch to backtest, if I have the overload AddDataSeries, then the strategy analyzer does not work because it does not load more for historical data, which strategy analyzer is using. So I need to switch to another AddDataSeries() to let the backtest run.

This is clearly an oversight in designing the strategy analyzer. Too many things along the lines of "you have to do this to make it work". I have to tweak so many things to make the strategy analyzer to work because I dont want to use playback mode to test. Even then, the fill is still not that accurate with 1 tick data series.