Erroneous Data in DataSeries

[RyanR]

I am trying to sync a MACD.Diff calculation to a higher timeframe range bar chart. At this point all I am doing is calculating the higher timeframe (15 range) MACD.Diff value and syncing it to that DataSeries. I then plot via a horizontal line either green or red for a positive or negative MACD differential value. The plot will occur for every lower timeframe (5 range) bar. This works for the most part but my line has an occasional incorrect plot. This error is confirmed by print statements in the code. I am getting duplicate values at the same time stamp, one correct and one incorrect. The incorrect/duplicate data does not always happen but when it does it is the last bar of the 5 range updates before the 15 range updates. See Below:

7/3/2013 10:36:26 AM R10 0.00694409322078236
7/3/2013 10:36:46 AM R10 0.00694409322078236
7/3/2013 10:36:55 AM R10 -0.0607314622361978<-------
7/3/2013 10:36:55 AM R11 4.48686402428761E-05
7/3/2013 10:37:11 AM R10 4.48686402428761E-05

I hope this makes sense and any help on what is causing this would be appreciated!

This is the code

public class DiffDataSeries : Indicator
{
#region Variables
// Wizard generated variables
private int fast = 12; // Default setting for Fast
private int slow = 26; // Default setting for Slow
private int smooth = 9; // Default setting for Smooth
// User defined variables (add any user defined variables below)
private DataSeries rangediff;
private DataSeries rangediff1; //Added range dataseries
#endregion
/// <summary>
/// This method is used to configure the indicator and is called once before any bar data is loaded.
/// </summary>
protected override void Initialize()
{
Add(new Plot(new Pen(Color.Blue, 5), PlotStyle.Line, "Line1")); //Current chart bar color
Add(new Plot(new Pen(Color.Blue, 8), PlotStyle.Line, "Line2"));// Higher range chart line 1 Plot 1
 
rangediff=new DataSeries(this);//current chart dataseries
Add(PeriodType.Range, 15);//New Higher range chart
 
Overlay = false;
CalculateOnBarClose = true;
}
/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{
Line2.Set(1); //Added data series 1 horizontal level -color will be changed below
 
if (rangediff1 == null) //test taken from 'SampleDataSeries'
{ 
rangediff1= new DataSeries(SMA(BarsArray[1], 50));
}
 
if (BarsInProgress == 1) //First added 10 range bar timeframe 
{ 
rangediff1.Set(MACD(BarsArray[1], Fast,Slow,Smooth).Diff[0]);//calculte the MACD.Diff and sync to dataseries
Print(Time+" R11 "+rangediff1[0]);//Prints rangediff1 (15bar range MACD.Diff value based on 15 range bar update)
} 
 
if (BarsInProgress == 0) //Current charted range bar timeframe -want to plot MACD.Diff pos/neg Status for 15 range bar timeframe
{ 
if (rangediff1[0]>0) //if postive MACD.Diff plot green otherwise plot red
{
PlotColors[1][0]=Color.Green;//Plots the 15 range MACD.Diff status (pos/neg) based on every 5 range bar 
}
else if (rangediff1[0]<0) 
{
PlotColors[1][0]=Color.Red;
}
Print(Time+" R10 "+rangediff1[0]);//Prints rangediff1 (15bar range MACD.Diff value based on current chart bar update)
} 
} 

[NinjaTrader_JC]

Hello RyanR,

Thanks for the code snippet.

The reason why this is happening is because the both DataSeries are creating a new Range bar at the same time. If you look at the time stamps it only happens when the both of them create a new bar at the same time which is making a new DataSeries entry that will not be referencing what you are looking for.

You may want to check the value of the DataSeries inside of the lower time frame to make sure that it is going to be referencing the bar that you are expecting it to be.

Happy to be of further assistance.

[RyanR]

JC - Thanks for the reply. Yes I did notice that it randomly happens when both bars update at the same time. My question is why it is only happening sometimes and how to avoid the issue. Since I am only writing and reading to the higher timeframe dataseries, how is it possible that this unrelated value is stored as a duplicate with the same time stamp? I am new to dataseries and syncing, so excuse my misunderstanding. I thought that the MACD Diff value is set to each higher bar update (so each bar has its own Diff value). It appears there are 2 values stored for the same bar?

The erroneous numbers are not the lower timeframe MACD Diff values. I am not sure if that was what your advice was. I don't understand what you meant by "check the value of the DataSeries inside of the lower time frame ", since I am not doing anything with any other dataseries (that I know of!).

The more I think I understand the more confused I get.

Any help would be appreciated (GREATLY!)

[NinjaTrader_JC]

Hello RyanR,

So you are syncing with the secondary data series which is great, but the only thing that you have to keep in mind is that if both Data Series are going to be creating a new bar the Primary Series is going to be called first, so when you are referencing "rangediff1[0]" it will not have the data that you expect since the Data Series will have a new object.

You may want to try to move the setting the "rangediff1" inside of BIP 0 but use BarsArray[1] like you are doing.

[RyanR]

JC - I tried to move the set command in BIP0 and received the OnBarUpdate ...index out of range error. I also do not understand how the higher time frame set function would work correctly only being executed on the lower timeframe updates.

At this point I am not married to what I tried, but would ask what method should be used to accomplish what I intended:

Plot a MACD.Diff (or any function) of a different timeframe (higher or lower than the base chart timeframe) that will accurately paint (pos/neg status) of the other timeframes?

[Harry]

There are at least 3 ways to build higher timeframe MACDs. One way is to add a secondary bar series, as you did. This is extremely complex with NinjaTrader, so I do not recommend it. A second way is to calculate composite bars from the primary bars, store those in an array and apply the MACD calculations to those composite bars. The third and simplest way is to tweak the MACD formula in order to calculate the higher timeframe MACDs.

Below is a sample chart with an indicator that plots 5 higher timeframe MACDs. I would never have been able to achieve that by adding a secondary bar series.

If you wish to know the details, just send me a private message.

[RyanR]

Harry-thanks for the feedback. I should explain that I wanted to understand dataseries use so I can build on the concept. I am getting the impression there is an error in how the dataseries operates.

I wish to have multiple higher and/or lower timeframe MACD calculations (slightly more complex than pos/neg diff). Since I am using range bars, to make composite arrays I would need to build these arrays from tick data to have the flexibility of range bar variations. Your simple version would be the easiest, but again I believe I would need the base chart to be a tick chart to accommodate odd range and MACD numbers. This would bring up the issue of ploting a 5 range bar chart on a tick chart with equal spacing. Are these assumptions correct or am I overcomplicating this?

I did try another approach with some success. It was kind of by accident since I was quite tired when I wrote it and it is over doing a bit but the results seem the most accurate. I created 2 loops. The outside loop is for each bar update of the added timeframes. the inside loop recalculates the MACD data for each timeframe and plots accordingly. So I am recalculating every timeframe on every timeframes bar update. Interestingly, when I only did the calculation updates on the base chart update the plots lagged sometimes by a bar. The issue with this approach is that I get an 'OnBarUpdate indexing out of range error' when I use lower time frames.

I would still like to understand the proper application of a dataseries for this application since it seems to be what it is designed for. Since it only happens sometimes when the BarUpdate timestamps are the same I am leaning toward a dataseries error. But not being an expert programmer I hope someone will show me the light, or at least which direction to look for the light.

Again thanks and any additional help would be appreciated.

[Harry]

I do not think that there is an error in how the bars are called. The architecture is simply too complex. There are a few problems to consider

- the secondary bar series is called by OnBarUpdate() after the primary bar series, so the values calculated cannot be used in real-time for the bar close, in case that primary and secondary bars have identical time stamps, however this is possible on Historical Data, this may create a discrepancy between historical and real-time indicators
- if you take snap shots by collecting values from a large timeframe bar and projecting those to a smaller timeframe bar in real-time with COBC = fasle, then the indicator values are similar to a random walk, in this case an interpolation becomes necessary or a step function should be used
- you need to take into account the session breaks, in particular that a bar with timestamp xx:xx:00 belongs to the prior sessions, if it is built from minutes, but to the new session, if it is built from ticks
- for Range and Renko bars things become more difficult because the bar completion can only be detected once the first tick of the new bar has been plotted

I have code a MTF EMA. The code has 980 lines within OnBarUpdate(). The logic is simply too complex, if you want to apply it to all cases.

The approach to use composite bars is not easy to adapt to range bars. You can replace three 100-tick bars with one 300 tick bar, but you cannot replace two 3-range bars with a 6-range bar, but would probably rather replace 4 3-range bars with a 6-range bars.

Attached is a MACD built from composite bars - the indicator calculates those bars -, which does not load a secondary bar series of range bars. The multiplier here used is 2, which simulates a MACD calculated from 6-range bars on a 3-range chart. The composite bars use the square root relationship between volatility (range) and time, which means that for a factor 2, one higher timeframe composite bar is built from 4 single lower timeframe bars.

Even if this sounds complicated. The concept of composite bars and of tweaked indicators is much easier to handle, than genuine MTF indicator. The bar processing logic of NinjaTrader is simply too complex to do it correctly, and also there are serious limitations.

[RyanR]

Harry- Thanks for sharing your experience. It is very disappointing that NT does not play well with MTF. That was the main reason I started playing with NT. Since I need to work with 3-15 Range bar time frames I am not sure how to proceed with composite bars. I am also confused how time is involved in a range bar conversion. If I use tick data then I would have to calculate each range bar for each timeframe and store them in arrays to use in my calculations (which of course is what I thought the dataseries was suppose to do)

If you could share a little more detail about your calculations maybe a light bulb would come on.

Thanks in advance!

[Harry]

I would have to write a book here to explain it. If you wish, send me a private message with your Skype ID and we can then discuss the details via Skype.