Using a DataSeries as source input for an Indicator

[dom993]

Hi there,

I have developed a very simple Indicator, which input might not be set on some bars, so I use IsValidPlot(CurrentBar) to figure this out:

protected override void OnBarUpdate()
{
if (Input.IsValidPlot(CurrentBar))
{
PriceLevel.Set( Input[0] );
}
}

I use this indicator using DataSeries as input, for example:

private DataSeries aTestSeries;
// ...
// in Initialize() :
//
aTestSeries = new DataSeries( this, MaximumBarsLookBack.TwoHundredFiftySix);
Add( VerySimpleIndicator( aTestSeries ) );
// ...
// In OnBarUpdate() :
//
if (condition)
aTestSeries.Set( Close[0] );
VerySimpleIndicator( aTestSeries )


The above doesn't work correctly, meaning in the indicator Input.IsValidPlot(CurrentBar) is always true.

However, if I create the aTestSeries using an infinite size, it works perfectly :

private DataSeries aTestSeries;
// ...
// in Initialize() :
//
aTestSeries = new DataSeries( this, MaximumBarsLookBack.Infinite);
Add( VerySimpleIndicator( aTestSeries ) );
// ...
// In OnBarUpdate() :
//
if (condition)
aTestSeries.Set( Close[0] );
VerySimpleIndicator( aTestSeries )


Of course, at this time this is what I have done, however in the long run I do not want to waste "infinite" amount of memories when a small size DataSeries object would do. I smell there is a bug somewhere in NinjaScript.

[NinjaTrader_Joydeep]

Hello dom993,
Thanks for your post.

Can you send a sample code to support[AT]ninjatrader[DOT]com depicting the issue.

Please append Attn:Joydeep in the subject line of the email and give a reference of this thread in the body of the email.

I look forward to assisting you further.

[dom993]

Well, I am definitively unhappy on this.

I did exactly what was asked from me ... I even went further & created a very simple demo of this bug. I sent it on Saturday morning via email, which I will copy below ... today I received a BS response, which is definitely BS because when I do exactly what is suggested, of course the bug remains.

Can someone-else from Ninja Support take a real serious look at this?!



1st my email, then the BS response:

Hi Joydeep,

This is following your response here : http://www.ninjatrader.com/support/f...073#post287073

Run the attached strategy and look at the Output window.

This bug is really easy to reproduce … it affects DataSeries.ContainsValue() & IDataSeries.IsValidPlot();

I didn’t check for other variants (IntDataSeries, etc.), but I would suspect it is also there.

Regards
Dominique


Here is code which was in the attachment:


#region Using declarations
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Data;
using NinjaTrader.Indicator;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Strategy;
#endregion

// This namespace holds all strategies and is required. Do not change it.
namespace NinjaTrader.Strategy
{
/// <summary>
/// DataSeries MaxmimumBarsLookBack=256 ContainsValue() bug demo
/// </summary>
[Description("DataSeries MaxmimumBarsLookBack=256 ContainsValue() bug demo")]
public class DemoDataSeriesBug : Strategy
{
#region Variables
// Wizard generated variables
// User defined variables (add any user defined variables below)
private DataSeries aDataSeries;
private DataSeries bDataSeries;
#endregion

/// <summary>
/// This method is used to configure the strategy and is called once before any strategy method is called.
/// </summary>
protected override void Initialize()
{
CalculateOnBarClose = true;
aDataSeries = new DataSeries( this, MaximumBarsLookBack.TwoHundredFiftySix);
bDataSeries = new DataSeries( this, MaximumBarsLookBack.Infinite);
}

void demoIsValidPlotBug(IDataSeries iDataSeries)
{
if (iDataSeries.IsValidPlot( CurrentBar ))
{
Print("Bar# = " + CurrentBar + " demo IsPlotValid() bug");
Print("iDataSeries[0] = " + aDataSeries[0] );
}
}

/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{
if (CurrentBar < 256 + BarsRequired)
{
aDataSeries.Set( (double) CurrentBar );
bDataSeries.Set( (double) CurrentBar );
}

Print("");
if (aDataSeries.ContainsValue( 0 ))
{
Print("Bar# = " + CurrentBar + " Demo ContainsValue() bug");
Print("aDataSeries[0] = " + aDataSeries[0] );
}

if (bDataSeries.ContainsValue( 0 ))
{
// This is just to show the problem exist only for aDataSeries, which has MaximumBarsLookBack set to 256
// bDataSeries has MaximumBarsLookBack set to Infinite & it's working fine
Print("bDataSeries[0] = " + bDataSeries[0] );
}

demoIsValidPlotBug( aDataSeries );
demoIsValidPlotBug( bDataSeries );

}

#region Properties
#endregion
}
}






On Mon, May 14, 2012 at 2:40 PM, NinjaTrader Customer Service <[email protected]> wrote:

## Reply ABOVE THIS LINE to add a note to this request ## Hello Dominique,

Please do not mix MaximumLookback 256 and Infinite in your code. Please use either 256 or infinite or still better the below overload to initialize the data series.
aDataSeries = new DataSeries( this);
bDataSeries = new DataSeries( this);
Also if you use Maximum lookback to infinite then please set the Min bar required to 0 (zero).
Please let me know if I can assist you any further.
Sincerely,
Joydeep
NinjaTrader Customer Service
NinjaTrader is a FREE application for advanced charting, market analytics, system development/backtesting and trade simulation.
Flexible payment plans for lifetime licenses starting from $299 per month
Kinetick our preferred market data service. FREE for end of day users. Real time from $50 per month!
Click here for information on our FREE Live Training Events
Connect to us on Twitter and Facebook!
Watch us on YouTube!

[NinjaTrader_Brett]

Hello,

I discussed this with Joydeep. He was able to make the below modifications to your code and get the code to work correctly.

To clarify do you not understand why the changes needed to be made for the code to work correctly or do you still run into issue after making the changes.

-Brett

[dom993]

Wow ... and what does he mean by the code working correctly ?!

The code is meant to set a value in the DataSeries objects for the 1st 256 bars of the chart ONLY, it then calls ContainsValue(0) for all the bars of the chart which returns TRUE for all bars of the chart whenever a DataSeries object was created with MaximumBarsLookBack = 256.

(when it should return FALSE once the CurrentBar gets beyond 256)

Even when BOTH the Strategy & the DataSeries ofbject are created with MaximumBarsLookBack = 256.

This is very easy to reproduce, just take my code, have a look at it to understand what it does, then run it.

[NinjaTrader_Brett]

I checked the code out and ran it as Joydeep did.

Please see my screenshot. I am seeing the expected output correct or am I missing the point?

Furthermore, your print lines where you print bug are actually not bug for the first 256 bars. For the first 256 bars we would expect both prints and then no further prints.

I believe you report you are seeing Print above 265 correct? In which case what is your MinBars set too? This is why Joydeep mentioned to remove this as you set the value for the first 256 + BarsRequired which adds another variable to the mix that you do not need.

-Brett

[dom993]

Brett,

Thanks for at least working on it ... I see that your Output windows stops at Bar# 256 ... I am using charts with way more than 256 bars (try it with 3 days of 1min data), and you'll see that once past Bar# 256 ContainsValue(0) keeps returning TRUE for MaximumBarsLookBack = 256

[NinjaTrader_Brett]

Thanks for the clarification.

I can duplicate what you are seeing here.

For now if you use Infinite on both data series this will resolve.

However I am checking into why this is occurring and will report back once I have more information.

-Brett

[dom993]

"For now if you use Infinite on both data series this will resolve." ... LOL

I have been saying that from the beginning, but my point is that I need to preserve memory, and the MaximumBarsLookBack = 256 feature was just meant for that, IT DOESN'T WORK and I am expecting it to be fixed.

Step one was to get the problem acknowledged on your side, it's been a struggle but I think we are almost there, please provide the tracking ID# for this problem.

[NinjaTrader_Brett]

I will update you when I have further information on this. It will most likely be tomorrow before I have an update as most likely I will need to run this by development as I cannot see anything off right now. That doesn't mean there isnt something we dont have off. In either regaurd I will update you when I have more information.

-Brett

[Light]

Thanks guys. I think I am seeing a related problem, and have for some time now. When I create a dataseries using an NT indicator for values, e.g. MyDataSeries.Set(MACD(12,9,6)[0]), and I have Maximum bars look back set to 256, I will occasionally get an NT error saying the code is looking back farther than 256 bars, even though it is not. The problem is resolved if I simply replace MyDataSeries[barsback] with MACD(12,9,6)[barsback] throughout my code (leaving Max bars look back = 256). In principle, that should have no affect -- but it does. (Of course, setting Maximum bars look back = infinite also "resolves" the problem, as a work-around.) Consequently, it also seems to me NT is not handling the Maximum bars look back effectively.

Thanks again for your help. I'll check back on this thread for a possible resolution to dom993's issue.

Best wishes,

Light

[NinjaTrader_Brett]

Thanks for the report.

Yes sounds like it could be related. I will update when I have news.

-Brett

[NinjaTrader_Brett]

Alright, thanks for your patience.

It turns out this is expected and not a bug. Here is the reason why and it is an area of the platform that no one on our NInjaScript team has worked in therefor we learned something today as well and I apologize for not getting you this information out of the gate to save you debugging time.

MaximumBarsLookBack.TwoHundredFiftySix

This mode is a ring buffer, meaning for every index >= 256 you are looking at old values which may or may not be set. Development lets me know this is the whole point of 256 is to reuse old memory(to save memory) and for programmers that use this mode to be aware that you are doing exactly that.

Since you are re using old already allocated memory it would be important to Reset the value before you access it again after the first 256 bars.

This explains why your code sample runs the way it does, I will also agree ahead of time that our help guide does not make this exactly clear. I will check into if we can update this to make this functionality more clear.

Light,

Please open a new support thread for your issue as it is most likely not related.

-Brett

[dom993]

Brett,

I understood the circular buffer idea, however I was expecting Ninja to automatically clear the oldest values as new bars were added ... In one other thread I did show that this isn't the case (I stopped updating a DataSeries at CurrentBar == 256, and ContainsValue() keeps returning true for the current bar to infinity (& beyond )

Is there a way to clear just ONE particular entry in a DataSeries ? I have to admit I am struggling to understand how the Reset() function works.

Thanks
Dominique

[NinjaTrader_Brett]

Basically to resue memory you need to reset the value, so after 256 bars IF you have a strategy that does not set the DataSeries for each bar then you would need to call DataSeries.Reset(). This will clear it and insert a dummy value(Just the close) so that it A) Will not plot and B) .ContainsValue will evaluate to false.

Then you can set it with what you need later in the code.

Basically the part we were missing is that NinjaTrader will not auto clear and make sure that memory is reset. The memory is reused so you need to do a reset call to make sure it is clear after the first 256 bars.

-Brett