Unable to pass BoolSeries values from indicator to strategy

[flonkle]

Hello,

I am having a problem passing BoolSeries values from an indicator to a strategy.

I began by simplifying the code found on this thread:

http://www.ninjatrader.com/support/forum/showthread.php?t=4991

Using my code, which is a modified and simplified version of the sample code from the above thread, I am able to pass a variable value of type double from the indicator to the strategy that calls it when this variable is updated in the indicator in the OnMarketData() method.

However, using my modified and simplified code, I am unable to pass values from a BoolSeries, that is modified in the OnMarketData() method of the indicator, to the strategy that calls it.

Please advise me on how to do this.

(I realize it is also possible that the functionality I am trying to achieve is just "not supported".)

I have attached a zip file of my modified and simplified code for anyone who wants to test this out.

Thanks.

[NinjaTrader_PatrickH]

Hello flonkle,

Thank you for your post.

I will test your strategy and indicator on my end and follow up with you when I have additional information.

[flonkle]

Thanks.

I should have set both the bearIndication and bullIndication series to true in the OnMarketData() method of the indicator to make things totally clear. (Instead, I set one to true and the other to false.) But I'm sure you can still see what's going on with the code, as is.

[NinjaTrader_PatrickH]

Hello flonkle,

Thank you for your patience.

I am seeing the same item on my end. I will continue to investigate this matter further on my end. I will follow up with you here when I have additional information.

[flonkle]

Sounds good.

Just so you know, this problem seems to extend to all classes that implement the IDataSeries interface.

The code I was developing when I ran across this problem exhibits this bug/error with multiple classes that implement the IDataSeries interface.

I just submitted this post using code that modifies and simplifies the NinjaTrader sample code for the SampleBoolSeriesStrategy and SampleBoolSeries indicator to illustrate the problem easily and concisely.

[koganam]

Hello,

I am having a problem passing BoolSeries values from an indicator to a strategy.

I began by simplifying the code found on this thread:

http://www.ninjatrader.com/support/forum/showthread.php?t=4991

Using my code, which is a modified and simplified version of the sample code from the above thread, I am able to pass a variable value of type double from the indicator to the strategy that calls it when this variable is updated in the indicator in the OnMarketData() method.

However, using my modified and simplified code, I am unable to pass values from a BoolSeries, that is modified in the OnMarketData() method of the indicator, to the strategy that calls it.

Please advise me on how to do this.

(I realize it is also possible that the functionality I am trying to achieve is just "not supported".)

I have attached a zip file of my modified and simplified code for anyone who wants to test this out.

Thanks.

Try putting statements in OnBarUpdate() to set the boolSeries, then run Update() in OnMarketData() after you set the boolSeries there. Use class variables as datastores.

OnMarketData() and OnBarUpdate() are not called in any determinate order, so let us force an update that ensures that the boolSeries are set and propagated to the Properties through OnBarUpdate().

[flonkle]

Hey Osikani,

Thanks for the suggestion.

As I said earlier, this problem applies to all classes that implement the IDataSeries interface. So, when I implemented changes to follow your suggestions, I found it easiest to work with the IntSeries class.

Unfortunately, it seems that even when you call Update() from within the OnMarketData() method of an indicator, the OnBarUpdate() method does not actually get called. This is my interpretation of the failure of your suggested fix, anyway.

I chose to try to increment my IntSeries values with each incoming MarketDataType.Bid event to make this most clear.

In the indicator/strategy combo attached below, the ExposedVariable of type double is properly incremented from within the OnMarketData() method of the indicator to count the number of MarketDataType.Bid events. This value is then passed properly from the indicator to the strategy that calls it.

Unfortunately, the BidUpdateCountSeries object of class IntSeries, whose value is supposed to be incremented within the indicator's OnBarUpdate() method as a result of calls to Update() from within the OnMarketData() method, is never incremented at all.

Again, I have attached a code example to this post that illustrates this problem.

Thanks again for your suggestion. It would've been great if your fix could have worked as a temporary workaround for anyone in this situation.

[NinjaTrader_PatrickH]

Hello flonkle,

Thank you for your patience.

When using a DataSeries class in NinjaScript you sync the DataSeries to the Bar Series in Initialize() with bearIndication = new BoolSeries(this);. This means that on each bar, there is a value. However, your indicator specifically only assigns 1 value OnMarketData(). The bars that follow are never assigned a value and therefore a check with .ContainsValue return false on every single bar.

Try setting instead the values to a bool and then calling the bool to .Set() in the OnBarUpdate().

Please let me know if I may be of further assistance.

[koganam]

Hey Osikani,

Thanks for the suggestion.

As I said earlier, this problem applies to all classes that implement the IDataSeries interface. So, when I implemented changes to follow your suggestions, I found it easiest to work with the IntSeries class.

Unfortunately, it seems that even when you call Update() from within the OnMarketData() method of an indicator, the OnBarUpdate() method does not actually get called. This is my interpretation of the failure of your suggested fix, anyway.

I chose to try to increment my IntSeries values with each incoming MarketDataType.Bid event to make this most clear.

In the indicator/strategy combo attached below, the ExposedVariable of type double is properly incremented from within the OnMarketData() method of the indicator to count the number of MarketDataType.Bid events. This value is then passed properly from the indicator to the strategy that calls it.

Unfortunately, the BidUpdateCountSeries object of class IntSeries, whose value is supposed to be incremented within the indicator's OnBarUpdate() method as a result of calls to Update() from within the OnMarketData() method, is never incremented at all.

Again, I have attached a code example to this post that illustrates this problem.

Thanks again for your suggestion. It would've been great if your fix could have worked as a temporary workaround for anyone in this situation.

This may be making your test dodgy:

 
   if (e.MarketDataType == MarketDataType.Bid) { testBool = true; }
   else { testBool = false; }

Maybe we can test without the else clause to see if the testBool state is even propagated at all, and turn it off in OnBarUpdate() instead. That way, we know whether or not OnMarketData() is properly changing the state, and if said state is being reflected in OnBarUpdate().

I wish that I had a pristine installation of NT to test this myself, but I am in the middle of a rather large development, and I really do not want to introduce any other code into the mix.

[flonkle]

PatrickH,

I hate to say it, but your analysis was way off. I'll just leave it at that.

However, in efforts to respond to you and to make the problem easier to understand and even more explicit than in my original post, I uncovered some new information and came up with what might be the best "solution" possible.

Unfortunately, I need a live data feed to be able to code a new example problem and then a solution that clarifies things. It is already close to the market close on Friday afternoon, so I'll just wait and post my findings next week when I can get a true live data feed. (I don't want to muddy the situation by trying to use anything other than truly live data from an actual market.)

To craft a solution, I am going to use an IntSeries instead of a BoolSeries. This makes things much clearer and addresses the same problem. As I have said in previous posts, the problem I posed in this thread applies to all classes that implement the IDataSeries interface.

In a nutshell, the problem is that in certain situations, the IDataSeries being set in the OnMarketData() method of the indicator is actually updating values for the "wrong" bar when you try to set the value for the current bar.

Generally speaking, either an off-by-one error or an off-by-two error often exists between the index of the IDataSeries bar being accessed by the calling strategy and the index of the IDataSeries bar being modified by calls to IDataSeries.Set(). Which, if either, of these two types of errors occurs depends on the combination of CalculateOnBarClose values between the calling strategy and the indicator.

(The situations listed below assume that the values for an IDataSeries of class DataSeries that is named myDataSeries are being set in the OnMarketData() method of the indicator.

These situations also assume that you are setting myDataSeries asynchronously with respect to MarketDataType.Last events. If you merely want to modify your myDataSeries values on each incoming tick, you should probably do so in the OnBarUpdate() method of the indicator, with CalculateOnBarClose = false for the indicator, and avoid the entire problem described in this thread. However, even if this is your aim, you should still read my Post #13 about some pitfalls of setting CalculateOnBarClose within an indicator that is first instantiated by a strategy, using Add(), and then called by the strategy.)

• If CalculateOnBarClose = true for the calling strategy and for the indicator, then setting the 0th value of the IDataSeries in order to update its values will always result in values for the IDataSeries lagging the current bar by one bar; i.e. myDataSeries[1] will hold values for the current bar, instead of myDataSeries[0], when OnBarUpdate() is called in the strategy on the first tick of each new bar.
• If CalculateOnBarClose = true for the strategy and CalculateOnBarClose = false for the indicator, then setting the 0th value of the IDataSeries in order to update its values will properly align IDataSeries values; i.e. myDataSeries[0] will hold values for the current bar when OnBarUpdate() is called in the strategy on the first tick of each new bar. In this case, the one bar lag of the IDataSeries is offset by assigning a value of false to the CalculateOnBarClose variable for the indicator that modifies the IDataSeries while the CalculateOnBarClose value of the strategy that calls the indicator is still true.
• If CalculateOnBarClose = false for both the calling strategy and the indicator, then setting the 0th value of the IDataSeries in order to update its values will properly align IDataSeries values once the first call to myDataSeries.Set() happens for that bar. However, unless this call is triggered by a MarketDataType.Last event, this situation will still result in the IDataSeries values lagging by a bar when the OnBarUpdate() method is called in the strategy on the first tick of each new bar. In other words, myDataSeries[1] will hold values for the current bar on the first tick of each new bar, and myDataSeries[0] will hold values for the current bar after the first call to myDataSeries.Set() happens for that bar.
• However, if CalculateOnBarClose = false for the calling strategy, and CalculateOnBarClose = true for the indicator, then the IDataSeries values will lag the current bar by two bars. In other words, setting the 0th value of the IDataSeries in order to update its values will mean that myDataSeries[2] holds values for the current bar when OnBarUpdate() is called in the strategy on the first tick of each new bar. And myDataSeries[1] will hold values for the current bar after the first call to myDataSeries.Set() happens for that bar.

(Important Note: If you're not careful, CalculateOnBarClose for the indicator will always be true, even if you try to set it to false. Please see Post #13 for a brief discussion of some pitfalls involved in assigning a value to CalculateOnBarClose for an indicator that is first instantiated by a strategy, using Add(), and then called by the strategy.)

--------------------------------------------------------------------------------------------------------
Suggested Solutions

Based on the results of testing that are listed in my Post #12, I would only modify the values for the current bar of the IDataSeries in the OnMarketData() method of the indicator using a call to: IDataSeries.Set(arg) when CalculateOnBarClose = true for the calling strategy and CalculateOnBarClose = false for the indicator.

In all other strategy/indicator CalculateOnBarClose states (3 possibilities), I would modify the values for the IDataSeries one bar into the future in the OnMarketData() method of the indicator by using a call to: IDataSeries.Set(-1, arg) --------------------------------------------------------------------------------------------------------

So, to set the value of the current bar of an IntSeries named myIntSeries to arg, you would code:

myIntSeries.Set(arg);

To set the value of myIntSeries one bar into the future to arg, you would code:

myIntSeries.Set(-1, arg);

But no matter how you choose to set the values of your IDataSeries, you will need to make sure to develop the code you use to access your IDataSeries values with those particular choices in mind.

--------------------------------------------------------------------------------------------------------

Please see my Post #12 for a thorough discussion of the problem space and for possible solutions.

Post #12 also provides sample code that you can run and experiment with to see how NT7 handles this problem for different scenarios. The output from this sample code should help you to choose how you want to develop your own code in order to achieve the desired results.

--------------------------------------------------------------------------------------------------------

[flonkle]

Hey Osikani,

Thanks again for taking the time to follow up with this thread.

As you may have read in my most recent post, I pretty much found out what the problem is, and I will post some code next week to better illustrate the problem and the solution.

Still, following your suggested solution did seem to uncover another problem: the idea that the OnBarUpdate() method is not actually called (at all) by calls to Update() within the OnMarketData() method.

I don't know why this is, but the problem is not with assigning values to testBool only in the OnMarketData() method.

This is happening properly, else the exposedVariable would not be incremented properly. Its being incremented also depends on testBool being assigned and re-assigned properly, and the exposedVariable does get incremented properly.

That makes the lack of Print statements, which should be called numerous times from within OnBarUpdate() if the Update() calls within the OnMarketData() method are doing their job, very mysterious.

[flonkle]

Attached to this post is a sample strategy/indicator combo that illustrates the problem using an IDataSeries of class IntSeries. Once you have imported the code into NinjaTrader, you should add the strategy to a 1-minute intraday chart of a fairly busy market during regular trading hours. I used the front month for the ES (S&P 500 E-mini) futures market for my testing.

The indicator, which is first instantiated by the strategy, using Add(), and then called by the strategy, counts the number of MarketDataType.Bid events that occur in the OnMarketData() method of the indicator. I chose to keep track of the number of MarketDataType.Bid events because this is something that can only be triggered in the OnMarketData() method. This cannot be triggered in the OnBarUpdate() method.

(The problem described in this thread only occurs when IDataSeries values are set within the OnMarketData() method of an indicator that is first instantiated by a strategy, using Add(), and then called by the strategy as the strategy accesses this IDataSeries.)

This bidEventCount is stored in a variable of type double called exposedVariable. The indicator also increments values of an IntSeries called BidEventCountSeries to keep track of the current bidEventCount.

The indicator properly passes the exposedVariable to the strategy that calls it. The indicator also passes the IntSeries BidEventCountSeries to the strategy that calls it. Various values for the BidEventCountSeries "agree" with the proper bidEventCount value as passed in the exposedVariable in different situations. Which BidEventCountSeries value agrees with the value of exposedVariable depends on the different parameter settings currently being used to run the strategy.

The strategy has 3 user-specified parameters:

1. Strategy CalculateOnBarClose: sets the CalculateOnBarClose value for the strategy.
2. Indicator CalculateOnBarClose: sets the CalculateOnBarClose value for the indicator.
3. Indicator BarsAgo: determines which bar of the IntSeries BidEventCountSeries that the indicator updates. The two possible values for this parameter are 0 (indicator updates the current bar, BidEventCountSeries[0]) and -1 (indicator updates one bar into the future, BidEventCountSeries[-1]).

Given that each of the three parameters can take on only 2 distinct values, there are 8 possible states for these parameter settings, 2 x 2 x 2.

You can run the strategy and adjust the parameter settings in order to select each of the 8 possible states to see how the values for the IntSeries BidEventCountSeries compare to the proper bidEventCount value which is properly exposed from the indicator to the strategy in the variable exposedVariable.

On the FirstTickOfBar for the strategy, the strategy prints to the output window the values of CalculateOnBarClose for both the strategy and the indicator. It indicates which value of BidEventCountSeries is being updated by the indicator (i.e. BidEventCountSeries[0] or BidEventCountSeries[-1]). And finally, the strategy prints out the values for BidEventCountSeries[-1], BidEventCountSeries[0], BidEventCountSeries[1], and BidEventCountSeries[2] at that time-step.

If CalculateOnBarClose = false for the strategy, the strategy also prints the above information to the output window on the first call to OnBarUpdate() in the strategy that occurs after the first call to BidEventCountSeries.Set() for the bar has been made in the indicator.

Seeing these values as they update will let you know how the different parameter settings are affecting the values passed from the indicator to the strategy via the IntSeries BidEventCountSeries.

Again, be sure to compare each value of BidEventCountSeries to the proper value of the bidEventCount passed to the strategy as the exposedVariable. This will tell you if the bidEventCount value has been passed properly to the strategy via any of the four BidEventCountSeries values displayed.

Below, I list a table of all possible parameter states and the results I obtained from running the strategy in each of the 8 possible states:

StratCOBC | IndCOBC | IndBarsAgo | Result
----------------------------------------------------------------
T | T | 0 | OK: 1
T | F | 0 | OK
F | T | 0 | OK: 2, 1
F | F | 0 | OK: 1, 0
--------------------------
T | T | -1 | OK
T | F | -1 | OK: -1
F | T | -1 | OK: 1, 0
F | F | -1 | OK: 0, -1
----------------------------

• The above table lists the values for Strategy CalculateOnBarClose in column 1.
• The above table lists the values for Indicator CalculateOnBarClose in column 2.
• The above table lists the values for Indicator BarsAgo in column 3.
• The above table lists the Results of running the strategy with the parameter values listed in columns 1 through 3 in column 4.

(In the following key and the text below it, "BECS" stands for BidEventCountSeries. "FTOB" stands for FirstTickOfBar.)

Below is a key that explains what the above Result values mean:

OK

The values agree as expected and work properly.
BECS[0] == exposedVariable on the FTOB.

----------
OK: 1

BECS[1] == exposedVariable on the FTOB.

-----------
OK: -1

BECS[-1] == exposedVariable on the FTOB.

------------
OK: 2, 1

BECS[2] == exposedVariable on the FTOB.
BECS[1] == exposedVariable after 1st call to BECS.Set() for the bar.

------------
OK: 1, 0

BECS[1] == exposedVariable on the FTOB.
BECS[0] == exposedVariable after 1st call to BECS.Set() for the bar.

-------------
OK: 0, -1

BECS[0] == exposedVariable on the FTOB.
BECS[-1] == exposedVariable after 1st call to BECS.Set() for the bar.

Notice that when CalcuateOnBarClose = true for the strategy, results only report on BECS values for the FTOB. This is because OnBarUpdate() for the strategy is only being called on the FTOB in such cases.

Otherwise, when CalculateOnBarClose = false for the strategy, results are reported for the FTOB and also after the first call to BidEventCountSeries.Set(), as well.

(You may want to read my Post #10 to get some more explanation about what is going on in NT7 to generate these results.

Quickly, the above results exhibit either agreement, an off-by-one error, or an off-by-two error between the index of the IntSeries bar being accessed by the calling strategy and the index of the IntSeries bar being modified by calls to to BidEventCountSeries.Set().

Choosing to set either the current bar (0th value) of the IntSeries BidEventCountSeries or one bar into the future (-1st value) of the IntSeries BidEventCountSeries only shifts these agreements/errors by a bar in each case.

One effect of setting the IntSeries BidEventCountSeries one bar into the future is that it assures that there is only agreement, or at worst an off-by-one error, between the index of the IntSeries bar being accessed by the calling strategy and the index of the IntSeries bar being modified by calls to to BidEventCountSeries.Set().)

-------
Suggested Solutions

Since all states have an OK result of some sort/flavor, you should always be able to manipulate the code you use to access your IDataSeries in order to obtain the values you want, regardless of how you set the values of your IDataSeries.

However, based on the above results I would only modify the values for the current bar of the IDataSeries in the OnMarketData() method of the indicator using a call to: IDataSeries.Set(arg) when CalculateOnBarClose = true for the calling strategy and CalculateOnBarClose = false for the indicator.

In all other strategy/indicator CalculateOnBarClose states (3 possibilities), I would modify the values for the IDataSeries one bar into the future in the OnMarketData() method of the indicator by using a call to: IDataSeries.Set(-1, arg)

------

Running and experimenting with the attached code should help to clarify this post.

------

Please see my next Post #13 for an important note about setting CalculateOnBarClose for an indicator that is first instantiated by a strategy, using Add(), and then called by the strategy.

(I have reached the maximum number of characters allowed for this post.)

[flonkle]

Important Note about setting CalculateOnBarClose within an indicator that is first instantiated by a strategy, using Add(), and then called by the strategy:

In order to actually get the CalculateOnBarClose value of your indicator to change, be sure to assign CalculateOnBarClose within your OnMarketData() method repeatedly (i.e. on each call to OnMarketData() or something similar).

Otherwise, if you assign CalculateOnBarClose within the Initialize() method of your indicator, your assignment will be ignored, and CalculateOnBarClose will be set to its default value of true.

Even if you assign CalculateOnBarClose within the OnMarketData() method of the indicator, but you do so only once, e.g. only on the first call to OnMarketData() or something similar, then CalculateOnBarClose will only retain the value you assign to it for the current bar. On the next bar and all subsequent bars, CalculateOnBarClose will revert to its default value of true.