BUG: Use of indicator with secondary timeframe does not compute when datafeed connect

[Jeremytang]

The issue is if you have an indicator A that uses indicator B, where B uses a secondary tick timeframe, indicator A's historical calculations all get called before indicator B's when you connect to a datafeed. Because indicator A's values depend on B, you don't get any values.

Note: this does not happen every time, but only some of the time (seemingly random - possible race condition)? It is still very repeatable.

Steps to reproduce:

1. Add the ParentIndicator to a tick chart
2. Open up output window
3. Connect to a datafeed (simulated datafeed will cause behaviour)
4. Note the output window showing ParentIndicator blowing through all historical calculations before the SubIndicator gets a chance (and lack of plot)

Jeremy

[NinjaTrader_ChelseaB]

Hello,

I have tested this script with one edit. Instead of a 150 tick added series to the SubIndicator script, I have changed this to 1 tick. I am adding the ParentIndicator to a chart with one tick.

I think the issue is that the bar size of the SubIndicator is larger than the chart the ParentIndicator is added to.

The output looks like this:

ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 0, Close[0] = 1955.75
SubIndicator Added timeframe
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 0, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 1, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 1, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 2, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 2, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 3, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 3, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 4, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 4, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 5, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 5, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 6, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 6, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 7, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 7, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 8, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 8, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 9, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 9, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 10, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 10, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 11, Close[0] = 1955.75
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 11, Close[0] = 1955.75
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 12, Close[0] = 1955.75

[Jeremytang]

The error does not occur 100% of the time, but is still very repeatable

I can verify this is not just on my machine that this occurs, but consistently on our customers and partners as well as other 3rd party developers I have spoken with who have observed this behaviour (or side effects of it).

If you run it with a 150 tick (and 150 tick internally) you will see the error. In any case, whether we are utilizing a smaller or larger timeframe, you shouldn't see the parent indicator evaluating all it's values before the sub indicator (since the parent indicator DEPENDS sub indicator's values).

The bug is that while you're in the OnBarUpdate of the ParentIndicator, getting the value from the SubIndicator (which indirectly calls its Update method) triggers an Update call for the ParentIndicator (which it shouldn't).

I've modded the code for the ParentIndicator slightly to report if it's seeing a value for the SubIndicator, you'll see in the output that the SubIndicator has no value throughout (even though they are actually using timeframe with the same tick - 150)

What follows is NT continues to call the Update method for the ParentIndicator for the entire historical graph.

This only happens when you connect a datafeed. When you don't everything runs normally.

ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 1, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 2, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 3, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 4, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 5, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 6, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 7, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 8, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 9, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 10, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 11, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 12, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 13, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 14, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 15, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 16, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 17, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 18, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 19, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 20, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 21, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 22, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 23, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 24, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 25, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 26, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 27, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 28, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 29, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 30, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 31, SubIndicator has Value? : False
ParentIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 32, SubIndicator has Value? : False

<truncated for brevity>

SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 0, Close[0] = 103.43
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 1, Close[0] = 103.42
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 2, Close[0] = 103.41
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 3, Close[0] = 103.39
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 4, Close[0] = 103.39
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 5, Close[0] = 103.47
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 6, Close[0] = 103.5
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 7, Close[0] = 103.49
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 8, Close[0] = 103.37
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 9, Close[0] = 103.4
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 10, Close[0] = 103.4
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 11, Close[0] = 103.47
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 12, Close[0] = 103.45
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 0, Close[0] = 103.43
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 13, Close[0] = 103.41
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 1, Close[0] = 103.46
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 14, Close[0] = 103.44
SubIndicator: BarsInProgress = 0, Historical = True, CurrentBar = 2, Close[0] = 103.45
SubIndicator: BarsInProgress = 1, Historical = True, CurrentBar = 15, Close[0] = 103.44

[NinjaTrader_ChelseaB]

Hi Jeremytang,

To get the same result that you are getting I have switched back to a 150 tick chart for my primary data series.

I am connected to Kinetick as I run this test.

I had been testing this with a particular amount of bars, with Load data based on set to Bars and the Bars to load was set to 300.
Then I realized that would cause the 1 tick series to be short. That will not work for this script. Using Load data based on Bars will cause the primary series to go back much much further than the tick series.

Using 0 days in the days to load causes too much data to print to the output window and the output window gets cut off so I cannot see what is happening as the script is loaded.

The only way I could test this was by writing the prints to a file instead of the output window.

After doing this, everything looked normal.

Let me know what is wrong in the attached file.

[Jeremytang]

Hi ChelseaB,

Your file shows everything running normally, so this is not the bug.

The Subindicator is also running a 150 tick alternate timeframe (use the original SubIndicator.cs file I sent you).

The bug sometime occurs, and sometimes doesn't. I do not know why... try connecting to a simulated feed (sometimes I get it to occur with Kinetick, sometimes with the simulated feed).

You'll know it's happening when you see the output showing:
ParentIndicator ...
ParentIndicator ...
ParentIndicator ...
ParentIndicator ...

before you see any:

SubIndicator ...
SubIndicator ...
SubIndicator ...

[NinjaTrader_ChelseaB]

Jeremytang,

How are you able to see the beginning output from the output window?

This is being truncated on my end because there are too many prints when this runs... so I had to switch to writing to a file.

What instrument are you using?

What are you using for 'Load data based on'?

What value are you using for 'Load data based on'?

[Jeremytang]

Info

Chelsea,

First, thanks for you patience and persistence with this issue. It's one that I've been tracking down for a long time now; but it's become a priority now as it's affecting some of our larger clients (and in turn their customers).

I'm not loading that much data (and using the 150 tick timeframe for both the main series and the secondary series loaded in the SubIndicator) so I'm able to see the top of the output window.

I'm using:

DataSeries: CL ##:## 150 tick
Load data based on: "Days"
Days to Load: 3

I've uploaded a video to YouTube showing the behaviour.

http://youtu.be/6Qab5NqAQvY

As I mentioned before, this doesn't happen 100% of the time, for example this morning it wouldn't happen on my office computer. I have however, seen it enough times to know it is repeatable.

Thanks again for you attention Chelsea,

Jeremy

[NinjaTrader_ChelseaB]

Hi Jeremytang,

Thanks for your reply.

I've made a quick video where I am trying to reproduce your results but am unable.
http://screencast.com/t/7llTTqVVU7U

We may want to schedule a call so we can look at this on your end.

First, I'd like you to upgrade to the latest version of NinjaTrader and try again.
http://www.ninjatrader.com/download-registration.php

If you still get the odd results, send me an email at support [at] ninjatrader [dot] com so that we may schedule a call. In the email please add a link to this forum thread.

[Jeremytang]

Made it easier to reproduce

Hi ChelseaB (or to whom this may concern).

I'm using build .25.

I've finally had some time to revisit this issue, as yet another one of our larger customers is now experiencing the side effects of this problem.

I have modified the two .cs files, please update them so you can reproduce the issue. Again, what is happening is the Update(); call in the SubIndicator is inducing the call in the ParentIndicator (never actually running the acutal OnBarUpdate in the SubIndicator... until the very end when ParentIndicator is already done the entire Historical bar).


Here is a link to the video of it happening:

http://screencast.com/t/nsjKY3gzx

New CS files are attached.

I'd be very interested in scheduling a call with you guys if you have trouble reproducing it.

Jeremy

[buddy858]

Added Confirmation

Hi Chelsea,

I just wanted to let you know that I have been dealing with this problem for a while now, and it has been rather difficult, at times, to reproduce, until now. It still seems rather random, but Jeremy has definitely made it much easier to figure this thing out. Here is a very short video of how I'm doing it: http://www.screencast.com/t/URbQnwkgk

Using a 1 min chart with the indicators Jeremy has provided, and while connected to a datafeed with the output window open, I just switch from instrument to instrument until the error occurs. Mostly, it's pretty random, although I have yet to see the error on ZW.

Hope this helps.

Thanks,

Daniel

[zacharydw00]

Hi Support,
I am having the same issue. Here is a video explanation.

http://www.screencast.com/t/buKlZz1K7t

Indicators are attached.

[NinjaTrader_ChelseaB]

Hi Jeremytang,

I'm using your scripts, and switching instruments but I am unable to reproduce.

Please send me an email at platformsupport [at] ninjatrader [dot] com so that we may schedule a call. In the email please add a link to this forum thread.

[zacharydw00]

Our NinjaScript lead has more information about this limitation.

The issue is with the data loading rules behind the scenes with the secondary series.

This is more pronounced on the Simulated Data Feed as this connection does not provide historical data.

This limitation is being addressed for NinjaTrader 8 and is currently in our tracking system with ID #125.

Currently, the workaround is to manually add data series to the chart that has the same symbol and interval as the secondary series in the script.

I've been testing this throughout the morning and it has consistantly reloaded the script properly while connected to the Simulated Data Feed.

Could you try testing this and let me know if this consistantly works for you as well?

I look forward to your reply.

Sincerely,
Chelsea
NinjaTrader Customer Service

Hi Chelsea,
Unfortunately adding a second dataseries, per your instructions, to the chart is not reliable. I was able to reproduce the issue after a couple of tries. Here is a video of it.. http://www.screencast.com/t/rXj0XjEPFO

Do you have any other possible solutions, or does everyone have to wait for NT 8?

Thanks for your help,
Zac

[NinjaTrader_ChelseaB]

Hi zacharydw00,

Thanks for giving that a try.

I'm currently looking into this and attempting to reproduce as well.

I appreciate your patience.

[NinjaTrader_CalH]

Hi Chelsea,
Unfortunately adding a second dataseries, per your instructions, to the chart is not reliable. I was able to reproduce the issue after a couple of tries. Here is a video of it.. http://www.screencast.com/t/rXj0XjEPFO

Do you have any other possible solutions, or does everyone have to wait for NT 8?

Thanks for your help,
Zac

Hello Zac,

This is a known limitation and has been fixed for the next build, NinjaTrader 8.

The workaround that was provided by Chelsea is not a permanent fix, but rather to help see less of the behavior.


Let me know if I can be of further assistance.