BarsArray[1].Count is 3 bars less than BarsArray[0].Count?

[tgn55]

Done... looks fine now.

[NinjaTrader_ChrisL]

Hi tgn55, thanks for fixing that.

If you suspect a bug and have already reduced your script and eliminated all possibilities of the custom script being the issue, I will need some reduced sample and steps you are taking to reproduce the issue. The bar count of each series is the same in my tests so I currently have no way of reproducing this.

Kind regards,
-ChrisL

[tgn55]

It's not my code that is causing it.

I have tested on a chart that ONLY has your testbarscount indicator on it - nothing else...and I can get discrepancy in bar counts in DataLoaded.

It displays:
BarsArray[0].Count = 672
BarsArray[1].Count = 670

However, if I reload historical data, it gets back in synch, although now, BOTH barcounts are 675 !! And this is on DAILY chart... so nothing changed in the last couple hours! No new bars.

Fine... well, no, not really. The bar counts simply don't make sense - something appears to be broken
It still means my indicator, which uses GetOpen, gets an Object Ref error when this kicks in, since it refers to a bar that doesn't exist in the secondary dataseries. I believe my code is kosher... but does not really expect BarsArray[1] to be 2 or 3 bars behind BarsArray[0] in State.DataLoaded. That's not my code's fault !

Have recorded a short vid to demonstrate, watch it here . It is MKV format, you probably want VLC Player to view it.

T.

[NinjaTrader_ChrisL]

Hi tgn55,

If you have a test script that shows each bars object has a different count then please send it to me. Every time I print out the bar count from State.DataLoaded the bar count is the same between each series. The video you uploaded is blocked or set to private, you will need to enable sharing on that link before I can see the video.

Kind regards.

[bltdavid]

Does clearing the cache help any?

[tgn55]

Chris I am using your script... just added in a bit of text so it now reads

 else if (State == State.DataLoaded)
{
Print("TestBars BarsArray[0].Count: " + BarsArray[0].Count);
Print("TestBars BarsArray[1].Count: " + BarsArray[1].Count);
}

I observe the aberrant behaviour on a chart that was running with my own indicator loaded. I have since removed my indicator, leaving only your script.. and it displays the different bar counts.
If I refresh, or open a new chart, it looks fine.

So, we might surmise that my script somehow broke the chart's dataseries that your script uses ?? If that's true... it's still pretty bizarre.
ANd, what does my script do when this thing bites? Well, not a lot.

As you can see from my pics, this thing emulates backtest of a strategy, showing entries, exits and trade lines just like you see in Strategy Analyser.
So, all it does when it closes a trade is

1. some trade P/L calcs
2. Remove a couple of drawing objects (all inside appropriate tests for being != null )
3. Draw a couple new drawing objects (the exit triangle and the trade line)
4. Draw a TextFixed object summarising the "backtest" results

I should point out that all of the above code regarding removing/adding drawing objects etc works just fine when a new bar is formed, but a trade remains open rather than is closed on the bar. Same at end of session. The little triangle and lines are updated correctly, I witness this happening on many charts, many times, without failure.

So.. it remains a mystery how things can get so messed up when I close a trade, albeit I do also show an up or down arrow on trade close... same as is shown in SA.

Hope that helps...

T.

[bltdavid]

Which instrument are you using for your charts?
Is it Sugar?

Does it happen on any other instruments?

[NinjaTrader_ChrisL]

Hi tgn55,

I will need a consistent way to reproduce the reported behavior. All scripts that I create are reporting the correct bars loaded when I check.

Kind regards,
-ChrisL

[tgn55]

I've been going back checking my charts which have my indicator running on them. Several had inconsistent bar counts... before the opening bell. /HG, /CT, /GC for instance. I walked away from the computer for an hour or so... (more like 2) came back just now, and when I reloaded Ninjascript (to print the values)... they are now all in synch again.. Grrrr..... And no, I'm not smokin anything ;-)

I cannot explain this... and it is a little different than I reported earlier, when I referenced closing trades. What I saw a couple hours back was showing on charts with no open trades showing, so all that may have been a red herring.

The transient nature of this makes it difficult to provide a repeatable test case... and, I'm not sure that any new chart you open and add your indicator to will display the issue... like I said, what I have seen is it gets resolved on reload of historical data... and I have not yet seen it on a fresh chart.

Sorry, I'll do what I can to capture it in play, but I suspect will now need to wait for at least another day.

If you have any ideas on what else I might log, let me know.

Cheers,
T.

[tgn55]

Hi bltdavid , I have seen this on Sugar... I think a week or 2 back... but as per above post, it was affecting /GC, /HG, /CT (and maybe more) earlier today/tonight (I'm in Australia ;-)

Trading hours/session start times seem somehow to be having an impact... but danged if I can explain how the bar counts should move like I reported above.
As for clearing cache... well... that seems a bit over the top. If I have charts open, with indicators that are working fine, and I just leave them running...DO NOTHING... I do NOT expect to get Object Ref errors thrown due to some deep in the bowels malfunction ;-)

To be candid... this is not a real big deal... I can refresh data and move on... but I have many charts open, running this thing (it's kinda like running a strategy on a chart... but has other helpful features that make it somewhat better than just doing that! (like... immediate performance reporting and other cool stuff). Now... if the system causes it (the indicator) to choke with an Obj Ref exception... and I don't notice it (coz I have so many of 'em running), then I am at risk of missing a trade signal.

So.. that's why I care about it! Plus... I am just a nerd who thinks stuff that's broken should be fixed!

I may one day discover that somehow, my code is ultimately causing this... but, it's pretty hard to see how. Add/remove a couple drawing objects... and change the chart's bars count on one series and not the other (by 2 or 3) ?? Really?

Thanks for your interest...
T.

[bltdavid]

I am just a nerd who thinks stuff that's broken should be fixed!

Me too!

Sometimes ya gotta work around things.

I would try something like this.

Setup an entirely empty workspace, I call mine BLANK.xml.
[Just save an empty 'Untitled' workspace as 'BLANK'.]

When exiting NinjaTrader, open BLANK.xml and close/save each
trading workspace, leaving BLANK.xml as the last open workspace.
Exit NinjaTrader.
[The point is to setup _Workspaces.xml so when NinjaTrader restarts
it uses a known completely empty workspace.]

Start NinjaTrader with a known completely empty workspace.
Connect to your data provider.
Manually reload historical data via Tools -> Historical Data -> Load tab.
Open your trading workspace.

Theory #1:
At this point all charts should be 100% completely in-sync
with regard to the bar counts of your primary & secondary
data series.

Theory #2:
If all charts start in-sync via 'Theory #1' being proven true,
then all charts will stay in-sync when new bars arrive from
the incoming real-time data.

First prove 'Theory #1' as true.
Can you then prove 'Theory #2' as true or false?

Why do this?
It would be very curious if you find cases where you've proven the
bar counts start in-sync but then somehow become out-of-sync
later as the real-time data forms new bars.

Good luck!

[bltdavid]

As for clearing cache... well... that seems a bit over the top.

Not at all.

It's very common, and that exact advice is given quite often.
(In NT7, clearing the cache was known to resolve many chart issues.)

It's also super simple. Just before you start NinjaTrader:
Delete the 'cache' folder inside the 'db' folder. When NT restarts,
it will automatically recreate the 'cache' folder and build new caches
as necessary.

[tgn55]

Hi bltdavid,
Just logged in this AM (was up till like 4 AM last night...:-( )

Have not yet performed the steps you made in post #26... but I am reasonably certain that your final para is exactly what is happening. Time will tell

I have added in code that dumps both counts in State.DataLoaded in my indicator. Yesterday I absolutely saw several instances where the numbers were different... and a couple hours later... with me doing nothing other than hitting F5, they were ALL back in synch. They CHANGED !! As reported, all charts now report synchronised... so, the trap is set... now I wait! Meanwhile, I'm going to examine the workspace .XML files... after a cup of coffee!

Re clearing cache... I can do this, sure, I was only saying that needing to do this on an ongoing basis isn't really an acceptable solution, for the reasons I mention in Post #25. It would be like running a strategy that you know sometimes just spontaneously craps out with an exception. Finding that it did at some later time and restarting it doesn't really cut it as a solution... the problem needs to be fixed at its source, so it NEVER aborts.

Oh... BTW, I think I did mention this somewhere in the history, but I do call OnRender... and do nothing more than tweak the vertical position of all my text drawing objects. Again... hard to see how that could responsible for messing up bar counts...but, just to be sure I'm "making full disclosure" of what this thing does... there you have it! (Why do I do this? So that if the chart scale is changed, my trade entry/exit text things stay "glued" to the place they should be, instead of floating/sinking to crazy positions!!)

Cheers,
T.

[tgn55]

So... I have a bunch of charts open in my trading workspace... they are static, I don't delete them, don't add indicators etc...

I checked ALL my charts with my indicator a week ago, roughly. All good. All daily Bar counts in synch.
Today... I have one chart that reports this in State.DataLoaded:

TestBars BarsArray[0].Count: 667
TestBars BarsArray[1].Count: 666

I have not changed this chart. My indicator made no changes (ie adding any drawing objects). It's on a symbol (/HE) that I have no activity in for over a month.

The count changed (I mean went out of synch)... with no action on my part.

It would be nice if I could capture (log) this in the Output window... but with many things happening, my Output Window fairly frequently gets cleared, so all I have is current (broken) state... I cannot show the previous unbroken state. I may have to resort to writing this to a text file.

The mystery continues...
T.

[tgn55]

Finally... I have caught this gremlin "in the act"... and have proof that with no intervention, bar counts did indeed go out of synch..

Chart: /ZC
Type: Heiken Ashi
Period: Daily.
Secondary data series: /ZC standard (not Heiuken Ashi) added in State.Configure

MI = Bars.Instrument.MasterInstrument;
MI_BB_periodtype = BarsPeriod.BaseBarsPeriodType;
MIperiod = BarsPeriod.BaseBarsPeriodValue;

AddDataSeries(Bars.Instrument.FullName, MI_BB_periodtype, MIperiod); // so I can ensure trade price is based on natural bars, not HA values...

then in State.DataLoaded, I include this to log things to a text file...

mywriter.WriteLine(string.Format("In State.DataLoaded, {0} BarsArray[0].Count: {1}... BarsArray[1].Count:{2}", filename, BarsArray[0].Count, BarsArray[1].Count));

and finally, in OnBarUpdate... I log things to both the Output window and my text file

protected override void OnBarUpdate()
{
    if (BarsInProgress == 1)
        return;

    if (CurrentBars[0] < BarsRequiredToPlot)
        return;

    if (BarsArray[0].Count != BarsArray[1].Count)
    {
        Print(string.Format("On CurrentBar {0} of {3} BarsArray[0].Count == {1} but BarsArray[1].Count == {2} !!!!", CurrentBar, BarsArray[0].Count, BarsArray[1].Count, MI.Name));
        mywriter.WriteLine(string.Format("On CurrentBar {0} of {3} BarsArray[0].Count == {1} but BarsArray[1].Count == {2} !!!!", CurrentBar, BarsArray[0].Count, BarsArray[1].Count, MI.Name));
    }

Had to wait a bit... I restarted Ninja on Sunday 5th September, all my charts with my indicator created text log files with the initial (ie State.DataLoaded) line... and on 7th Sept at 9:30 AM local time (which is when a new bar is created)... I caught this in my ZC text file.

In State.DataLoaded, ZC 12-21 Globex BarsArray[0].Count: 671... BarsArray[1].Count:671
On CurrentBar 670 of ZC BarsArray[0].Count == 672 but BarsArray[1].Count == 671 !!!!

My indicator did nothing... I had no open trade, my indicator was just waiting for a setup. Last action by my indicator was on May 17... so many bars prior.

So there you have it. Proof that left entirely undisturbed... a multi-series chart's BarsArray.Counts can initially be in agreement... and then, diverge.

I include a screenshot of my chart.



It appears that the primary series bar count was incremented from 671 to 672... while the secondary series was not.
I have seen other instances where the difference in bar counts is 2 or even 3 bars... not just 1, but (for now) cannot provide the evidence, as it was only (then) logged to the Output window.

Cheers,
T.