Do not use CurrentBar in Plot() method

[steevcoco]

[Edited slightly due to new info, which can be found in the replies below ...)

As the title says, it is not reliable. For the sake of the Manual (in open source programming circles, Google search is called "The Manual"), I thought I would add this: I have a perfect "test case".

I replay mode, with the 6E 06-14 instrument, on a 5-Minute Candlestick chart, on 4/30/2014 (originally, I had posted 4/29/2014): the candle that closes at 05:00 is an example of when Ninja will "stall" briefly before drawing the next candle. -- For whatever reason, absolutely every time I replay past this candle, the candle will close, and Ninja will take a second to open the next candle on the chart. There is a little bit of volume, and the next candle fills in in a spurt (obviously).

I have an indicator doing custom drawing in the Plot method, and had begun using one or two invocations of CurrentBar (in a helper method invoked by Plot) ... and at the close of this candle, my indicator would draw some of the next candle's data on this candle. It is a perfect test case for this specific symptom.

[NinjaTrader_Bertrand]

steevcoco, thanks for the post - would you have perhaps a sample I could give a test-run on our end? If this is possible, would appreciate if you could contact us directly at our support at ninjatrader dot com.

[steevcoco]

Bertrand: I could try (at this point I've modified the code heavily; but I may have some version history I could conjure up for you). I will let you know either way.

[steevcoco]

Bertrand: I'm sorry, but I don't have anything for you. I don't use a versioning system; and I use Ninja's editor for development. I make file copies by hand for crude safety. But I don't have anything left with the offending code in it; and I did try to make something that reproduces that behavior but couldn't do it!

The good news is that when it was occurring, it would ALWAYS happen when replaying past this bar; so perhaps if someone somewhere needs a test case this could come back up ...

Anyway, my indicator gathers ask/bid/last data in OnMarketData, and stores that in a custom class, which is indexed by bar ID. It draws in the Plot method (which itself, as I said, calls some private helper methods). The OnBarUpdate method is essentially empty. The Plot method accesses the data by bar ID (which I was coding using ChartControl.FirstBarPainted - LastBarPainted -- by bar ID); and getting OHLC price data using High, etc. -- which requires a barsAgo parameter. For that I was passing the bar ID to a helper method, which would make a call like High[CurrentBar - barID]. And when this particular bar closed, the code inside the helper method was getting data from the next bar before returning -- and then the Plot method would draw that data on this bar. Then the chart would spurt forward and the rest of the data would draw in sync on the new bar.

The indicator IS gathering tick-by-tick data in the OnMarketData method, and drawing many items (including transparency if that means anything) in the Plot method. However, it doesn't seem to be more sluggish than other indicators; and I'm not sure if I can even notice for SURE, but possibly when replaying at >100X speeds it SEEMS the chart is filling a bit slower when my indicator is present (but I can't even really be sure -- it's not excessively sluggish as far as I can perceive).

Oh, well, It was for certain drawing wrong before I removed all the calls to CurrentBar (which instantly cured it) -- FYI, I simply began storing OHLC data in my custom class. And I also just tried to quickly change those invocations back to High[CurrentBar - barID], etc; but the problem did not recur: perhaps my code has slimmed down, and it's running with less lag, or perhaps it's something else ... (Originally, simply removing the CurrentBar invocations DID cure the bug; so I don't know at this point.)

Hope it's helpful!

[sledge]

I replay mode, with the 6E 06-14 instrument, on a 5-Minute Candlestick chart, on 4/29/2014: the candle that closes at 05:00 is an example of when Ninja will "stall" briefly before drawing the next candle. -- For whatever reason, absolutely every time I replay past this candle, the candle will close, and Ninja will take a second to open the next candle on the chart. There is a little bit of volume, and the next candle fills in in a spurt (obviously).

I see in my recorded tick data that 04:59:50 AM is the last trade, 1.3867

then 05:00:03 through 05:00:09 AM is still 1.3867 with really light volume.

[steevcoco]

Yes sledge: I don't know at this point! (I don't use version control, and I have not been able to reproduce it again!) I don't necessarily feel that it's a high volume issue; but in hindsight, just perhaps the particular particular sequence of ticks -- so that the Indi was [seemingly] reading CurrentBar before it was synchronized. -- That's basically the way it appeared to be happening:

public class MyDataClass {
    // ... data set here by OnMarketData()
}

Plot() {

    // fields to collect paint data
    // here
    for (int barID = ChartControl.FirstBarPainted; barID <= ChartControl.LastBarPainted; barID++) {
        // ... Make one loop through the painted bars to collect needed pre-paint data
    }

    // Then loop through the bars and plot
    for (int barID = ChartControl.FirstBarPainted; barID <= ChartControl.LastBarPainted; barID++) {
        // paint decorations:
        double high = High[CurrentBar - barID];
        double low = Low[CurrentBar - barID];
        // ... There was actually a call to High and Low in here

        // paint bar data
        getBarPlotData(barID);
        // ... paint
    }
}

getBarPlotData(int barID) {
    MyDataClass data = MyDataClass.get(barID);
    double low = Low[CurrentBar - barID];
    // ... process and return
}

So, when barID == LastBarPainted, the decorations were drawn correctly, but getBarPlotData was coming back with data processed from the NEXT bar (and it would then paint incorrectly) -- the Low of the next bar was lower than the low of the current by the time the chart plotted, so the plots were below where they should have been (and plotting bad data ...)!

As I said, I immediately solved the problem by caching the OHLC and being able to remove the invocations of CurrentBar; and it then painted correctly.

[steevcoco]

Heh: I still have NOT reproduced the exact behavior, however, perhaps I got the date wrong: I see more edge-case type behavior on the (6E 06-14) 5-Minute candle closing at 05:00 on 4/30/2014. And I can explain it away now i believe:

In my indi, I capture Last events in OnMarketData, and store data in a custom class -- as I said, this includes updating an OHLC based on every incoming Last event. I store bar data in a (Generic) dictionary, hashed by bar ID; and I am calculating the bar ID in OnMarketData with this code:

Bars.GetBar(evt.Time)

In my implementation, I am seeing that this can cause data to be stored in bars sometimes differently than what Ninja's bars would be. I don't know how to rectify that (I WOULD be interested to know); but in the end I won't care: it will not matter to me since it will only happen on that edge where this data is bringing back the prior bar instead of the new one -- all the data is still getting stored, just those edge cases in adjacent bars; and my indi will not be affected by this. (It certainly does potentially affect the OHLC -- and that's what I can see clearly in cases like the above-mentioned bar; since the new data is coming in from what Ninja would add to the new bar, but winding up in the prior bar, and causing it's high to be higher than Ninja's. -- But that's just a candlestick edge issue: unless your data needs to line up with some other, it's not a definitive science as to which ticks those bars need to precisely close and open on.)

To recap back to the initial "bug" I had: in exactly a case like this, what would seemingly happen is this first data of the new bar would come in while Plot was executing; and my indi stores it in the prior (still plotting) bar, but some sync issue arises: here's the pseudo-code:

Plot() {

//loop the plotted bars:
for (int barID = ChartControl.FirstBarPainted; barID <= ChartControl.LastBarPainted; barID++) {

... collect pre-draw data

... Meanwhile the next event hits OnMarketData
... My class winds up storing this in the barID bar
... -- but Ninja actually has closed and opened a new bar for this data

... get OHLC using ninja's High[CurrentBar - barID]
... and Low[CurrentBar - barID]
... plot the OHLC-based data

... Plot goes on to:
... Plot price-based data
... for the prices found in my custom class for barID
... (this includes the new event(s) from OnMarketData
... MY data does not line up since the OHLC used above is out of sync with the now-current price data

... This is not EXACT but with more notations below: ...
}
}

So I can't exactly say that it is truly a thread lock issue, but the basic cure was achieved because (what may seem obvious) no matter when the incoming market data arrives, if my class updates it's own OHLC AND price data, then at least on the next invocation of Plot, the updated data will be read and plotted.

In my original implementation, the OHLC gotten from CurrentBar was seeming to return the NEW BAR's OHLC -- once that new data event arrived; and that's what was staying out of sync on the plot.