Bizarre null reference exception in Market Replay

[traderpards]

When you run a background data series and you want to check the Time series object for that series, it mostly works, until you get to the very last bar when loading a chart.

As you can see from the screen shot, you apparently cannot call Bars.IsFirstBarOfSession because for some reason, that guy is out of scope... Or is it? You should probably make sure you're on the correct bar series you want but should it throw a null reference exception if you aren't? I don't see why.

This issue first became apparent on the Times object. The screen shot shows you that too - same crash - NullReferenceException.

Load the strategy on a range chart - I used 18 range, 3 days, CL 04-17 - in a market replay connection and it may crash for you, or it should because it crashes on me every time. I'm not ever crashing in a live connection environment, just MR. (Of course, I had it on a bar that was just right to expose the crash.)

You could have some trouble reproducing this. If you run through about three weeks of market replay and it doesn't do it to you, then you'll need to restart the MR, restart the strategy and run it. If it runs, then let it go a few bars and stop the MR, restart the strategy and sooner or later - hopefully sooner - you'll hopefully land on a bar that hasn't the foggiest clue about the Time object for the background data series.

[NinjaTrader_Jim]

Hello traderpards,

Thanks for reporting this issue to us.

We will be looking into this matter further on our end as the bars should be created when you are calling OnBarUpdate() and you should not be seeing exceptions in the Time objects or by calling Bars.IsFirstBarOfSession.

In the meantime, if there is any specific test case scenario that you can elaborate on for us, it would be greatly appreciated. I noticed you mentioned you were using 3 weeks of data using Market Replay. Is this more reliably reproducible with market replay data or historical data, what specific time period should we be looking at?

We look forward to being of further assistance.

[traderpards]

No, there isn't any specific test case that makes this happen. I was running my strategy and had a problem so I stopped the strategy, fixed the problem (I think) and then restarted the strategy while the MR was paused. That's when the problem exposed itself. I noticed that I could run the MR sometimes for a few bars and restart the strategy and it would load like it's supposed to - it wouldn't crash. Then I would go on my merry way testing. And then I found that going a few bars wouldn't work. It would throw that exception. It's semi-random but if you have enough patience, you'll find a bad bar. I can make it do this rather quickly - less than 10 minutes. I'm in a state where I can't even run MR at all for CL because I can't get it started without it throwing that exception. It works fine at the beginning of the period from Feb 20th, but if I stop the MR for any reason, on seemingly any bar, I'm screwed.

The developers probably aren't going to like it much because there isn't a concrete set of steps you have to follow. You have to play with it to find a bar that makes it throw that exception and you do that by letting it run a few bars, stopping the strategy, re-starting the strategy and it will crash (if you've found a bad bar) on the last bar that it needs to load. But once you find that bar, it will crash every time. So that's the good news. The MR is in a paused state when you find that bad bar so you can debug it there as many times as you need to and see what the problem is.

I haven't tried it using historical, just market replay. I started using CL on February 20th and had those three weeks, not including this week. GC did it to me too.

[traderpards]

Did it to me on a live feed

Well, you're in luck. This happened last night on a live feed - EMD, June. I switched over to the March contract and there is no issue at all.

2017-03-15 07:15:22:396 ERROR: Indicator 'TSI': Error on calling 'OnBarUpdate' method on bar 432: Object reference not set to an instance of an object.
2017-03-15 07:15:22:398 ERROR: Indicator 'EMA': Error on calling 'OnBarUpdate' method on bar 432: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2017-03-15 07:15:22:398 ERROR: Indicator 'TSI plus': Error on calling 'OnBarUpdate' method on bar 432: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2017-03-15 07:15:23:247 ERROR: Indicator 'Momentum': Error on calling 'OnBarUpdate' method on bar 432: Object reference not set to an instance of an object.
2017-03-15 07:15:23:247 ERROR: Indicator 'EMA': Error on calling 'OnBarUpdate' method on bar 432: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2017-03-15 07:15:23:247 ERROR: Indicator 'Momentum': Error on calling 'OnBarUpdate' method on bar 432: Object reference not set to an instance of an object.
2017-03-15 07:15:23:250 ERROR: Indicator 'SMA': Error on calling 'OnBarUpdate' method on bar 432: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2017-03-15 07:15:23:251 ERROR: Indicator 'Std. dev.': Error on calling 'OnBarUpdate' method on bar 432: You are accessing an index with a value that is invalid since it is out-of-range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
2017-03-15 07:15:23:252 ERROR: Failed to call 'Add' method: System.NullReferenceException: Object reference not set to an instance of an object. at NinjaTrader.Data.BarsSeries.GetClose(Int32 index) at NinjaTrader.Data.Bars.GetClose(Int32 index) at NinjaTrader.NinjaScript.NinjaScriptBase.Update(Int 32 idx, Int32 bip) at NinjaTrader.NinjaScript.NinjaScriptBase.<>c__Displ ayClass448_2.<TickReplayOrUpdate>b__2(Object s, BarsUpdateEventArgs o) at NinjaTrader.Data.BarsSeries.Add(Bars bars, Double open, Double high, Double low, Double close, DateTime time, Int64 volume, Double tickSize, Boolean isBar, Double bid, Double ask)
2017-03-15 07:20:31:453 Cbi.Instrument.RequestBars (to Provider): instrument='EMD 06-17' from='3/15/2017 5:00:00 AM' to='3/15/2017 11:59:59 PM' period='1 Tick'

The difference is, this is on an indicator that is basically taking your TSI and adding a moving average to it. It doesn't care about the Time object.

More good news. I just loaded the data for this week because I suspected that you could go to that first bar of today's session and it would generate the exception. So I tried it and it did! I stopped the MR at around 4:10 (Mountain) and loaded the strategy and it crashed.

So there's something you can try. (EMD)

The thing I might mention is the chart that is running has Tick Replay enabled. The strategy I sent you I didn't have tick replay enabled because I could generate the error without it. It doesn't make sense to me that would matter but I just thought I would mention it.

I was hoping to start my strategy this morning, thinking we've gotten passed the bar causing my woes but that didn't work. It still threw the exception.

Update (about an hour later after first trying to load my strategy.) It loads. See what I mean about a bad bar? I would guess not only was that the wrong time to load my strategy last night, but also when I tried it this morning it was apparently the wrong time. Just now was the right time. This mirrors exactly my experience in the MR except now it happened for the first time in a live feed.

[traderpards]

More...

The second bar of the session is also bad - it will generate the error. However, my strategy loads on the third bar. If I would have known I could have loaded my strategy last night at 10:48 and I would have been fine.

[NinjaTrader_Jim]

Hello traderpards,

So I am following correctly, could you confirm my understanding?

We will be able to recreate this crash by...

1. Downloading Market Replay data for EMD 06-17 for the past week (3/08/2017-3/15/2017)
2. Stopping Market Replay at 4:10 Mountain time
3. Loading your Strategy

Could you provide a screen shot of your Strategy Properties so we can reproduce these scenarios verbatim? These can be found by right clicking the in your chart and selecting Strategies. The properties we are looking for are in the right panel of the window.

Could you also elaborate more on "second bar of a session is also bad?" Is this particular to the EMD 06-17? Which specific instrument(s) and expiry/expiries did this occur with? Is there any other information on this occurrence that you could detail for me?

I look forward to your reply.

[traderpards]

Hello traderpards,

So I am following correctly, could you confirm my understanding?

We will be able to recreate this crash by...

1. Downloading Market Replay data for EMD 06-17 for the past week (3/08/2017-3/15/2017)
2. Stopping Market Replay at 4:10 Mountain time
3. Loading your Strategy

Could you provide a screen shot of your Strategy Properties so we can reproduce these scenarios verbatim? These can be found by right clicking the in your chart and selecting Strategies. The properties we are looking for are in the right panel of the window.

Could you also elaborate more on "second bar of a session is also bad?" Is this particular to the EMD 06-17? Which specific instrument(s) and expiry/expiries did this occur with? Is there any other information on this occurrence that you could detail for me?

I look forward to your reply.

Yes, you should be able to recreate that crash by following those three steps - you do understand. It works for me, each time and I tried it twice on two different strategies - the one I sent you and the one I really use. They both generate the crash.

The second bar of the session means just that - if you miss the first one, which is unlikely because you have several hours to stop the MR, then the second bar will also produce the crash. That's what I mean by "bad." If you miss that one for whatever reason, don't because loading your strategy on the third bar will not recreate the crash. I don't know if it's a "bad bar" that is causing this. I just know this manifests itself on certain bars and it's reproducible on those bars. In the case of EMD, it crashed on the first bar of the session both live and in market replay. It also crashed on some bar in the morning and I bet it would in MR if I could figure out which bar it was. Since I can't, just use the first bar of today's session, which was yesterday afternoon around 4:00 pm.

I don't know of any other circumstances. As I told you in my first and second posts, you would have had to play with it to find a bar that generates the crash. But now you don't need to do that - just follow the exact steps as you understand them - it seems to me you understand perfectly. You also don't need a screenshot of the properties settings since the defaults are what I used. Just load it and run it! (I've learned to make anything I send you run straight "out of the box" without having to monkey around with the properties.)

And since you're curious, I first crashed in MR with GC 04-17, then CL 04-17 and now live and MR with EMD 06-17.

[NinjaTrader_Jim]

Hello traderpards,

I have been trying to reproduce this issue on my platform using the instructions provided, but I have been unable to reproduce thus far.

Can you confirm that is is happening with the current version of NinjaTrader 8? The current version number is 8.0.5.2. The platform can be updated at http://ninjatrader.com/PlatformDirect

Could provide your response along with the Log and Trace files of this occurrence? Please provide me timestamps of when the crash occurs so I can more easily read your log and trace files.

Please send me your log and trace files so that I may look into what occurred.

You can do this by going to the Control Center-> Help-> Mail to Platform Support.

Ensuring 'Log and Trace Files' is checked will include these files. This is checked by default.

Please reference the following ticket number in the body of the email: 1676191

I would then like to continue correspondence through email. (You may also manually attach your Log and Trace files to an email and send them to platformsupport[at]ninjatrader[dot]com with the above ticket ID.)

I look forward to working with you further on this matter.

[traderpards]

I didn't realize you had yet another version out since it's usually a month before you get a new one. I have subscribed to that thread but I never get notices about new releases although I do get them on other threads I subscribe to. So the only way I know there's a new release out is when someone puts a bug report on the forum and mentions the new release number. This was produced with version 5.1.

I installed 5.2 and will wait for it to happen again. If it does, I'll email you the trace and log files and see if you can see anything.