Multi-DataSeries throws exception with tick replay

[aslane]

When adding multiple data series to an indicator, it fails throwing an Object reference not set exception when running on a chart with TickReplay turned on. When tick replay is turned off, it runs fine.

The attached trivial indicator shows the issue. Apply it to a 5-min chart of ES 03-17, with 5 days of data loaded, and tick replay turned on.

When data is loaded, it works for the first N bars (approx 10K), and then it will throw the exception on all subsequent bars. I have a try catch block printing out a msg, so there is no actual exception raised into the log, but you can breakpoint on the print.

It almost seems like it ran out of memory or there is an internal limit as I for sure did not run out of memory on my machine.

This is using 8.0.3.1, using Continuum

[NinjaTrader_ChelseaB]

Hello aslane,

Adding TickReplay and changing the Calculate to either .OnEachTick or .OnPriceChange will cause the script to trigger OnBarUpdate differently.

Are you ensuring that the indexes you are using exist before attempting to use them?

For example
if (CurrentBars[BarsInProgress] > 0)
Print(Times[BarsInProgress][0].ToString());

[aslane]

Did you even look at the code or try it?

Here is the only line of code

DateTime temp = Times[BarsInProgress][0];

Like I said, this works fine until it gets to larger bar numbers. Always fails with BarsInProgress == 1 or 2.

[NinjaTrader_ChelseaB]

Hello aslane,

I did look at the script and I am not seeing any code to prevent the script from using an index that does not yet exist.

This is why I have mentioned this in my previous post.

May I confirm you have adjusted the code to prevent any indexes from being called before they exist and this did not correct the behavior?

[aslane]

As originally stated, it fails after approx 10K bars, so that is not an issue. However, to cover all bases, if you change the code to

            try
            {
                DateTime temp = DateTime.MinValue;
                
                if (CurrentBars[BarsInProgress] > 0)
                    temp = Times[BarsInProgress][0];
            }
            catch
            {
                Print("Bad object ref, BarsInProgress = "+BarsInProgress+" numBars = "+Times[BarsInProgress].Count);
            }

It fails in exactly the same fashion.

When it fails, the count on the Times[] array is very large, but when you go to access the [0] member it fails internally with the exception.

[aslane]

Anything on this?

This is easy to debug on your side, just comment out the try/catch block, and you will take an exception.

[NinjaTrader_ChelseaB]

Hello aslane,

I am not able to reproduce this behavior.

Attached is a test script I have used to test.

Below is a link to a video of the test.

https://www.screencast.com/t/Cw8plglg

I've added a print to ensure that at least 10,000 bars are appearing for all series.

Please test the script I have created and let me know if you are able to reproduce the error using this script.

[aslane]

Good news and bad news.

Your script and my script both are now working fine (with and without the CurrentBars check). My underlying complex script, which originated my investigation into the issue, is also working with no changes. I changed nothing on my end, had same workspace, data, etc. The only item different is this is a new instance of NT (fresh run).

This is great, but is rather concerning since there is clearly a transient issue in NT. I will post back if it returns.

[aslane]

Sadly, clean restart this AM, and the problem is back. This happened on initial data load. Your indicator takes an exception. Here is the tail of the output.

BIP: 0, Bar#: 1037, time: 1/17/2017 3:00:00 PM
BIP: 1, Bar#: 69679, time: 1/20/2017 2:59:58 PM
BIP: 2, Bar#: 69679, time: 1/20/2017 2:59:58 PM
BIP: 0, Bar#: 1037, time: 1/17/2017 3:00:00 PM
Indicator 'SecondarySeriesTickReplayTest': Error on calling 'OnBarUpdate' method on bar 69680: Object reference not set to an instance of an object.

My study takes the exception also, and the CurrentBars check makes no difference (above output was with check in place though), as you are well into the data stream when the exception happens. If you reload NinjaScript (F5), it fails again in same spot. When I shut down NT and restarted it, the workspace was unable to restore your indicator (because it took exception?), but when I reapply the study it failed again in same spot.

[NinjaTrader_ChelseaB]

Hi aslane,

I would like to request some sensitive information from your log and trace.

Please send an email to platformsupport [at] ninjatrader [dot] com. In the email, please include a link to this forum thread.

[aslane]

I sent trace/log files via email.

I think this may have something to do with having multiple workspaces with TickReplay data. I am not 100% sure on this, as when I try to build simple workspaces to reproduce I do not always see the issue. However, when I only have a single workspace open, I do not recall seeing the issue.

Let me know if there is more info that you need.

[aslane]

After going the email route, this problem remains unsolved. It is also not being submitted to development by support because it "can not be reproduced", so clearly it is not considered a real issue.

The problem is very difficult to reproduce in a reliable manner, and this is because may be threading/timing related. It is a real issue though, as I see it over and over again in my normal environment.

The issue is internal to NT, in that a Series object becomes corrupted. Every time I have trapped/debugged this issue, the Series object (i.e. Times) has a valid Count > zero and the CurrentBars object has valid values, but the access operator (i.e. Times[BarsInProgress][0]) fails with an exception.

At this point, I am chalking this up to another NT issue that has been submitted and abandoned. I am starting to lose track of how many of these have been sent in.

Personally, I am going to go back to using SierraChart for anything that requires Multi-DataSeries and TickReplay until NT stabilizes.

[NinjaTrader_ChelseaB]

Hello aslane,

You are correct that to report an issue to our development, we must have reproducible steps, otherwise the development will not be able to reproduce and correct the behavior.
(They would be chasing a ghost without reproducible steps)

However, we do want to get to the bottom this, and there is a process for doing so.
The process is to start with the script and test this completely by itself. If the script when running in a clean test environment is able to reproduce the behavior we can immediately report this.
This is not the case though. To move forward we need to rebuild the workspace one item and one change at a time until the behavior returns. The last step taken may be the step needed to reproduce the behavior and this can be reported to our development.

Basically, we cannot report that there may be a issue without providing any steps to reproduce the issue.

Please let me know if you would like to continue troubleshooting.

[Harry]

Hello Chelsea,

I am having the same problem here as Aslane. I have coded a Multi-Data-Series indicator that works fine with live data, but it throws exceptions with tick replay activated. The indicator adds two secondary bar series

-> BarsArray[1] holds minute bars
-> BarsArray[2] holds daily bars

I have tracked down the problem to the use of GetBar().

If I use GetBar() with BarsArray[1], it works

if(CurrentBars[1] >= 0)
{    
    currentHistoricalHigh = BarsArray[1].GetHigh(BarsArray[1].GetBar(Times[0][0]));
    currentHistoricalLow = BarsArray[1].GetLow(BarsArray[1].GetBar(Times[0][0]));
    currentHistoricalClose = BarsArray[1].GetClose(BarsArray[1].GetBar(Times[0][0]));
}

However, if I use GetBar() with BarsArray[2], it throws an exception:

if(CurrentBars[2] >= 0)
{    
    currentHistoricalHigh = BarsArray[2].GetHigh(BarsArray[2].GetBar(Times[0][0]));
    currentHistoricalLow = BarsArray[2].GetLow(BarsArray[2].GetBar(Times[0][0]));
    currentHistoricalClose = BarsArray[2].GetClose(BarsArray[2].GetBar(Times[0][0]));
}

The exception thrown is as reported by Aslane:

"Error on calling 'OnBarUpdate' method on bar 111: Object reference not set to an instance of an object."

The issue is reproducible on my side. The funny way of catching daily data is necessary because of a design flaw / bug, as NinjaTrader cannot process yesterday's daily bars on historical data. This piece of code has also been used for the Pivots indicator that ships with NinjaTrader 8.

Basically, I have adapted the code from the pivots indicator, because there is no other option. When using the code with minute data it works, but when used with daily data it fails!

Any help please?

[Harry]

I have now further tested the issue.

The code which was used

currentHistoricalHigh = BarsArray[2].GetHigh(BarsArray[2].GetBar(Times[0][0]));
currentHistoricalLow = BarsArray[2].GetLow(BarsArray[2].GetBar(Times[0][0]));
currentHistoricalClose = BarsArray[2].GetClose(BarsArray[2].GetBar(Times[0][0]));

works fine for all chart bars with the exception for today. For today GetBar() throws an exception. The very bar 111 for which I got the error message

"Error on calling 'OnBarUpdate' method on bar 111: Object reference not set to an instance of an object."

is today's very first bar. The question here is why in my case this call throws an exception, while in other cases it works.

I think that the problem is related to another problem of today's daily bar not being processed ...