Threading exception

[ETFVoyageur]

How much does my indicator need to be aware of threading, assuming I'm not explicitly doing anything multi-thread? My indicator just executed this line (s is a non-null string):

Output.Process(s, PrintTo.OutputTab1);

and then things went quiet for a humanly-obvious amount of time. The next thing that happened was that I got

An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in WindowsBase.dll

All that Visual Studio shows on the stack above my code line is "[External Code]". The basic thing my code was doing was processing a property set() function, called from another property set() function, at the time.

I think this is quite reproducible -- I'm working on verifying that.

Anyway, the bottom-line question is whether there is something my code needs to do with respect to thread safety. A second question is whether there is anything obvious I could be doing that would provoke this behavior.

--EV

[sledge]

Again, does it happen without VS attached?

[ETFVoyageur]

This appears to be a timing problem somewhere. Doing the same thing sometimes causes a problem and sometimes not. I have been changing a little code in an expandable property. As far as I can tell, exercising my code seems to provoke the problem (at least sometimes). It seems clear that either I have a bug, or else I am provoking some bad behavior by NT. So far I have no opinion which it is. The issue is the word "sometimes". I can come up without VS -- sometimes that works fine, but other times it dies at various places along the way.

• Sometimes NT dies coming up.
• Sometimes it dies bringing up the Indicators dialog
• Sometimes it dies when I double-click my indicator to add it to the list of indicators for the chart
• Sometimes it dies when I configure the indicator
• Sometimes it dies when I click the Apply button
• Sometimes it does not die at all -- sometimes it runs just fine.

I have also tried to run it under VS and here is the most common scenario:

• I get to my breakpoint
• I single step
• Everything seems to hang -- I am not in the debugger, because there is no call stack showing, and NT is not responsive
• After waiting a little bit I get the exception with the following stack

[External Code]    
>
NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaPropertyBase.PrintPropertyEntry(string getSet, string message, string property) Line 138    C#
     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaPropertyBase.PrintGetEntry(string message, string property) Line 127    C#

     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaMA.MA_Period.Length.get() Line 598    C#
     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaMA.MA_Period.ToString() Line 543    C#
     Evaluation of: Unknown evaluation    
     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaMA.MA_Period.Length.set(int value) Line 610    C#
     [External Code]

Here is the code (in Length.Set()) I am executing at the time:

                    double len = value;
bpt ==>  switch (ma_PeriodType)                                     // Convert to the equivalent number of bars
                    {
                        case MA_PeriodTypeEnum.Bars:
                            break;
                        case MA_PeriodTypeEnum.Days:
                            len = len * 21 / 30.5;
                            break;
                        case MA_PeriodTypeEnum.Weeks:
stack top ==>      len = len * 7 * 21 / (365.25 / 12);
                            break;
                        case MA_PeriodTypeEnum.Months:
                            len = len * 21;
                            break;
                        case MA_PeriodTypeEnum.Years:
                            len = len * 21 * 12;
                            break;
                        default:
                            throw new InvalidOperationException($"Unexpected Period type: '{Type}'");
                    }

The line at the top of the stack should not have any side effects. The fact that there is something unknown (Evaluation of: Unknown evaluation) and then a call to the same property's ToString() method is bizarre (translation: I do not understand it).

Illustrating the variability, I just tried the above again. This time, as I was about to configure the indicator (but before I had done anything) NT just vanished.

--EV

[ETFVoyageur]

Again, does it happen without VS attached?

We crossed in the mail.

the answer is yes, it does -- with varying symptoms and no diagnostics other than Windows saying NT is not responsive.

[ETFVoyageur]

I just tried again and got a different behavior -- I double-clicked my indicator in the Indicators dialog and immediately got the following message:

Unhandled exception: Index was out of range. Must be non-negative and less than the size of the collection.

This sounds similar to the ConstantLines message I reported recently, except that this message does not identify any specific miscreant. That was the first time I have seen that symptom.

Clarifying my description from the previous message, when I single-step from the switch statement, I get to the proper case, but only after VS puts up its "I'm waiting" little blue circle for some seconds. Single-stepping again, from the line setting "len" gets the same exception:

An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in PresentationFramework.dll

Although behavior varies, if I get to the breakpoint at the switch I usually get the exception after waiting a variable number of seconds. The exception is always the same type and the last of my code on the stack is always the same line, followed by "[External Code]"

Output.Process(s, PrintTo.OutputTab1);

In other words, this case does not always happen, but when it does it always ends the same way.

I do not know whether or not this is the same issue I have previously reported, but whether it is or is not this case is far more repeatable.

--EV

[ETFVoyageur]

To eliminate the code in that set() from being the cause, I commented it all out except the final standard vanilla assignment.

I added my indicator, and was able to do the configuration that often fails. No problem. I then added a second instance of the indicator and when I went to do that configuration NT just vanished.

And, before Sledge asks, I tried again with no VS involved and NT still vanished (sometimes).

As I have said, the way the symptoms vary from run to run makes me worry that it is a timing / threading problem. That is reinforced by the fact that the exception I do get to see is always a threading exception.

--EV

[ETFVoyageur]

I found a bug in my Clone() processing, and thought that could be the problem -- NT handling cloned objects on different threads, not expecting interaction. I fixed it and ran a few time with no VS and no problems. Too good to last. The last time I got the now-familiar Windows message "NinjaTrader has stopped working".

Back to using VS -- same issues as before. I waited over 2 minutes wall clock time, then had VS "Break ALL". Found I was at the usual exception statement. Continued and soon got the exception. What's the chance that whatever that code calls is looping chewing up resources until something finally breaks?

I don't know that I can take this much further. What I could do to give you more useful information is not obvious to me. Looking forward to hearing from NT folks in the AM.

--EV

[NinjaTrader_PatrickH]

Hello ETFVoyageur,

Thank you for your notes.

When you receive the message "An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in WindowsBase.dll" and "An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in PresentationFramework.dll" does the platform stop responding or Visual Studio?
I am not sure this is specific enough to point to your code or to NinjaTrader 8.

In each case where the behavior is different but it ends the same way, are you seeing "Evaluation of: Unknown evaluation" in all cases?

As each item is different, do you consistently see "Unhandled exception: Index was out of range. Must be non-negative and less than the size of the collection." when selecting that same indicator? Or does this only occur randomly as well?

May we test the indicator on our end? And if so can you detail which lines exactly are commented out in the varying version?

When you ask "What's the chance that whatever that code calls is looping chewing up resources until something finally breaks?", what code are you referring to? The code to duplicate the indicator?

[koganam]

This appears to be a timing problem somewhere. Doing the same thing sometimes causes a problem and sometimes not. I have been changing a little code in an expandable property. As far as I can tell, exercising my code seems to provoke the problem (at least sometimes). It seems clear that either I have a bug, or else I am provoking some bad behavior by NT. So far I have no opinion which it is. The issue is the word "sometimes". I can come up without VS -- sometimes that works fine, but other times it dies at various places along the way.

• Sometimes NT dies coming up.
• Sometimes it dies bringing up the Indicators dialog
• Sometimes it dies when I double-click my indicator to add it to the list of indicators for the chart
• Sometimes it dies when I configure the indicator
• Sometimes it dies when I click the Apply button
• Sometimes it does not die at all -- sometimes it runs just fine.

I have also tried to run it under VS and here is the most common scenario:

• I get to my breakpoint
• I single step
• Everything seems to hang -- I am not in the debugger, because there is no call stack showing, and NT is not responsive
• After waiting a little bit I get the exception with the following stack

[External Code]    
>
NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaPropertyBase.PrintPropertyEntry(string getSet, string message, string property) Line 138    C#
     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaPropertyBase.PrintGetEntry(string message, string property) Line 127    C#

     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaMA.MA_Period.Length.get() Line 598    C#
     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaMA.MA_Period.ToString() Line 543    C#
     Evaluation of: Unknown evaluation    
     NinjaTrader.Custom.dll!NinjaTrader.NinjaScript.Indicators.VsaMA.MA_Period.Length.set(int value) Line 610    C#
     [External Code]

Here is the code (in Length.Set()) I am executing at the time:

                    double len = value;
bpt ==>  switch (ma_PeriodType)                                     // Convert to the equivalent number of bars
                    {
                        case MA_PeriodTypeEnum.Bars:
                            break;
                        case MA_PeriodTypeEnum.Days:
                            len = len * 21 / 30.5;
                            break;
                        case MA_PeriodTypeEnum.Weeks:
stack top ==>      len = len * 7 * 21 / (365.25 / 12);
                            break;
                        case MA_PeriodTypeEnum.Months:
                            len = len * 21;
                            break;
                        case MA_PeriodTypeEnum.Years:
                            len = len * 21 * 12;
                            break;
                        default:
                            throw new InvalidOperationException($"Unexpected Period type: '{Type}'");
                    }

The line at the top of the stack should not have any side effects. The fact that there is something unknown (Evaluation of: Unknown evaluation) and then a call to the same property's ToString() method is bizarre (translation: I do not understand it).

Illustrating the variability, I just tried the above again. This time, as I was about to configure the indicator (but before I had done anything) NT just vanished.

--EV

I would be surprised if that is really a critical section, but have you tried placing a lock? (Is it time to start grasping at straws?)

[ETFVoyageur]

Hello ETFVoyageur,

Thank you for your notes.

When you receive the message "An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in WindowsBase.dll" and "An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in PresentationFramework.dll" does the platform stop responding or Visual Studio?

NT stops and VS reports the exception to me. When I continue, NT vanishes.

Note that when I run NT without VS there are two different behaviors -- sometimes NT just vanishes. Other times Windows reports NT is no longer responding. That would be consistent with what I see with VS -- sometime NT vanishes and sometimes it hangs, followed by an exception.

I am not sure this is specific enough to point to your code or to NinjaTrader 8.

I have no current opinion on that, other than that my code seems to be involved somehow. I have no opinion on whether I have a bug, or whether my code is causing a situation that reveals a bug in NT.

In each case where the behavior is different but it ends the same way, are you seeing "Evaluation of: Unknown evaluation" in all cases?

I see that in all cases that end in the threading exception, which one of the more common cases.

As each item is different, do you consistently see "Unhandled exception: Index was out of range. Must be non-negative and less than the size of the collection." when selecting that same indicator? Or does this only occur randomly as well?

No, I saw that only once. That message struck me because that is so similar to the "ConstantLines" bug I reported. I am highly suspicious that both of the exception-based bugs I have reported recently ("ConstantLInes indicator exception" and "How can this get an exception") are the same as this one. This one has the advantage that it is much more reproducible.

May we test the indicator on our end? And if so can you detail which lines exactly are commented out in the varying version?

Absolutely -- I'd like you to. Note that the lines I commented out made no difference -- all that accomplished was to prove that they are not somehow the bug, so I uncommented them.

When you ask "What's the chance that whatever that code calls is looping chewing up resources until something finally breaks?", what code are you referring to? The code to duplicate the indicator?

I am referring to NT8 -- whatever system code is on the stack below my "Output" call. I notice that when in this situation things appear to hang, but if you have patience you eventually get the exception. Something is either looping or else waiting for something long enough that it is glaringly obvious on a human timescale.

=====

As I recently mentioned, Tools | Export | NinjaScript will not work for me. What is in the zip file that would have been created? Is it just the source files, or is there something else? I can manually zip up the files I believe you need, but you may have to compile them in Visual Studio, not NT. I'll wait to hear from you in case you need something other than a consistent set of source files.

--EV

[ETFVoyageur]

I would be surprised if that is really a critical section, but have you tried placing a lock? (Is it time to start grasping at straws?)

I have not tried locking anything. I am about to read the documentation about multi-threading to see whether there is something my indicator is supposed to be doing that I am not doing. That's why my original post asked whether there are any multi-threading issues that indicators need to allow for.

--EV

[ETFVoyageur]

I just tried again, to get another piece of data. On my first attempt, NT just vanished. On my second attempt NT hung. I verified that VS was still functional, but NT was non-responsive. I waited approximately 3 minutes (wall clock time) and then then got

An unhandled exception of type 'System.Threading.ThreadAbortException' occurred in WindowsBase.dll

As always in this particular situation, the last line of mine on the stack was

Output.Process(s, PrintTo.OutputTab1);

The sort of thing I keep wondering about is whether that code is a problem with respect to multi-threading. For example, is that something that should be using Dispatcher? (If so, what should the code look like?)

The three minutes is why I earlier suggested that something was either looping or else waiting for something.

My basic concern is that my indicator needs to be doing something special now that NT is multi-threaded, and that I am not doing whatever is needed. From what the documentation says, NT would be multi-threaded on my machine (i7 quad-core).

--EV

[koganam]

I just tried again, to get another piece of data. On my first attempt, NT just vanished. On my second attempt NT hung. I verified that VS was still functional, but NT was non-responsive. I waited approximately 3 minutes (wall clock time) and then then got
As always in this particular situation, the last line of mine on the stack was

Output.Process(s, PrintTo.OutputTab1);

The sort of thing I keep wondering about is whether that code is a problem with respect to multi-threading. For example, is that something that should be using Dispatcher? (If so, what should the code look like?)

The three minutes is why I earlier suggested that something was either looping or else waiting for something.

My basic concern is that my indicator needs to be doing something special now that NT is multi-threaded, and that I am not doing whatever is needed. From what the documentation says, NT would be multi-threaded on my machine (i7 quad-core).

--EV

All of which raises the suspicion that there may be an indeterminate switch away from the thread while it is executing, hence my suggestion of using a lock, treating it as a critical section.

[NinjaTrader_PatrickH]

As I recently mentioned, Tools | Export | NinjaScript will not work for me. What is in the zip file that would have been created? Is it just the source files, or is there something else? I can manually zip up the files I believe you need, but you may have to compile them in Visual Studio, not NT. I'll wait to hear from you in case you need something other than a consistent set of source files.

Exactly, just zip the source .cs file itself.

[ETFVoyageur]

Patrick,

I just sent email to your attention. I zipped up the toy indicator I have been using to create the problem along with the other files you will need to compile it.

In the email I describe exactly how to demonstrate the bug. The description is using Visual Studio, but the bug happens even without VS. Feel free to investigate without VS if that is easier for you.

--EV