Execution sequence for my indicator -- need help

[ETFVoyageur]

I am in the VS debugger right now, at a breakpoint at the entry to OnStateChanged() and one of the class values is not what it should be. That prompted me to look at the stack, which has me quite puzzled. Here is what is on the stack:

• OnStateChanged() with State = State.SetDefaults
• [External Code]
• My own indicator base class constructor -- just entering it, no code yet executed
• My indicator's default constructor -- no code run yet, because the initializer call to the base class constructor has not yet returned
• [External Code]

What I do not understand is how OnStateChanged() can be getting called before the constructor has completed. There is no way that should be happening. No object can be assumed ready to be called before its constructor has completed.

I cannot see how this is happening unless it is some sort of multi-threading thing. I certainly was not aware that I had to ensure my indicator instance is thread-safe, and I hope that is not the case.

So ... please ... someone tell me how OnStateChange() is getting called before the constructor completes! That should not be happening!
=====
I got here by:

• Starting Ninatrader 8. It came up in a workspace with one chart that had no indicators. No sign of any problem so far.
• Visual Studio attach to the process. That went cleanly, too.
• Bring up the Indicators dialog on the chart. As I understand it, NT generates an instance of each indicator during that process.
• Hit the only breakpoint I have set -- at the entry to OnStateChange()
• Noted the stack as described above -- constructors had been called, but not yet done anything.

At that point I verified a couple of things

• I keep an internal object instance serial number, set by variable initialization that is done prior to the constructor. It was set to "1", consistent with this being the first instance created.
• I keep an internal state. It is compile-time initialized to "not constructed" and then updated at runtime as appropriate. It was still saying "not constructed", even though I am sitting at a breakpoint is OnStateChange(). That is consistent with what the stack is telling me -- that my constructors had not yet had a chance to construct anything. (The very first thing the constructor does is set that internal state to "being constructed".)

All I can say is that from all I am seeing the situation should not be possible. My internal state is consistent with what the stack is telling me -- that I am in OnStateChanged() before the constructor has run. Not Nice!

FWIW: this is not a one-off freak -- I tried again and the same thing happened.

--EV

[ETFVoyageur]

To concisely summarize my question:

• External code calls my indicator's default constructor. Good so far.
• The indicator's constructor has my indicator base class constructor as the only item on its initialization list, so the base class constructor gets called. This happens before any code runs in the indicator's own constructor. All is good so far.
• The next thing that should happen is the base class constructor executing. It has no initializer list. Nothing I have heard/read says the NT base class constructor expects to get called. In any case, in this example, there is no such call.
• But my base class constructor does not get to execute. Before it can, [External Code] shows up on the stack!

At the point where my base class constructor got called, assuming we are talking about single-threaded, no code other than my code should be running until after the constructors return. My base class constructor should run, and when it returns the indicator's own constructor should run. When that returns the system is free to do as it wishes -- including calling OnStateChange().

What I do not understand, again assuming a single-threaded model, is how any code but mine could run before the constructor even got a chance to execute.

--EV

[koganam]

To concisely summarize my question:

• External code calls my indicator's default constructor. Good so far.
• The indicator's constructor has my indicator base class constructor as the only item on its initialization list, so the base class constructor gets called. This happens before any code runs in the indicator's own constructor. All is good so far.
• The next thing that should happen is the base class constructor executing. It has no initializer list. Nothing I have heard/read says the NT base class constructor expects to get called. In any case, in this example, there is no such call.
• But my base class constructor does not get to execute. Before it can, [External Code] shows up on the stack!

At the point where my base class constructor got called, assuming we are talking about single-threaded, no code other than my code should be running until after the constructors return. My base class constructor should run, and when it returns the indicator's own constructor should run. When that returns the system is free to do as it wishes -- including calling OnStateChange().

What I do not understand, again assuming a single-threaded model, is how any code but mine could run before the constructor even got a chance to execute.

--EV

OnStateChange() would have to run if it is to know that State == State.SetDefaults, the very first state into which it will transition?

[ETFVoyageur]

OnStateChange() would have to run if it is to know that State == State.Default, the very first state into which it will transition?

I'm not sure I understand the comment. If you are asking how I knew the state, that is because I was at a breakpoint at the start of OnStateChange() and I looked at the state.

My problem is that I did not even belong in OnStateChanged() prior to the constructor having completed.

--EV

[NinjaTrader_Brett]

I agree with what you say but there is something thats being missed that we needed to find. Yes there is multi-threading going on here with NinjaTrader but I wanted to make sure I understand your scenario. Since this could be a C# detail you might be overlooking in terms of multiple constructors and calling the correct one or you you might have run into a multi-threading issue that I needed to understand since that may or may not be expected.

Can you please define the scenario succinctly with example/psuedo code. For example:

Class A
{
int myString;
A()
{
myString = "Class A";
}
}

Class B : A
{
B()
{
myString = "Class B";
}
}

class Indicator
{
B myBVar = new B();

OnStateChanged()
{
Print(myBVar.myString); // Prints "Class B" but I expected "Class A" for example
}
}


Once I have the exact scenario (like above) I could let you know right away whats expected / setup a simple test my side to reproduce easily enough.

[ETFVoyageur]

First thing to note is that there is good news -- this is utterly reproducible. I just did it again so i would have a live stack to look at while answering your request. Here's the basic code involved:

// External code calls the indicator's default constructor; the initializer call to base() is the only thing that runs
// "DbgLvl.EntryAndExit" is an enum value; so no code was executed to produce it
public VsaModelIndicator() : base(DbgLvl.EntryAndExit) { ... }

// Next the base class' constructor is called, because of the indicator constructor's initializer call to base()
// This stack line points at the constructor entry; nothing from the constructor got to run
protected VsaIndicatorBase(DbgLvl level) { ... }

// That's as far as it makes sense. Up to this point all went as expected.

// Next thing on the stack is external code; I have no idea how / why that even got a chance to run unless there is a multi-threading issue
[External Code]

// Then, called by the mysterious External Code, is the indicator's OnStateChanged()
// I have a breakpoint at the entry to OnStaeeChanged(), so anything it might do is moot
// The issue is that we should not get here while the constructor is still on the stack
protected override void OnStateChange() { ...}

===== the stack =====

3df04307054a492e95257838317bc190.dll!NinjaTrader.NinjaScript.Indicators.VsaModelIndicator.OnStateChange() Line 157    C#
[External Code]    
3df04307054a492e95257838317bc190.dll!NinjaTrader.NinjaScript.Indicators.VsaIndicatorBase.VsaIndicatorBase(NinjaTrader.NinjaScript.Indicators.VsaIndicatorBase.DbgLvl level) Line 268    C#
3df04307054a492e95257838317bc190.dll!NinjaTrader.NinjaScript.Indicators.VsaModelIndicator.VsaModelIndicator() Line 125    C#
[External Code]

I hope that answers your request.
--EV

[koganam]

I'm not sure I understand the comment. If you are asking how I knew the state, that is because I was at a breakpoint at the start of OnStateChange() and I looked at the state.

My problem is that I did not even belong in OnStateChanged() prior to the constructor having completed.

--EV

Not "How did you know it?": rather, "How would the NT code know it?"

The point is that the OnStateChange() event must run in order to query the state. That includes the first state that will be hit, State.SetDefaults. Unless OnStateChange() is run, the code cannot determine that it is in State.SetDefaults, and so to start configuring the defaults. No?

[ETFVoyageur]

I'm still not with you. Nothing in my indicator should be called before the constructor has returned. By definition the instance is not ready to be used until after it returns from its constructor.

--EV

[ETFVoyageur]

It occurred to me that I was not calling the NT base class constructor, so I tried adding that:

  protected VsaIndicatorBase(DbgLvl level) : base()

The only difference is adding " : base()". Doing so made no difference in the behavior -- still the same issue.

Any information on whether I should be calling the NT constructor is welcome. Intuitively, it seems as if I should, but I have never heard about doing so. That could just be because constructors are de-emphasized, though.

FWIW: mine does very little. Mainly just saving a perishable value, setting trace/debug state, and providing a more complete trace record so I can see the details of what is going on in my indicator. In NT7 there were a few things that needed to get done prior to the call to Initialize(), and the constructor was the only place to do them.

--EV

[NinjaTrader_Brett]

Understand thanks for posting and understand your scenario.

In this case the answer is simple and the behavior you see is expected.

In IndicatorBase constructor we call SetState(State.Defaults); Since the Indicator base constructor gets before any of your constructors (Since you inherit from it) this is why you see the State.SetState() call.

Here's the thing:

In principle your trying to set some defaults in the constructor. This is not the intended place to be setting defaults. Instead you should use State.SetDefaults. Since that's what its there for.

[koganam]

I'm still not with you. Nothing in my indicator should be called before the constructor has returned. By definition the instance is not ready to be used until after it returns from its constructor.

--EV

Not quite. Your indicator is part of a framework, or calling into a framework. The framework can and should prepare to call your code. How else is the framework to know what to do with its own internal processing unless it has already prepared itself to handle your code?

When the framework chooses to prepare itself is out of our hands, and the issue still remains: "How is the framework to know what state it is in, unless it runs code to determine such?"

Specifically,

• How is the framework to know when to SetDefaults, unless a part of the framework is already loaded and knows that the state is State.SetDefaults()?
• How else can the framework query the state other than by running SetState()/OnStateChange()? Well, that just looks like the simplest way: there might be others.

It is almost a chicken and egg problem, but not quite. If the framework is to run your indicator, then at least a part of the framework must be loaded in order to call your indicator. Think of how malware (not that NT is malware: far from it!) would act to take control of your code. Same principle.

[ETFVoyageur]

I am in very strong disagreement here. Show me anywhere reputable that says an instance method should be able to get called prior to the instance's constructor having returned. That just is not done. By definition, my instance is not expected to be ready to deal with the rest of the world until its constructor has run. That's just bedrock object programming. The NT constructor can do anything it chooses to do within its own code, but it has absolutely no business calling my code until after my constructor returns. Furthermore, in the case at hand, there is no logical need to do so. The premature call in question is just an artifact of a bad design.

Brett: In IndicatorBase constructor we call SetState(State.Defaults); Since the Indicator base constructor gets before any of your constructors (Since you inherit from it) this is why you see the State.SetState() call.

How you manage it is your problem, but you have no business calling my instance before its constructor has returned. You can call me right after that; how you wait that long is your problem.

Brett: In principle your trying to set some defaults in the constructor. This is not the intended place to be setting defaults. Instead you should use State.SetDefaults. Since that's what its there for.

No, I am not setting any defaults. I am setting a minor amount of instance state that OnStateChange() needs to run correctly. I set all default values in OnStateChanged(). I have every right to get the instance ready to run in the constructor. That's different from setting defaults, which I do in OnStateChanged(), as you recommend.

Kognam: Not quite. Your indicator is part of a framework, or calling into a framework. The framework can and should prepare to call your code. How else is the framework to know what to do with its own internal processing unless it has already prepared itself to handle your code?

Of course the framework should prepare to deal with my indicator. It has no business, however, calling anything in my indicator instance until after the indicator's constructor has returned. Doing so is a huge no-no!

Koganam: When the framework chooses to prepare itself is out of our hands, and the issue still remains: "How is the framework to know what state it is in, unless it runs code to determine such?"

I agree with that, but nothing says it is allowed to call my code before the constructor returns. It can prepare itself any way it wants to, but it cannot call my code prematurely. As to "How is the framework to know what state it is in, unless it runs code to determine such?" -- of course it has to run code -- its own code. And calling OnStateChanged() will do nothing to help NT know what the current state is anyway.

Koganam: How is the framework to know when to SetDefaults, unless a part of the framework is already loaded and knows that the state is State.SetDefaults()?

It can call its own SetDefaults() any time it chooses, including from its constructor, as it does. What it cannot do is to allow that to trigger a pre-mature call to my code. Calling my code must be delayed until my constructor returns. The fact that the NT constructor triggers an immediate call into my code from its constructor is a huge design bug and must get fixed. Either find a way to delay calling my code, or else find a way to call SetDefaults() after my constructor has returned.

Koganam: How else can the framework query the state other than by running SetState()/OnStateChange()? Well, that just looks like the simplest way: there might be others.

That statement confuses several things:

• Neither of those calls has anything to do with querying the state. They set the state and announce the state has changed, respectively.
• Nothing in that statement justifies a premature call to my code to announce the state change.

Brett: In this case the answer is simple and the behavior you see is expected.

Yes, it is expected because that is the way the NT8 code is designed. You have a huge design bug! It is simply WRONG, WRONG, WRONG to call into my code before its constructor has returned. How and when you determine the state is up to you. That is private to your own code, and I have no reason to care. What is not up to you, and what I have every reason to care about, is getting calls into my code before the constructor has returned. Doing that is a huge design bug! What you have effectively done, although you have failed to document it, is made OnStateChanged() with State == SetDefaults into the very first part of my constructor! Now that's unconventional, and non-intuitive to anyone reading my code!

I am honestly unable to fathom how any software professional thinks it is OK to make calls into methods of an object that has not yet been constructed. In this case, the call is not even to get something the NT constructor needs -- it is to pass information to me that I have no need of, or use for, until after my constructor has completed.

Please point me to any reputable literature explaining why calling methods in an object that has not yet been constructed is a good thing to do. Is there such a design pattern? To me it is anathema, and an NT8 design bug that must get fixed ASAP. Let the constructor complete and then call OnStateChanged().

--EV

[sledge]

I don't think your object has been instantiated at this point so no constructor necessary.

[ETFVoyageur]

I don't think your object has been instantiated at this point so no constructor necessary.

That depends on your definition of "instantiated". The constructor got called, and so presumably memory for he object has been allocated. What "instantiated" normally means, however, is that the constructor has returned. Calls to the instance's methods are not legal until that point.

--EV

[ETFVoyageur]

Brett,

Another thought just occurred to me. Based on the evidence, and on what you have said, OnStateChange() with State==State.SetDefaults is a total waste. You can just get rid of it all together and tell people to set their defaults in the constructor. You algorithmically know that will be after the OnStateChanged() call would have happened.

Why not document that indicators should set defaults in the constructor, and remove one call to OnStateChange()? From the indicator's point of view, it is not a state change -- it is the default state when the first of the indicator's code (the constructor) gets to run.

Doing this makes OnStateChanged() simpler by one case, and it addresses my biggest concern -- you would no longer be calling into an instance before its constructor has run. You would also be back to more conventional programming -- initialization (in this case setting defaults) happens in the constructor.

---EV