Debugging questions

[ETFVoyageur]

How do I know when an object is constructed? I'd normally put a Print in the constructor(s), but I can't with NT because the constructor(s) are automagically generated. Can I use Initialize() instead? That would assume that Initialize() is the first thing called after construction, and that Initialize() is called only once for any given instance of an object. Is that correct? If Initialize() is not suitable, then what is?

-----

For debugging, how do I identify a specific instance of an object? In C++ I would just print "this" in hex. That does not work in C#. I do not care about an address -- just something guaranteed to be unique to that instance of the object. If I had access to the constructor, this would be trivial -- in the constructor I'd just increment a static int and save a copy. If I can use Initialize() as a proxy for a constructor (previous question) then I'm good to go.

-----

Can someone please tell me the flow for a single indicator when the user brings up a configuration dialog? What is constructed, destructed, and what methods are called in what order. I have tried some trace printing, and the results are surprising so I am hoping someone can tell must just what I should be expecting.

-----

Thanks,
EV

[NinjaTrader_Austin]

ETFVoyageur, what kinds of objects are you trying to check if they're constructed yet?

Initialize() is called many, many times throughout typical NinjaTrader processes, so it is not a suitable place to check if objects have been created yet.

Have you tried debugging with Visual Studio yet?

I don't know the specifics about the flow for bringing up an indicator, but I do know that Initialize() will be called multiple times, and OnStartUp() will be called right when the indicator is ready to be "placed" on the chart (after the Initialize() stuff).

[ETFVoyageur]

> what kinds of objects are you trying to check if they're constructed yet?

I'm checking on my indicator object. I'm not trying to check whether an instance is constructed yet -- it obviously is or my code would not be running. The constructor is the only place I can set a value that I know will be unique per object, and I want to do that to so I have confidence I know which instance is which in my tracing of what is going on.

Since I cannot place code in the constructor, I am hoping that I can set the unique identifier in Initialize(). Based on my tracing so far, that is looking llikely.

> Initialize() is called many, many times throughout typical NinjaTrader processes, so it is not a suitable place to check if objects have been created yet.

At least in the simple scenario I am looking at, it looks as if each Initialize() call is has a subsequent destructor (I tagged the Initialize() calls uniquely). Are you really sure that Initialize() is called multiple times on the same instance of an object?

My tracing is showing that a surprising number of objects (2) seem to be constructed and destroyed just for simple action such as bringing up the configuration dialog and Closing it (not OK or Apply). I really want to be sure to tag each instance with a unique identifier before I try to draw any conclusions about what is going on.

> Have you tried debugging with Visual Studio yet?

No ... my version is .NET, but too old for NT. I'm just using Print statements. Besides, at least for some tracing, Print statements are more useful than VS. I may get to updating my VS one day to work with NT, but indicator development is really pretty simple stuff, so no real need so far.

> I don't know the specifics about the flow for bringing up an indicator, but I do know that Initialize() will be called multiple times, and OnStartUp() will be called right when the indicator is ready to be "placed" on the chart (after the Initialize() stuff).

It is not just bringing up an indicator. For example, it looks as if just bringing up the configuration dialog and doing a Close constructs and destroys two instances, and neither one gets as far as OnStartUp(). All that gets called is Initialize(), then Dispose() and eventually the destructor.

My tracing agrees with you that there are quite a few calls to Initialize(), but that is not the question. The question is about calls per object instance. It looks as if a destructor call is associated with Initialize() call, which would mean each Initialize() call is to a different instance.

-----

As an example, one of the things I am looking into is that someone suggested to me that Dispose() is called when bringing up the configuration dialog, and I could make use of that. My tentative tracing says that Dispose() is called, but only on a newly-constructed object, not on the instance doing the current plotting. That's an example of why I want to be certain which instance I am talking about.

--EV

[ETFVoyageur]

Dispose() and OnTermination()

Sorry this is long. Since I am to some extent questioning common wisdom, clarity and completeness seemed more important than brevity.

I am trying to understand how Dispose(), OnTermination(), and destructors work. I am also trying to understand what, if anything, gets called when the Indicator dialog is brought up (the answer appears to be that nothing is called).

There is a good tutorial on Dispose(), the destructor, and their relationship here. What they suggest is not what I see in any of the NT code I have looked at. I wonder why.

The Help page for OnTermination() says

For advanced programmers: You can override the OnTermination() method to clean up all resources owned by the custom indicator or strategy.

Note: Please do NOT overload the Dispose() method. Dispose() method could be triggered much later than expected resulting in resources not being released early enough.

I question the accuracy of that comment. In my limited testing, Dispose() got called reliably, but OnTermination() did not. In the one case where OnTermination() did get called, Dispose() got called first. It would seem far better to put your cleanup in Dispose(), and call Dispose() from both OnTermination() and the destructor. Then you get cleaned up at the earliest opportunity, regardless of which the system calls when. Naturally, Dispose() needs to be prepared to handle multiple calls to it, but it should already be doing that anyway, because it is a public method and multiple calls to it are legal.

I have been doing some tracing and it is pretty common for the system to just call Initialize() and Dispose() without ever calling OnTermination(). That's fine if you do not allocate anything in Initialize(), but a lot of code seems to do all of its initialization in Initialize(), in which case it will leak resources if it waits for OnTermination() before cleaning up.

So what is the architectural intent of OnTermination()? When is it supposed to get called? What is its purpose? In my trace, it only got called for the instance that actually did the plotting, and even then Dispose() got called first. Most of the objects that got instantiated got taken down without ever calling it. Why even bother with OnTermination()?

Here is a record of my trace output. I labeled it as code so that lines would not get folded. Some extra spacing is getting added, I don't know why, but the result is readable anyway. The numbers (#x) are a unique instance identifier set up in Initialize() so that one can correlate other method calls with which instance the call is for. Some of the things the trace shows are:

• It appears that Initialize() gets called exactly once per instance, and is a reliable stand-in for the constructor (which the indicator developer has no access to). If that observation is wrong, please explain why.

• The destructor for the first object (object #0) gets called repeatedly. I do not know who/why is doing this, but on the surface it would seem like a problem.

• Every time I bring up the Indicators dialog a couple of indicator objects are getting created and destroyed. OnTermination() is never called for them.

• Plot() is getting called a lot more than necessary. This seems at best inefficient.
  • Simply giving focus to the chart window triggers Plot()), even if nothing has changed.
  • Clicking once in the chart window triggers 3 calls to Plot(), even if nothing has changed. Repeated clicks each trigger three calls to Plot().
  
  

[LIST][*]Restart NT, indicator not on the chart, to ensure a      clean start[*]Bring up the Indicators dialog (either way,      double click or context menu)[LIST][*]Initialize(       #0 )[*]Dispose(       #0)[*]Destructor(       #0)[*]Dispose(       #0)              // Called by the       destructor[/LIST]
 [*]Double-click the indicator to add it to the list of      active indicators[LIST][*]Initialize(       #1 )[/LIST]
 [*]Click “Apply”                        //      Less complicated than “OK”[LIST][*]Initialize(       #2 )[*]Initialize(       #3 )[*]Dispose(       #1 )[*]OnStartUp(       #2 )         // This is the one that       will actually do the plotting[*]OnBarUpdate(       #2 )[*]Plot(       #2 )                   //  This indicator overrides Plot()[/LIST]
 [*]Click “Close”[LIST][*]Dispose(       #3 )[*]Plot       ( #2 )[/LIST]
 [*]Bring up the Indicators dialog again[LIST][*]Initialize(       #4 )[*]Dispose(       #4 )[*]Initialize(       #5 )[*]Destructor(       #0 )[*]Dispose(       #0)              // Called by the       destructor[*]NOTE:       This is the second time that the destructor for #0 has been called – what       is going on?[*]SUMMARY       so far:[LIST][*]#0        – destructed multiple times[*]#1        – disposed, no destructor yet[*]#2        – still plotting[*]#3        – disposed, no destructor yet[*]#4        – disposed, no destructor yet[*]#5        – initialized only[/LIST]
 [/LIST]
 [*]Click “Close”[LIST][*]Dispose(       #5 )[*]Plot(       #2 )[/LIST]
 [*]One single click on the chart (restoring focus to the      chart)[LIST][*]Plot(       #2 )[*]Plot(       #2 )[*]Plot(       #2 )                   // I’m not       sure why plot at all, and certainly do not need to do it 3 times[*]My further       observation is that:[LIST][*]Restoring        focus to the chart causes one call to Plot()[*]Clicking        on the chart causes 3 calls to Plot()[*]The        above is true, even when there has been no actual need – nothing about        the chart changed, fully visible the whole time[/LIST]
 [/LIST]
 [*]Bring up the Indicators dialog again[LIST][*]Initialize(       #6 )[*]Dispose(       #6 )[*]Initialize(       #7 )[*]Destructor(       #0 )[*]Dispose(       #0)              // Called by the       destructor[*]Plot(       #2 )[*]NOTE:       This is the third time that the destructor for #0 has been called – what is       going on?[*]SUMMARY       so far:[LIST][*]#0        – destructed multiple times – it gets destroyed EVERY time I bring up        the Indicators dialog – why?[*]#1        – disposed, no destructor yet[*]#2        – still plotting[*]#3        – disposed, no destructor yet[*]#4        – disposed, no destructor yet[*]#5        – disposed, no destructor yet[*]#6        – disposed, no destructor yet[*]#7        – initialized only[/LIST]
 [/LIST]
 [*]Click “Close”[LIST][*]Dispose(       #7 )[*]Plot(       #2 )[/LIST]
 [*]Bring up the Indicators dialog again[LIST][*]Initialize(       #8 )[*]Dispose(       #8 )[*]Initialize(       #9 )[*]Destructor(       #0 )         // Note that #0 gets       destroyed EVERY time I bring up the Indicators dialog – why?[*]Dispose(       #0)              // Called by the       destructor[/LIST]
 [*]Click “Remove”[LIST][*]Disposing(       #9 )          // Note that we       dispose a temporary object, not the active one[/LIST]
 [*]Click “Apply”[LIST][*]Dispose(       #2 )             // This is the one       that has been doing the plotting[*]OnTermination(       #2 )[*]Dispose(       #2 )             // Called from       OnTermination()[*]Dispose(       #2 )             // System called       this … interesting, since OnTermination() already called[/LIST]
 [/LIST]

[NinjaTrader_JoshP]

ETFVoyageur,

On the contrary, Dispose() is not sufficient because we have found in scenarios where it would actually not be called in time on termination of the script. This is why we have introduced the OnTermination() method which would indeed come after Dispose(), but is guaranteed to trigger when closing out the script. If you use Dispose() and you run into some of the specific scenarios in which it wasn't working properly, then you will have resources that will not be released.

[ETFVoyageur]

ETFVoyageur,

On the contrary, Dispose() is not sufficient because we have found in scenarios where it would actually not be called in time on termination of the script. This is why we have introduced the OnTermination() method which would indeed come after Dispose(), but is guaranteed to trigger when closing out the script. If you use Dispose() and you run into some of the specific scenarios in which it wasn't working properly, then you will have resources that will not be released.

As I noted, my traces have shown it is common to call Dispose(), but not OnTermination(). That is no problem, unless you allocate resources in Initialize(), and I'll bet some code does.

The web page I cited says that, because you never know when the GC will get around to deleting an object, you should count on Dispose() and call Dispose() from the destructor. They recommend a destructor, just as a fail safe, in case Dispose() never gets called.

After seeing the OnTermination() is not reliably called, what my code now does is follow the model that web page suggests -- the real work is done in Dispose(), and both the destructor and OnTermination() call Dispose().

Any answers to my other questions?

--EV

[NinjaTrader_JoshP]

OnTermination() is where you want to dispose your resources. That is where you are guaranteed they will be released. If you just try to use solely Dispose() you will potentially run into issues. That is the purpose and intent of OnTermination(). Do not use Dispose().

Initialize() is called whenever you are in the indicator/strategy dialogue box. It can be called several times. If you have resources you want to set up, please do it from OnStartUp() and not Initialize().

[ETFVoyageur]

OnTermination() is where you want to dispose your resources. That is where you are guaranteed they will be released. If you just try to use solely Dispose() you will potentially run into issues. That is the purpose and intent of OnTermination(). Do not use Dispose().

Initialize() is called whenever you are in the indicator/strategy dialogue box. It can be called several times. If you have resources you want to set up, please do it from OnStartUp() and not Initialize().

Josh,

I set up some Print() tracing that included a unique identifier per instance. What it shows does not agree with what you are saying.

> OnTermination() is where you want to dispose your resources. That is where you are guaranteed they will be released.

I know that is what Help says. According to my tracing, though, it is pretty common for Dispose() to be called but OnTermination() to never get called. The only thing I have heard against Dispose() is the assertion that it is not called reliably. That is solved by writing the code the way I indicated -- with OnTermination() calling Dispose(). Kind of similar to the relationship between the destructor and Dispose() discussed in that web page I referred to.

> If you just try to use solely Dispose() you will potentially run into issues.

We are not discussing using just Dispose() -- I never suggested that. Nor would I use just OnTermination(). Both are unsafe practices, because they depend on code I have no control over being bug-free. My tracing shows it is best to just have OnTermination() call Dispose(), where the real work is done. That way I am covered, even in the case where OnTermination() does not get called. My way releases resources when the first of Dispose(), OnTemination(), or the destructor gets called.

> Initialize() is called whenever you are in the indicator/strategy dialogue box. It can be called several times.

Are you really sure about that? My tracing shows that Initialize() is called only once on any given instance. (Bringing up an Indicators dialog creates more than one instance.) If you just put a Print( ) in Initialize() without tagging the instance, it looks like multiple calls, but as far as I can see the calls are each to a different instance. Try tagging the instances and see what your Print() calls show you.

> If you have resources you want to set up, please do it from OnStartUp() and not Initialize()

I whole-heartedly agree with that, for at least three reasons I can think of immediately. However, I have seen code that does not do so. Furthermore, OnStartUp() is not even mentioned in the tutorials, so there is no good reason to think that new programmers will get the word. (I turned in a suggestion that the tutorials need to be upgraded to cover this.) If the system is depending on this, it is making a mistake.

--EV

[NinjaTrader_JoshP]

ETFVoyageur,

Yes, I am definitely sure Initialize() can be called in the indicator dialogue several times depending on what actions you take exactly. Whether it comes from a different instance could be the case, but the point is simply resources should not be created here and logic should not be placed in Initialize().

OnTermination() is always called when the script was added and removed. Triggering an Initialize() print and then not adding the script would not necessarily trigger OnTermination(). This would be expected.

Thank you for the suggestion on the reference samples.

[ETFVoyageur]

I guess we agree to disagree to some extent. Which instance a method is called on is important. Furthermore, it seems clear to me that the safest thing is to:

• Do all cleanup in Dispose(), handled as the web page suggests
• Call Dispose() from both OnTermination() and the class destructor

I have heard nothing to make me think that there is anything wrong with this, and it will get my object cleaned up no matter who does what to whom at what time. I do not have to worry about the order of calls from other software, or even whether the calls get made.

I do not know what you mean by "when the script was added and removed.". That is not a concept that indicator code has any reason or way to be aware of.

Note that this is completely independent of the roles of Initialize() and OnStartUp(). From what have seen, I strongly do agree with you on their roles.

--EV

[koganam]

OnTermination() is where you want to dispose your resources. That is where you are guaranteed they will be released. If you just try to use solely Dispose() you will potentially run into issues. That is the purpose and intent of OnTermination(). Do not use Dispose().

Initialize() is called whenever you are in the indicator/strategy dialogue box. It can be called several times. If you have resources you want to set up, please do it from OnStartUp() and not Initialize().

Are you saying that we have to rewrite indicators to now call the DataSeries constructor, for example, from OnStartUp()?

[NinjaTrader_JoshP]

No, DataSeries can be handled fine in Initialize().

I am referring to custom resources like for instance opening up a StreamWriter object or a timer object, etc.

[koganam]

I guess we agree to disagree to some extent. Which instance a method is called on is important. Furthermore, it seems clear to me that the safest thing is to:

• Do all cleanup in Dispose(), handled as the web page suggests
• Call Dispose() from both OnTermination() and the class destructor

I have heard nothing to make me think that there is anything wrong with this, and it will get my object cleaned up no matter who does what to whom at what time. I do not have to worry about the order of calls from other software, or even whether the calls get made.

Your method certainly seems to cover all bases. Would it be too onerous to request that you release some sample demonstration code which demonstrates the bare-bones concept, without getting into any heavy code processing?

This has certainly been an education on why sometimes, things should not just be left to the framework.

Thanks.

[ETFVoyageur]

Are you saying that we have to rewrite indicators to now call the DataSeries constructor, for example, from OnStartUp()?

I do not know about "have to", but it might be better that way.

My tracing looks like the system creates temporary objects from time to time. Initialize() will get called for those objects, but OnStartUp() will not. The best way to avoid wasting resource is to do in Initialize() only the minimum needed for the configuration to work correctly. Wait until OnStartUp() for all that can stand waiting that long -- surprisingly often that will not happen.

--EV

[ETFVoyageur]

Your method certainly seems to cover all bases. Would it be too onerous to request that you release some sample demonstration code which demonstrates the bare-bones concept, without getting into any heavy code processing?

This has certainly been an education on why sometimes, things should not just be left to the framework.

Thanks.

Ask, and ye shall receive. The code has quite a few comments, so looks a lot bulkier than it really is. Ignore the debug statements That's just something else I have set up for myself (recall my comments about my tracing?).

    #region Dispose(), OnTermination(), destructor
        /// There is a good tutorial discussion of Dispose() and Finalize() at:
        ///         http://www.devx.com/dotnet/Article/33167
        /// 
        /// Managed resources' lifetimes are controlled by .NET, but unmanaged ones are not.
        /// 
        /// Finalize() is not created explictly -- it is generated implicitly whenver there is a destructor.
        /// The object should have a destructor (and therefore an implicit Finalize() iff there are unmanaged
        /// resources that need cleaning up.
        /// 
        ///     The Garbage Collector will call Finalize() before destroying the object.  .NET does not guarantee
        ///     that Finalize() will ever get called.  Because when/whether Finalize() gets called can't be
        ///     controlled, it is only a fallback mechanism for releasing unmanagedresources.
        /// 
        ///     The software should normally release resources that are no longer needed by calling Dispose(),
        ///     rather than relying on Finalize() ever geting called.
        /// 
        ///     Failure to do provide a destructor (and hence Finalize()) when needed can cause memory leaks.
        /// 
        /// Dispose() is called by the appplication, not the Garbage Collector.  The application should always
        /// call Dispose() for any object that implements it, once the object is no longer needed.  That allows
        /// freeing unmanaged resources as soon as they are no longer needed.
        /// 
        ///     Because there is no point in calling Finalize() once Dispose() has been run, Dispose() should
        ///     always call GC.SuppressFinalize(this).
        /// 
        /// In the NT environment, Dispose() may be called without OnStartUp() ever having been called.
        /// 
        /// OnTermination() is called iff OnStartUp() was called.
        ///     In the cases I have checked, Dispose() is called before OnTermination().
        
        ////////////////////////////////////////////////////////////////////////////////////////////////////
        /// COMMENT OUT, OR OTHERWISE REMOVE, ALL OF THESE METHODS (destructor, Dispose(), OnTermination()
        /// IFF NO CLEANUP IS NEEDED
        ////////////////////////////////////////////////////////////////////////////////////////////////////

        ~RwbDemoModel()
        {
            if (debugLevel >= Debug.Entry) Print(ID("Destructor"));
            Dispose(false);
        }
        
        public override void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);    // Optimization, because Finalize() is no longer needed
        }
        
        /// <summary>
        /// Here is the one-and-only place where the real work gets done
        /// 
        /// The NinjaTrader documentation says to prefer OnTermination(), because it is unclear when Dispose()
        /// will be called.  In spite of that, there are cases where Dispose() is called, but OnTermination()
        /// is not called.  Furthermore, in the cases I have checked, Dispose() is called before OnTermination().
        ///     * For example, when the user brings up the configuration dialog, Dispose() is called,
        ///       but OnTermination() is not.
        /// </summary>
        protected void Dispose(bool disposing)
        {
            if (debugLevel >= Debug.Entry) Print(ID("Dispose", String.Format("disposing={0}", disposing)));
            
            if (!isDisposed){        // The check is in case we get called twice.  Should never happen.

                if (disposing) {
                    // Code to dispose the managed resources of the class
                }
                
                // Code to dispose the un-managed resources of the class
            }
        
            // Cleanup
            isDisposed = true;
            base.Dispose();        // Example code called base.Dispose(disposing); not supported in the NT environment
        }

        
        /// <summary>
        /// //////////////////////////////////////////////////////////////////////
        /// This gets called any time the user changes configuration.
        /// There will be subsequent calls to Initialze() and and OnStartUp()
        /// </summary>
        protected override void OnTermination()
        {
            if (debugLevel >= Debug.Entry) Print(ID("OnTermination"));
            Dispose();
        }
    #endregion