Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

NT8 Sometimes Runs Termination During OnBarUpdate

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    NT8 Sometimes Runs Termination During OnBarUpdate

    NT8's multithreading model as of beta 9 sometimes destroys the indicator object during the time OnBarUpdate is running, which can lead to exceptions as the resources required are destroyed while they are still being used.

    As an example, consider the code:

    namespace NinjaTrader.NinjaScript.Indicators
    {
    public class TestPrematureTermination : Indicator
    {
    private bool IsUpdating = false;

    protected override void OnStateChange()
    {
    if (State == State.SetDefaults)
    {
    Description = @"Test premature termination issue, NT 8 beta 9";
    Name = "TestPrematureTermination";
    }
    else if (State == State.Terminated)
    {
    if (IsUpdating) throw new Exception("Premature termination");
    }
    }

    protected override void OnBarUpdate()
    {
    IsUpdating = true;
    System.Threading.Thread.Sleep(1);
    IsUpdating = false;
    }
    }
    }

    This demonstration is very basic, and obviously could be done differently. But unless it is by deliberate design that an indicator can be terminated while an individual bar or tick is still in OnBarUpdate, this seems like an oversight and likely to cause many complications. A reasonable expectation on the part of an indicator designer might be that the indicator may be terminated at any time between ticks or bars, but not concurrent with the calculation of any one tick or bar. What actually happens is that randomly, OnStateChange changes to Terminated the state while OnBarUpdate is still calculating a tick or bar. This can be easily demonstrated by putting this indicator on a chart and hitting F5 a few times until it happens (example screenshot attached).

    I can easily imagine how this happened, but it would be quite reasonable to assume that despite the multithreading approach the indicator will not be destroyed while it is still calculating - if that is by design, it would require extraordinary handling on the part of indicator developers to deal with it, especially if they use any unmanaged resources.

    The consequence of it working the way that it currently does is that indicator designers cannot rely on the Terminated state change to safely dispose of resources the indicator uses, because OnBarUpdate might still be running and using them.

    While it is possible to work around this in NinjaScript code, it seems like it is most likely an oversight and the sort of thing that will cause many people many issues in the future because the resources will be disposed while they are still being accessed.

    A simple solution NT's developers could use is to wait until OnBarUpdate exits before changing the state to terminated. If that were done, the common sense assumption could once again be that the state change to Terminated is where resources can be safely cleaned up.

    This does not happen only when there is a Sleep() involved - however, by adding the Sleep() it is easier to demonstrate quickly. This issue originally surfaced here on indicator code that only does calculations.
    Attached Files
    Last edited by QuantKey_Bruce; 03-04-2016, 06:32 PM.
    Bruce DeVault
    QuantKey Trading Vendor Services
    NinjaTrader Ecosystem Vendor - QuantKey

    #2
    Thank you for your notes, Bruce DeVault.

    We will look into this matter and follow up here when we have additional details.

    Comment


      #3
      Originally posted by NinjaTrader_PatrickH View Post
      Thank you for your notes, Bruce DeVault.

      We will look into this matter and follow up here when we have additional details.
      Probably a manifestation of this: http://ninjatrader.com/support/forum...ad.php?t=83608

      Comment


        #4
        Originally posted by Bruce DeVault View Post
        NT8's multithreading model as of beta 9 sometimes destroys the indicator object during the time OnBarUpdate is running, which can lead to exceptions as the resources required are destroyed while they are still being used.

        As an example, consider the code:

        namespace NinjaTrader.NinjaScript.Indicators
        {
        public class TestPrematureTermination : Indicator
        {
        private bool IsUpdating = false;

        protected override void OnStateChange()
        {
        if (State == State.SetDefaults)
        {
        Description = @"Test premature termination issue, NT 8 beta 9";
        Name = "TestPrematureTermination";
        }
        else if (State == State.Terminated)
        {
        if (IsUpdating) throw new Exception("Premature termination");
        }
        }

        protected override void OnBarUpdate()
        {
        IsUpdating = true;
        System.Threading.Thread.Sleep(1);
        IsUpdating = false;
        }
        }
        }

        This demonstration is very basic, and obviously could be done differently. But unless it is by deliberate design that an indicator can be terminated while an individual bar or tick is still in OnBarUpdate, this seems like an oversight and likely to cause many complications. A reasonable expectation on the part of an indicator designer might be that the indicator may be terminated at any time between ticks or bars, but not concurrent with the calculation of any one tick or bar. What actually happens is that randomly, OnStateChange changes to Terminated the state while OnBarUpdate is still calculating a tick or bar. This can be easily demonstrated by putting this indicator on a chart and hitting F5 a few times until it happens (example screenshot attached).

        I can easily imagine how this happened, but it would be quite reasonable to assume that despite the multithreading approach the indicator will not be destroyed while it is still calculating - if that is by design, it would require extraordinary handling on the part of indicator developers to deal with it, especially if they use any unmanaged resources.

        The consequence of it working the way that it currently does is that indicator designers cannot rely on the Terminated state change to safely dispose of resources the indicator uses, because OnBarUpdate might still be running and using them.

        While it is possible to work around this in NinjaScript code, it seems like it is most likely an oversight and the sort of thing that will cause many people many issues in the future because the resources will be disposed while they are still being accessed.

        A simple solution NT's developers could use is to wait until OnBarUpdate exits before changing the state to terminated. If that were done, the common sense assumption could once again be that the state change to Terminated is where resources can be safely cleaned up.

        This does not happen only when there is a Sleep() involved - however, by adding the Sleep() it is easier to demonstrate quickly. This issue originally surfaced here on indicator code that only does calculations.
        I agree, and it is egregious.

        An even simpler demonstration of encapsulation violation. ref: http://ninjatrader.com/support/forum...ad.php?t=83608

        Comment


          #5
          Bruce,

          Thanks for taking the time to analyze the issue and write up a post. It triggered some internal discussion, so for that, I greatly appreciate it. In my discussion with the Development team, you raise a valid point which is exacerbated by the fact that NinjaTrader 8 is multi-threaded.

          The current thinking is that we want to continue to call terminated right away to shut down the NinjaScript object immediately. If the NinjaScript is in fact in the middle of something that does reference a resource that ends up being cleaned up by that Termination call it would cause an exception which would force it to close out even sooner. All NinjaScript is protected by a try/catch on the entire NinjaScript object by NinjaTrader, so any cases of this occurring will make it to the log and not cause an issue with NinjaTrader operation and also subsequently the NinjaScript object as it is in the middle of being terminated.

          For now, I want to keep an eye on how often this case is hit in regular NinjaScript code since any proposed changes on changing this has implications which we liked to avoid if possible.

          Thanks for bringing this issue to our attention.

          -Brett

          Comment


            #6
            Brett,

            Thank you for your reply. I did not anticipate that NT would take this stance - it seemed like so clearly a problem. The primary way that it manifests is that any unmanaged resources can no longer be safely cleaned up in the terminate state change.

            It sounds like you are saying having exceptions in the log file is acceptable. Have you considered that during an optimization you might end up with, say, 1,000 lines of exceptions?

            Clearly, we can trap this and can implement our own locking to stop the threads from over-running each other, but I was anticipating that most indicator programmers would not think to do that.

            Bruce DeVault
            Last edited by QuantKey_Bruce; 03-10-2016, 08:14 PM.
            Bruce DeVault
            QuantKey Trading Vendor Services
            NinjaTrader Ecosystem Vendor - QuantKey

            Comment

            Latest Posts

            Collapse

            Topics Statistics Last Post
            Started by usazencort, Today, 01:16 AM
            0 responses
            1 view
            0 likes
            Last Post usazencort  
            Started by kaywai, 09-01-2023, 08:44 PM
            5 responses
            603 views
            0 likes
            Last Post NinjaTrader_Jason  
            Started by xiinteractive, 04-09-2024, 08:08 AM
            6 responses
            22 views
            0 likes
            Last Post xiinteractive  
            Started by Pattontje, Yesterday, 02:10 PM
            2 responses
            20 views
            0 likes
            Last Post Pattontje  
            Started by flybuzz, 04-21-2024, 04:07 PM
            17 responses
            230 views
            0 likes
            Last Post TradingLoss  
            Working...
            X