NT8 sometimes runs OnStateChange termination during OnBarUpdate

[koganam]

Well, all that flag accomplishes is to make sure that we don't, for instance, dispose of an object before it's been set up. If you think about it, if you do something like if (a != null) a.dispose(); you should not need a flag for whether or not a has been allocated, provided you allocate it in the Configure or later state. Could you clarify what the issue would be with that deallocating only if it is allocated? If NT bungles the object state order and calls Terminated before calling Configure (or where ever the allocation happens) it would just not be allocated yet and so would be unavailable to deallocate...

Reading over the NT documentation example of adding and later removing a toolbar, they are simply trying to show how to handle it if you had something that wasn't so simple e.g. you couldn't quickly check if (a != null) and needed to know if the toolbar was added before you try to remove it.

To put it another way, given these quotes from the literature:

When NinjaTrader instantiates a NinjaScript object
There are two categories of instances instantiated by NinjaTrader:

• "UI" instances representing its default properties on various user interfaces
• The "configured" instance executing your custom instructions

State.Terminated logic should be specific in when it resets a value or destroys a resource. Since the running instance can be cloned back to a UI instance, checking that a mutable property exists before accessing sometimes is not enough. You may need to consider adding a flag to help decide when a resource needs to be reset or destroyed.

, in your skeleton code, how sure can you be that State.Terminated is not being triggered from outside your indicator code, and so the cause of the issue?

[QuantKey_Bruce]

To put it another way, given these quotes from the literature ... in your skeleton code, how sure can you be that State.Terminated is not being triggered from outside your indicator code, and so the cause of the issue?

Well, by using a unique instance identifier to keep track of each instance that is created so I can tell them apart, and carefully logging which instance is receiving the invocation of OnStateChange and OnBarUpdate. It is in this way that I know that the instance having OnStateChange called during OnBarUpdate's execution on another thread is on the same instance, and NT has acknowledged as I understand it that this is the way it works (at least for now) and has created a documented developer request to change that so that it does not change State to State.Terminated until OnBarUpdate exits.

It seems to me the flag would be required for instances that do things like add chart controls (as in the NT provided example), but in this case I don't follow how that would help. If a running instance were cloned to make a UI instance, any field variables like flags are not copied unless you were to specifically do that yourself, which is the key reason they suggest adding a flag: it would not be copied. In my example, I didn't specifically make a flag, but the object references I was checking are to the same effect - they're not cloned because they are initialized anew if the object is cloned.

Isn't it the case that the main consequence of this cloning is that the States could be hit out of order, or that it could get confusing if you have modified something like a chart control that is shared between all instances? But any variables you set up yourself should be kept straight, should they not? All testing seems to indicate that is the case...

[ren37]

State.Terminated is being triggered only inside OnStateChange()

Bruce,

OnStateChange() is not a static method, so you don't need to check indicator instance.
Use lock() or mutex. They work perfect.

[QuantKey_Bruce]

OnStateChange() is not a static method, so you don't need to check indicator instance. Use lock() or mutex. They work perfect.

I was describing using an instance counter only in order to show in my logs which instance is being terminated, and that it is the same one running OnBarUpdate(). I do not normally use instance counters - I only did this to make a better log to demonstrate that it is not a UI instance being terminated but the running chart instance for the issue originally reported in this thread. It turns out that is the case, as NT has acknowledged.

The issue I have with using a lock around the entire OnStateChange() and the same lock around OnBarUpdate() so that they are mutually exclusive is that a small percentage of the time, this results in a platform hang. It can be seen if you put some non-trivial computations in OnBarUpdate that take at least a few milliseconds to complete, put the indicator on a chart (with locks preventing OnStateChange and OnBarUpdate from happening concurrently in the same instance) and then hit F5 a whole bunch of times on the chart. When the chart hangs, the scroll bar will scroll left and right but the bars no longer update on the chart, and NinjaTrader must be killed with Task Manager.

When I have time I'll construct a test case for this, but that is the reason I am not presently using locks for each method to preclude the others is that a small percentage of the time it leads to the platform hanging the chart, while without the locks and living with the OnStateChange happening concurrently, there is no platform hang.

The fact that there is a deadlock-type situation happening there implies that NT is doing something similar on their side e.g. possibly attempting to wait for one thread to finish before proceeding, such that when the user adds this within the indicator instance, there is a race condition - that is why I asked NT whether they recommended using locks to solve this problem. If they are doing something similar, we don't want to lock on our end or there will be a race and possible deadlock - we need guidance from the platform as to whether or not this is the recommended approach, and one they fully support. Or, alternatively, they need to fix the platform so that this doesn't happen, and therefore, we do not need to lock the instances.

Since we do not have 100% visibility of what they are doing on their end, that is why it makes the most sense for them to handle it, and then we would no longer have to worry about this or be concerned we may inadvertently introduce a deadlock while trying to work around this unexpected concurrent execution of the Terminated "clean up and shut down the indicator's resources" logic during OnBarUpdate's simultaneous execution which relies on those same resources.

[ren37]

How it could be a deadlock? Are you talking about this?

lock(obj)
{
...
lock(obj)
{
...
}
}

[QuantKey_Bruce]

A deadlock is (typically) when a complex application with more than one lock e.g. lock a and lock b has two different routines executing concurrently. The first grabs lock a, then tries to grab lock b. The second grabs lock b, then tries to grab lock a. If each gets its first, and is unable to get its second, there is a deadlock from which the program cannot escape.

Depending on the kind of lock, sometimes locking the same object in a nested fashion as you show can also result in a deadlock. Some kinds of locks allow this (using a reference counter to see if it's the same thread locking it again), and others do not check for this, resulting in a deadlock.

What I don't know and was asking NT is if they are doing something like this outside of our code, such that it could be resulting in a deadlock, and if locks are the recommended way to resolve this issue - the original issue reported in this thread (which they still have not answered, although they created a request to stop it from happening and said it was up to the developers to decide if they will do that).

I do understand that if we use locks on the same object on both OnStateChange and OnBarUpdate that it addresses the issue of two threads concurrently running both of these methods at the same time from two different threads. What I don't know is why it occasionally hard freezes the chart under stress testing conditions, and that is why I asked if NT recommended we use locking or some other method.

[mballagan]

I attempted to use some multithreading in one of my indicators with code such as the following where I parallelise a for loop using tasks but the indicator just seems to hang. I also tried a parallel for loop with the same results. Please advise thanks. I am aware of the fact that if accessing any UI elements from within the loop an asynch call needs to be made. Could be a deadlock issue of some sort?

var tasks = new Task[UPPERTIMEFRAME-LOWERTIMEFRAME+1];

for (int index = LOWERTIMEFRAME; index <= UPPERTIMEFRAME; index++)
{
var arraynum = index;

tasks[index - LOWERTIMEFRAME] = Task.Factory.StartNew(() =>
{
int barindex = arraynum - LOWERTIMEFRAME + 1;

if (BarsInProgress == barindex && CurrentBars[barindex] > 30)
{
try
{
//DO SOME PROCESSING HERE (which includes database locks)

}
catch(Exception ex)
{
Log(ex.Message, NinjaTrader.Cbi.LogLevel.Information);
}
}

},TaskCreationOptions.None);

}

Task.WaitAll(tasks);

[QuantKey_Bruce]

Use an attached debugger to see what your threads are waiting for. It clearly sounds like you have a logical error resulting in a deadlock. The debugger can show you what the threads are doing at the time that it appears to be hung.