I posted this last week about my setup/teardown problems and asking for advice. I've written a strategy which illustrates the problem. The first is a bug, and the second is a highly undesirable side-effect of how you're implementing event-handlers.
1. Bug - Rendering ghosting
Steps to reproduce:
-- open a chart connected to a realtime feed
- add strategy to chart
- observe that metadata is printed on the chart - bid/ask datetime random guid etc
- right click chart and choose "reload ninjascript"
Expected behaviour
- Chart should clear and reload afresh
Actual behavior
- The previous instance of the strategy is not cleared from the chart and we can see output from both.
2. Strong event references
This relates to the way I am utilising your API by instantiating my own classes and hooking into certain events in Ninjatrader - all in all my strategy has about 50 classes so far.
The example strategy attached has two classes. An OrderManager and a BarsProxy - both hook into Ninjatrader events to do their work. Just start and stop the strategy a few times and you will see what the problem is...
The order manager hooks into the Account.Execution_Update event - and is going to ensure my positions are covered by stops and targets; which will be reduced or increased according to fill amount, and also prevent overfills by identifying duplicate order submissions.
The BarProxy is going to grab bars from another time series and sent them off to a processing pipeline - I've got Three pipelines for three timeframes - each with about 20 classes for parsing, scoring, aggregating and storing. I'm scanning for significant areas of price action in Forex which where subject to significant institutional-influence and determining the likelyhood that the institutional order was completely-filled. Because if not, then price is sure to turn there again when price action returns and the rest of their order is filled - its all working pretty great so far.
Anyway... Reload the attached strategy a few times - or enable/disable. Look at the output window and you will see that the there are lots of BarsProxy instances and Order Manager instances still alive and processing.
These instances survive beyond the lifetime of the Strategy because they are linking into Internal Ninjatrader events, by the += method. Objects are only garbage collected by the Common Language Runtime when nothing is referencing them - and although the original strategy instance that made the BarProxy and OrderManager is gone and has, itself, been garbage-collected the things it created are living on and still processing because the += method of binding to event handlers constitutes a strong incoming reference to my classes so they remain on the heap and continue to process events - using up memory and CPU cycles.
This is not a bug - because events should be cleaned up and handlers unbound; but it is very highly undesirable. If an error/exception occurs and the call to unbind the handler via the -= method is never called then the instances will live on and keep processing. Or if a developer just forgets (as I did a couple of times).
Its a really common problem and their are loads of solutions. But given the nature of Ninjatrader'appliation which is built for third parties to code against its pretty high risk exposing strong event references as you can see from the zombies that occur in the attached strategy - and microsoft has invested a lot of work in providing support in the .NET framework for this problem with the WeakEventManager
This allows event handlers to be bound using WeakReferences - a weak reference is an one that does not protect the instance from the Garbage Collector. Meaning when my strategy instance is gone, and is no longer referencing the OrderManager and BarsProxy that it created, all strong-references to those objects are deemed to have gone and the ordermanager and barsproxy will be garbage collected from by the Runtime and stop using memory and processing cycles.
I realise this is pretty advanced stuff but I think that it is an unacceptable risk for you to be allowing event binding by strong references in this way - given the nature of the Ninjatrader application and its third-party ecosystem, there is so much that can go wrong and cause a gradual degradation in performance. This has really tripped me up - I spent most of last week trying to track down an issues caused by an exception being thrown before an event was un-binded.
I think one of the great strengths of Ninjatrader is that we are able to hook into these events. It enables me to write an enterprise-grade system of the back of Ninja and not just a single-file strategy. However I would suggest looking at the WeakEventManager to expose your underlying events to the third-party ecosystem in order to eliminate the risks exemplified in the attached strategy caused by strong event references.
1. Bug - Rendering ghosting
Steps to reproduce:
-- open a chart connected to a realtime feed
- add strategy to chart
- observe that metadata is printed on the chart - bid/ask datetime random guid etc
- right click chart and choose "reload ninjascript"
Expected behaviour
- Chart should clear and reload afresh
Actual behavior
- The previous instance of the strategy is not cleared from the chart and we can see output from both.
2. Strong event references
This relates to the way I am utilising your API by instantiating my own classes and hooking into certain events in Ninjatrader - all in all my strategy has about 50 classes so far.
The example strategy attached has two classes. An OrderManager and a BarsProxy - both hook into Ninjatrader events to do their work. Just start and stop the strategy a few times and you will see what the problem is...
The order manager hooks into the Account.Execution_Update event - and is going to ensure my positions are covered by stops and targets; which will be reduced or increased according to fill amount, and also prevent overfills by identifying duplicate order submissions.
The BarProxy is going to grab bars from another time series and sent them off to a processing pipeline - I've got Three pipelines for three timeframes - each with about 20 classes for parsing, scoring, aggregating and storing. I'm scanning for significant areas of price action in Forex which where subject to significant institutional-influence and determining the likelyhood that the institutional order was completely-filled. Because if not, then price is sure to turn there again when price action returns and the rest of their order is filled - its all working pretty great so far.
Anyway... Reload the attached strategy a few times - or enable/disable. Look at the output window and you will see that the there are lots of BarsProxy instances and Order Manager instances still alive and processing.
These instances survive beyond the lifetime of the Strategy because they are linking into Internal Ninjatrader events, by the += method. Objects are only garbage collected by the Common Language Runtime when nothing is referencing them - and although the original strategy instance that made the BarProxy and OrderManager is gone and has, itself, been garbage-collected the things it created are living on and still processing because the += method of binding to event handlers constitutes a strong incoming reference to my classes so they remain on the heap and continue to process events - using up memory and CPU cycles.
This is not a bug - because events should be cleaned up and handlers unbound; but it is very highly undesirable. If an error/exception occurs and the call to unbind the handler via the -= method is never called then the instances will live on and keep processing. Or if a developer just forgets (as I did a couple of times).
Its a really common problem and their are loads of solutions. But given the nature of Ninjatrader'appliation which is built for third parties to code against its pretty high risk exposing strong event references as you can see from the zombies that occur in the attached strategy - and microsoft has invested a lot of work in providing support in the .NET framework for this problem with the WeakEventManager
This allows event handlers to be bound using WeakReferences - a weak reference is an one that does not protect the instance from the Garbage Collector. Meaning when my strategy instance is gone, and is no longer referencing the OrderManager and BarsProxy that it created, all strong-references to those objects are deemed to have gone and the ordermanager and barsproxy will be garbage collected from by the Runtime and stop using memory and processing cycles.
I realise this is pretty advanced stuff but I think that it is an unacceptable risk for you to be allowing event binding by strong references in this way - given the nature of the Ninjatrader application and its third-party ecosystem, there is so much that can go wrong and cause a gradual degradation in performance. This has really tripped me up - I spent most of last week trying to track down an issues caused by an exception being thrown before an event was un-binded.
I think one of the great strengths of Ninjatrader is that we are able to hook into these events. It enables me to write an enterprise-grade system of the back of Ninja and not just a single-file strategy. However I would suggest looking at the WeakEventManager to expose your underlying events to the third-party ecosystem in order to eliminate the risks exemplified in the attached strategy caused by strong event references.
Comment