BarsRequest and MarketData subscription

[cls71]

Hello,

I have some issues with subscription to MarketData events (Level1 or Time&Sales) on BarsRequest objects.

I can subscribe well to OnBarUpdate (Update event) from BarsRequest objects and works fine.

But I need to subscribe also to MarketData event from BarsRequest objects.
If I subscribe to MarketDataUpdate event from BarsRequest.Bars.Instrument.MarketDataUpdate looks like all works fine: first an OnBarUpdate event is received and then inmediately an OnMarketData event is received. The volumens match exactly and the time secuency is correct.

The next image reports the following sequence:
First: OnBarUpdate event, with Close price and Volume of the bar.
Next: MarketDataUpdate event, with Last Price and volume of the print.




The bug happens when I try unsubscribe to MarketDataUpdate when the indicator is Terminated. Then I get an exception, and the connection continues live receiving OnMarketData events.




Other alternative is subscribe to BarsRequest.Bars.Instrument.MarketData.Update like the Help advises. In this case the desubscription doesn't raise any exception, but the live events (OnBarUpdate vs OnMarketData) are very unsynchronized.



But the main issue is that some times the OnMarketData events are duplicated as you can see in the image.



I attach the code. It is an indicator but should works equal like add-on (the bugs are the same).
For example, I load the indicator on ES 06-17 Range-4, 3-days.

In short, there are two bugs:
1.- Subscribing to MarketDataUpdate: exception when I make the disposing (bug).
2.- Subscribing to MarketData.Update: desynchronization (can be expected) and duplicated events (bug).

Sorry for the length but my english level is not high and wanted explain correctly the context.

Thanks.

[NinjaTrader_JessicaP]

Hello cls71, and thank you for your question. I noticed this was an Indicator. May I ask why we are attempting to use Market Data subscriptions, rather than simply using the built-in hook to OnMarketData ? From an indicator, the following code will, by itself, allow you to subscribe to market data events :

// http://ninjatrader.com/support/helpGuides/nt8/en-us/onmarketdata.htm
// http://ninjatrader.com/support/helpGuides/nt8/en-us/marketdataeventargs.htm
[FONT=Courier New]protected override void OnMarketData(MarketDataEventArgs marketDataUpdate)
{
    // Print some data to the Output window
    if (marketDataUpdate.MarketDataType == MarketDataType.Last)
          Print(string.Format("Last = {0} {1} ", marketDataUpdate.Price, marketDataUpdate.Volume));
    else if (marketDataUpdate.MarketDataType == MarketDataType.Ask)
        Print(string.Format("Ask = {0} {1} ", marketDataUpdate.Price, marketDataUpdate.Volume));
    else if (marketDataUpdate.MarketDataType == MarketDataType.Bid)
        Print(string.Format("Bid = {0} {1}", marketDataUpdate.Price, marketDataUpdate.Volume));
}[/FONT]

This said if there is a reason such as code reuse for wanting to subscribe to market data the same way you would from an add-on, you will need to use Bars.Instrument.MarketData.Update and not Instrument.MarketDataUpdate. This example from the help guide may clear up how it is intended these sorts of market data subscriptions are designed to work.

[FONT=Courier New]// http://ninjatrader.com/support/helpGuides/nt8/en-us/marketdata.htm
/* Example of subscribing/unsubscribing to market data from an Add On. The concept can be carried over
 to any NinjaScript object you may be working on. */
 public class MyAddOnTab : NTTabPage
 {
      private MarketData marketData;
  
      public MyAddOnTab()
      {
           // Subscribe to market data. Snapshot data is provided right on subscription
           // Note: "instrument" is a placeholder in this example, you will need to replace 
           // with a valid Instrument object through various methods or properties available depending
           // on the NinjaScript type you are working with (e.g., Bars.Instrument or Instrument.GetInstrument()
           marketData = new MarketData(instrument);
           marketData.Update += OnMarketData;
  
           // Printing snapshot market data for the last price and time
           NinjaTrader.Code.Output.Process(marketData.Last.Price.ToString() + " " + marketData.Last.Time.ToString(),
                PrintTo.OutputTab1);
      }
  
      // This method is fired on market data events
      private void OnMarketData(object sender, MarketDataEventArgs e)
      {
           // Do something with market data events
      }
  
      // Called by TabControl when tab is being removed or window is closed
      public override void Cleanup()
      {
           // Make sure to unsubscribe to the market data subscription
           if (marketData != null)
                marketData.Update -= OnMarketData;
      }
  
      // Other required NTTabPage members left out for demonstration purposes. Be sure to add them in your own code.
 }[/FONT]

You are correct that these events are asynchronous. This publicly available MSDN C# code will show you how to use a lock statement to synchronize multiple asynchronous threads.

https://msdn.microsoft.com/en-us/library/c5kehkcz.aspx

As you are using Ninja in a way other than intended we can not consider these behaviors bugs. However, if you can describe how you would like Ninja to work I will be happy to present a case for a new feature being added.

[cls71]

Hello JessicaP,

How I already said in my previous post, when I subscribe via Instrument.MarketData.Update I receive duplicated events.

I attach an add-on that would be the expected way, instead an indicator. In order to reproduce the bug, please follow these steps:
1.- Load the add-on. A new menu item is added which name is "Bars Request Bug".
2.- Open the NinjaScript Output window.
3.- Connect to one realtime datafeed with ES 06-17 marketdata.
4.- Click on menu "Bars Request Bug".

From now you will receive events of type MarketData. You will see them in NinjaScript Output.

5.- If you observe closely you will see the marketdata events are duplicated. Select an infrequent volume and you will detect better the bug. In the attached image I have selected an volumen equal 21. As you can see, above and below each volume, events are repeated.



So it seems like this is not about locks. The attached code is simple and I think has not any error.

Regards

[NinjaTrader_JessicaP]

Thank you for your report.

This behavior was confirmed on our end. We will be investigating this further. Please keep an eye on the NinjaTrader 8 Release Notes page for updates and bugfixes.

http://ninjatrader.com/support/helpGuides/nt8/en-us/release_notes.htm

Tracking ID: NTEIGHT-11394

[NinjaTrader_JessicaP]

Hello cls71,

I have some feedback regarding this behavior. This behavior is to be expected. If you compare the output of the two attached scripts, an add-on and an indicator, you will see that they output the same data. The duplicate ticks are coming in from your data feed provider. If you would like a convenient way to filter out these duplicate ticks please examine the dictionary inside the Test11394.cs add-on.

Steps to reproduce
1) Place Test11394Addon.cs in bin\Custom\AddOns folder
2) Place Test11394Indicator.cs in bin\Custom\Indicators folder
3) In NT open a NS Editor and compile
4) Restart NT8
5) New -> NinjaScript Output
6) New -> Chart -> ES 06-17 1 minute
7) Connect to data
8) Apply Test11394Indicator.cs to chart
9) Wait a little bit then copy everything from output tab 1 into one side of a diff checker and everything from output tab 2 into the other side
10) Line up the results by time (Because we didnt start both subscriptions at the same time -> Delete some of the beginning of the addon output)
11) Do the diff

You will see that the results are exactly the same.

[pjsmith]

Apologies to resurrect an old thread, but I believe Jessica did comment on something here that seems to have resolved an issue I was having. I had reason to subscribe and unsubscribe (on termination) from Marketdata in an indicator. I was using the same example as per the add-on code, and for the most part, it worked fine. About 3 times out of 10 though, when one of my charts initially subscribed to marketdata, particularly if I refreshed the chart, the chart would lock, and that was that... I tested a lot of things, but I am guessing a threading issue of some sort.

I came across this statement made above

"This said if there is a reason such as code reuse for wanting to subscribe to market data the same way you would from an add-on, you will need to use Bars.Instrument.MarketData.Update and not Instrument.MarketDataUpdate. This example from the help guide may clear up how it is intended these sorts of market data subscriptions are designed to work."

So, instead of declaring a marketdata = new MarketData(Instrument) and marketdata.update += myOnMarketDataEvent, I am now using just Bars.Instrument.MarketData.Update += myOnMarketDataEvent. The chart freeze issue seems to have gone away, though I have yet to extensively test.

My question really, especially as one of the NT tech support supplied me a working example using the former, why might this be, and is it expected?

Can you add any comments? More just so that I have some reason behind this all. Just trying to ensure I am using this correctly now. Just for info - I had specific reasons for subscribing/unsubscribing, rather than just putting OnMarketData override in my code.

Thanks.

[NinjaTrader_Jim]

Hello pjsmith,

Jessica's comment was in regards to using Instrument.MarketDataUpdate vs. Instrument.MarketData.Update. The Help Guide advises to use Instrument.MarketData.Update. Adding a new MarketData object and subscribing to update events there may be redundant if you can already use Bars.Instrument.MarketData, but I would not necessarily expect freezes.

I have attached some simplified test scripts if you can encounter freezes with TestA vs. TestB, could you reply back and let me know how I can reproduce on my end? (Multiple tries is fine, I would just like some steps to take.) If you need to modify them so the issue is demonstrable, it would be appreciated as well.

Also, if you are doing this from an indicator, I am curious also as to why you want to subscribe to your own MarketData updates instead of using the built in OnMarketData method. I see you mentioned use in State.Terminated, but could you elaborate?

I look forward to assisting.

[pjsmith]

OK, so, I've been hammering this all morning, testing, and I can confirm it looks like this addresses the freeze issue. Thank you for the test code. I will try it when I am able and report findings.

To clarify.

I was doing this to subscribe to market data

if (!OMDEnabled && State != State. Historical && BarsProcessed > 0)
{
if (marketData == null)
{
PPrint("Enabling OnMarketData");
marketData = new MarketData(Instrument);
marketData.Update += OnMarketData;
OMDEnabled = true;
PPrint("Subscribed OnMarke tData");
}
}

But, Chart frequently freezes after printing Enabling OnMarketData. Other charts seem responsive. NT will not shut down as that chart remains stuck.

Solution. Change above to this

if (!OMDEnabled && State != State. Historical && BarsProcessed > 0)
{
PPrint("Enabling OnMarketD ata");
Bars.Instrument.MarketData.Update += OnMarketData;
OMDEnabled = true;
PPrint("Subscribed OnMarke tData");
}

My OnMarket data, declared as below, remains exactly the same bar the above code changes.

private void OnMarketData(object sender, MarketDataEventArgs marketDataUpdate)

In State.Terminated I am properly disposing/unsubscribing from the event in either case, so it is not that.

Freezes no longer seem to occur with the last example. They happened quite frequently before, especially when trying to use playback (click the start button, chart would often freeze as the subscription was requested) or refreshing the chart through the day (f5). As above, this does not seem to happen any more. I've hit this with enough playback time this am that I am fairly confident this issue is resolved when used per the latter example, but I will update post this should I find more. I hope this adds a little more colour.


Thanks.

[NinjaTrader_Jim]

Thanks for the update pjsmith,

I'll keep watching this thread. In case there is something else involved in your script, I will hold on further testing until you reply back. Modifying my tests scripts so they act like yours and including some specific steps to take will be exactly what I need to move forward to confirm and report.

I look forward to assisting.