OnMarketData & OnMarketDepth should be the same method

[Fernand0]

Hi! I hope you are having a good day.

Ahm, i really don't know where to put this subject...
I'm trying to establish a data structure from raw data.... and this raw data sometimes doesn't come in the right order

private void OnMarketData(object sender, MarketDataEventArgs obj)
        {

            lock(obj.Instrument.SyncMarketData)
            lock(obj.Instrument.SyncMarketDepth)
            {

                if (obj.MarketDataType != MarketDataType.Last)
                    return;


                double Position = 0;
                MarketDataType MarketType = MarketDataType.Ask;

                if (obj.Price >= obj.Ask)
                {
                    MarketType = MarketDataType.Ask;
                    Position = obj.Price - obj.Ask;
                }
                else if (obj.Price <= obj.Bid)
                {
                    MarketType = MarketDataType.Bid;
                    Position = obj.Bid - obj.Price;
                }


                try { Position = Position / TickSize; }
                catch(Exception e) { Position = 0; }


                Row row = new Row(obj) {
                    MarketType    = MarketType,
                    Position    = (int)Position
                };

                Registry.Add(row);


                if (Printing)
                    Output(row);

            }

        }

private void OnMarketDepth(object sender, MarketDepthEventArgs obj)
        {

            lock(obj.Instrument.SyncMarketData)
            lock(obj.Instrument.SyncMarketDepth)
            {

                Row row = new Row(obj);
                Registry.Add(row);

                if (Printing)
                    Output(row);

            }

        }

It's basically the same code....
I flagged the problem in the image, you can see that there are other cases, sometimes they share the DateTime, sometimes they don't, but the most important is that they are not capable to receive the events in the right order.
Therefore you will have troubles to reconstruct this data.

If you know how to solve this, or some kind of temporary solution.... I'm all ears(or.. eyes..)


My regards.
Fernando.-

[Fernand0]

I get this in Simulated Data Feed from the same AddOn, what's going on..?

[NinjaTrader_ChelseaB]

Hello Fernand0,

There is some undocumented code you are using.

Further, the code provided does not seem complete for what is printing.

What is Output()?
Is this a method you have created for printing?
Are you willing to share this method (or an export of a complete example script that demonstrates the behavior)?

Are you finding that if you use all documented code and use the MarketDataEventArgs without locking these the same behavior can be demonstrated?

[Fernand0]

Hello Chelsea.

There is some undocumented code you are using.

Where?

private void Output(Row row)
        {
            Print(string.Format("{0:HH/mm/ss.fff}        {1}[{2:00}]:        {3:0.00}            ->        {4:000}            ({5})",
                    row.Time, row.MarketType, row.Position, row.Price, row.Volume, row.Procedure).Replace("/", ":"));
        }

As you can see, is a simple Print.

And the image that i'm uploading is without any locks... i tried with just the Depth lock... i tried all the combinations...


Why the Customer Service always thinks that is a user problem?

NinjaTrader_ChelseaB I'm sending the code(is not long or complex) by email with the subject "ATTN: Chelsea B". And if you want you can test it yourself just changing the locks and checking the Row class.

[NinjaTrader_JoshG]

Are you able to reproduce the same thing after removing this part from your code?

lock(obj.Instrument.SyncMarketData)
lock(obj.Instrument.SyncMarketDepth)

What about when you use the MarketDataEventArgs? If you have already done this can I please see what you are doing now?

[Fernand0]

Are you able to reproduce the same thing after removing this part from your code?

lock(obj.Instrument.SyncMarketData)
lock(obj.Instrument.SyncMarketDepth)

What about when you use the MarketDataEventArgs? If you have already done this can I please see what you are doing now?

Ahm, sure, can you see the email that i sent? (i don't want to publish all my hours of learning and programming here in the open)
I think i sent the code with 1 lock, Depth lock in both. Anyway, it's the same, put the lock of Data in the MarketData, the Depth in the MarketDepth, the other way around, both or none.
The result is the same, the OnMarketData() sent the information after the Operation.Remove of OnMarketDepth()

[Fernand0]

the strange thing is that this works...

https://ninjatrader.com/support/foru...34#post1035534

but that's on simulated data feed... and for some reason the remove Price is "0.00"

[NinjaTrader_JoshG]

I never got your email. Can you please send it in again? PlatformSupport(AT)NinjaTrader(DOT)com
Please be sure to put "RE:Josh" in the subject line.

[NinjaTrader_JoshG]

Thanks for sending that in. I have some follow up questions for you so that I can better understand what you are trying to accomplish.

First, why are you locking at all?

they are not capable to receive the events in the right order.

What is the correct order for these methods to be called?

lock(obj.Instrument.SyncMarketData)
lock(obj.Instrument.SyncMarketDepth)

- What is obj.Instrument.Sync.MarketData and obj.Instrument.Sync.MarketDepth?

[Fernand0]

I have some follow up questions for you so that I can better understand what you are trying to accomplish.

I need the right order to feed another algorithm
why things have order? well for start.. because otherwise it would be a chaos... the data don't need just an structure.. it needs order too

First, why are you locking at all?

did you try not locking? is a mess.... you get updates or transactions AFTER the remove, for example, you get updates BEFORE the price is added.

What is the correct order for these methods to be called?

I told you in the email, that the only thing that it was supposed to "fix", is the OnMarketData(), delivering the data LATE, it puts the Print in the buffer, but late
If you check the Print, just keep the "time" test. The others were sketchs...

private DateTime time = DateTime.Now;
        private void Output(Row row) // TODO keep finding errors
        {

            if (time > row.Time)
                Print("ERROR: running late");

            Print(string.Format("{0:HH/mm/ss.fff}        {1}[{2:00}]:        {3:0.00}            ->        {4:000}            ({5})",
                    row.Time, row.MarketType, row.Position, row.Price, row.Volume, row.Procedure).Replace("/", ":"));

            time = row.Time;

        }

You will see in the Ouput window, "ERROR: running late", because it printed a "Transact" late. You can check the DateTime, you will se the list, (example)"100, 100, 120, 120, 80, 150, 150". The 80ms, it's the print from OnMarketData(), so, why is after 120ms? and not at the beginning? I need to receive the data in order.
Of course most of the time it works fine, but there are exceptions. If you keep the indicator running eventually you will see the msg "ERROR: .." in the Output

- What is obj.Instrument.Sync.MarketData and obj.Instrument.Sync.MarketDepth?

https://ninjatrader.com/support/help...vel_ii_dat.htm

protected override void OnMarketDepth(MarketDepthEventArgs e)
        {
            // protect e.Instrument.MarketDepth.Asks and e.Instrument.MarketDepth.Bids against in-flight changes
            lock (e.Instrument.SyncMarketDepth)
            {

Hope it helped you.

[NinjaTrader_JoshG]

Thanks for the details.

Let me rephrase my question, 'what order do you believe is the proper order?' These two methods are called on updates to level 1 and level 2 data, and it's noted in the help guide these are not guaranteed to be in order.

Also, "Transact" is not a documented operation. Why are you concerned with this operation?

[Fernand0]

it's noted in the help guide these are not guaranteed to be in order.

Then, if there is no way to change this, even making some arrangements on the platform, there is nothing else to talk about.
Thats why the title is "OnMarketData & OnMarketDepth should be the same method", to eradicate differences, a change is a change, the level should not matter

Also, "Transact" is not a documented operation. Why are you concerned with this operation?

That's why i created my own enum, and i am concerned with this operation because if you analyze the data, you need to know how many contracts were hitted and how many were removed.
There are 100 contracts(ES) that disappear, ok, what happened? 100 contracts were hitted or a big player removed 100 contracts?

[NinjaTrader_JoshG]

Thanks for the details. I have submitted this as a feature request to the Development Team. I will follow here up with an internal tracking number for your reference as soon as it is created.

[Fernand0]

thanks Josh

[NinjaTrader_JoshG]

Thanks for your patience.

The internal tracking number for your feature request is SFT-3592. Please reference this internal tracking number if you ever have questions regarding this feature request.

When a feature request is implemented, you'll find it in the release notes:

http://ninjatrader.com/support/helpGuides/nt8/en-us/?release_notes.htm