Handling Rejected orders under unmanaged

[pstrusi]

Back to NT7, rejected orders should be reset their State to Null cause the order wouldn't be any longer in the exchange or broker, so that signalname or order would be available for future triggers. So, having in mind that, I write simple lines in OnBarUpdate where the Order is reset to Null, hoping to clear it, however, the Order Status seems to be in a limbo that I don't quite understand cause I can't submit any new Short orders with that SignalName again. Here the code:

protected override void OnOrderUpdate(Order order, double limitPrice, double stopPrice, int quantity, int filled, double averageFillPrice, OrderState orderState, DateTime time, ErrorCode error, string nativeError)
		{			
			if ( order.Name == "SHORT" )
			{
				setshort = order;
			}
      		if ( setshort != null && setshort == order)				
			{	
				if (order.OrderState == OrderState.Rejected) 
				{
					setshort = null;
					SendMail ("[email protected]", "SHORT ORDER REJECTED ! ", Position.MarketPosition.ToString() + " " + Position.Quantity.ToString()+" at " +Position.AveragePrice.ToString("0.0000"));	
				}
				if (order.OrderState == OrderState.Cancelled || order.OrderState == OrderState.Filled)						
				{	
					setshort = null;		
				}						
			}
			
			
		}

So, I want to ask:

1.When an order is rejected from the exchange and it returns to NT8 under that State, what does exactly NT8 do to that Order object properties? I really thought that resetting its state would free the signalname and order for eventual future triggers again.

2. Besides reset to Null, what else is necessary to do in the code in order to reset it properly ?

Thanks in advance

[NinjaTrader_Jesse]

Hello,

Thank you for the post.

Resetting to null should be the only action you need to take on your Order variables, Internally the order would be updated as it is processed and NinjaTrader receives updates about that order.

You noted "however, the Order Status seems to be in a limbo that I don't quite understand cause I can't submit any new Short orders with that SignalName again"

What specifically are you seeing happen? Is your logic to null the order variable not happening, or are you seeing an actual error?

If you are not seeing the rejected state and your logic is not being enacted, this could be caused due to the multi-threaded nature of the platform. One item you could try would be to reference the orderState property from the overloads instead of the order.OrderState as this would be the sequential orderstate instead of the current order state. There is a note about this in the help guide:

The "order" method parameter represents the core order object updated by NinjaTrader
• The supplementary method parameters provide an updating value representing each order change in sequence. Think of this as the relevant information on the order at the time the state changed.
• Since the "order" method parameter represents the current order object state, it is possible for the updating values of that object to be out of sync with the corresponding method parameters during a particular order update event.


If you can provide more detail on what specifically is happening or any errors you see, that would be helpful in understanding what is happening.

I look forward to being of further assistance.

[pstrusi]

Hi Jesse, thanks for attending this post.

In order to give you an exact idea of what happened, I describe it below:

1. The logic submitted a Sell limit order but at an incorrect price below than currentbid() so the NT8 rejected the order giving these messages:

"Sim101, Limit price can't be smaller than current bid. affected Order: Sell 100000 Limit @ 1.18936"

"Order='2433b56b78f643ff8c8b3a7517aab3a9/Sim101' Name='SHORT' New state='Rejected' Instrument='EURUSD' Action='Sell' Limit price=1.18936 Stop price=0 Quantity=100,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='Order rejected' Native"

2. As you can see in my code, this situation is addressed resetting the order

if (order.OrderState == OrderState.Rejected) 
				{
					[B]setshort = null;[/B]
					SendMail ("[email protected]", "SHORT ORDER REJECTED ! ", Position.MarketPosition.ToString() + " " + Position.Quantity.ToString()+" at " +Position.AveragePrice.ToString("0.0000"));	
				}

3. After this, the strategy continues to detect signals for buy orders and also sell orders, but since the setshort = null; seems not done, any sell order was not submitted any more.

So, you have quoted something that could explain the issue:

One item you could try would be to reference the orderState property from the overloads instead of the order.OrderState as this would be the sequential orderstate instead of the current order state. There is a note about this in the help guide:

The "order" method parameter represents the core order object updated by NinjaTrader
• The supplementary method parameters provide an updating value representing each order change in sequence. Think of this as the relevant information on the order at the time the state changed.
• Since the "order" method parameter represents the current order object state, it is possible for the updating values of that object to be out of sync with the corresponding method parameters during a particular order update event.

So according to what you state, I'd like to know how to access reliably the most current orderState avoiding any previous sequential states.

Could you provide me an example to see the difference ?

Thanks

Looking forward

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

As you noted the logic is not happening, because of that I would suggest using prints to find why this is the case.

In my prior message, I noted one possible item to check would be to use the passed in orderState variable instead of order.OrderState as you have now. You would just need to replace "order.OrderState" with "orderState"

You can see this in the override you are using:

protected override void OnOrderUpdate(Order order, double limitPrice, 
double stopPrice, int quantity, int filled, double averageFillPrice, 
[B]OrderState orderState[/B], DateTime time, ErrorCode error, string nativeError)
{	
    if ([B]orderState [/B]== OrderState.Rejected) 
    {

Can you try making this change and see if this resolves the problem? If this does not help, I would suggest adding Prints to see if your other logic is the problem:

[B]if ( setshort != null && setshort == order)[/B]		
[U]// is this allowing the next few lines to exec[/U]ute? 		
			{	
                                [U]// can you see a print happening here? [/U]
				if (order.OrderState == OrderState.Rejected) 
				{
                                        [U]// can you see a print happening here? [/U]
					setshort = null;
					SendMail ("[email protected]", "SHORT ORDER REJECTED ! ", Position.MarketPosition.ToString() + " " + Position.Quantity.ToString()+" at " +Position.AveragePrice.ToString("0.0000"));	
				}

I look forward to being of further assistance.

[pstrusi]

Thanks Jesse for your detailed answer and suggestion, I'm sure that will solve the problem !

Regards!

[pstrusi]

Jesse, given your solution now I wonder:

Wouldn't it be ok to use

if (orderState == OrderState.XXXXX ) to detect the most current State of any order instead of using the sequential and more unreliable if (order.OrderState == OrderState.XXXXX ) ?

[NinjaTrader_Jesse]

Hello,

The sequential state would be orderState, the actual order object and its current state would be order.OrderState. It is suggested to use the sequential orderstate in case you need to see updates as they occur rather than the end result or current state of the order.

The supplementary method parameters (orderState) provide an updating value representing each order change in sequence. Think of this as the relevant information on the order at the time the state changed.
• Since the "order" (order.OrderState) method parameter represents the current order object state, it is possible for the updating values of that object to be out of sync with the corresponding method parameters during a particular order update event.

I am unsure if this is the exact solution but is one item to try, the prints would really be the best way to observe what specifically is happening with the logic to guide what needs to be changed.

I look forward to being of further assistance.

[pstrusi]

I did a trial, and once again it detected a Rejected condition, here what happened with messages, can you please explain to me how to avoid the last one ? is it because some syntax problem with the suggestion of

if (orderState == OrderState.Rejected) ?

Here the messages:

1. Sim101, Limit price can't be smaller than current bid. affected Order: Sell 200000 Limit @ 1.1842

2. There was an error sending your message: Syntax error, command unrecognized. The server response was: 4.3.2 STOREDRV.ClientSubmit; sender thread limit exceeded <fe73d6da-a5a7-4765-844e-eb6ef151d20f@DB6PR03MB2919.eurprd03.prod.outlook.c om> [Hostname=DB6PR03MB2919.eurprd03.prod.outlook.com]

I'm searching meanwhile

[NinjaTrader_Jesse]

Hello,

This error seems to be unrelated to the rejection and is related to your sending of the email.

I did a quick search for this error and some results came up indicating this was related to the use of outlook.com for sending emails.

If you comment out the email sending, do you still see the problem? If the problem goes away, have you tried a different email address to send from?

I look forward to being of further assistance.

[pstrusi]

Hello,

This error seems to be unrelated to the rejection and is related to your sending of the email.

I did a quick search for this error and some results came up indicating this was related to the use of outlook.com for sending emails.

If you comment out the email sending, do you still see the problem? If the problem goes away, have you tried a different email address to send from?

I look forward to being of further assistance.

Weird, cause I'm receiving emails with no problem. I'm still searching slowly every detail of what was wrong in this trial. I'll let you know if something found that I can't address it.
Thanks

[pstrusi]

I'm sorry but this stubborn behavior continues and I can't manage it. Here a new round:

Synchronized: 100.000 Short and then an attempt to reverse position, so here it continues below:

9/25/2017 13:52 NinjaScript NinjaScript strategy 'PRS/117729227' submitting order

9/25/2017 13:52 Order Order='0bcf28ae94574099ab2de33f1785c40d/Sim101' Name='LONG' New state='Submitted' Instrument='EURUSD' Action='Buy' Limit price=1.184 Stop price=0 Quantity=200,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='No error' Native error=''

9/25/2017 13:52 Order Order='0bcf28ae94574099ab2de33f1785c40d/Sim101' Name='LONG' New state='Rejected' Instrument='EURUSD' Action='Buy' Limit price=1.184 Stop price=0 Quantity=200,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='Order rejected' Native error='Limit price can't be greater than current ask.'

9/25/2017 13:52 Order Sim101, Limit price can't be greater than current ask. affected Order: Buy 200000 Limit @ 1.184

9/25/2017 13:52 Default NTliveps - Message sent successfully

9/25/2017 13:55 Order Order='63fd625e9c4c4ba48a320bea3dcef0c0/Sim101' Name='STOPSHORT' New state='Cancel submitted' Instrument='EURUSD' Action='Buy' Limit price=1.19169 Stop price=1.18919 Quantity=100,000 Type='Stop Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='No error' Native error=''

9/25/2017 13:55 Default Disabling NinjaScript strategy 'PRS/117729227'

9/25/2017 13:55 DB Error on executing DB command: System.Data.SqlServerCe.SqlCeException (0x80004005): A foreign key value cannot be inserted because a corresponding primary key value does not exist. [ Foreign key constraint name = FK__Strategy2Order__0000000000000403 ] at System.Data.SqlServerCe.SqlCeCommand.ProcessResult s(Int32 hr) at System.Data.SqlServerCe.SqlCeCommand.ExecuteComman dText(IntPtr& pCursor, Boolean& isBaseTableCursor) at System.Data.SqlServerCe.SqlCeCommand.ExecuteComman d(CommandBehavior behavior, String method, ResultSetOptions options) at System.Data.SqlServerCe.SqlCeCommand.ExecuteNonQue ry() at NinjaTrader.NinjaScript.StrategyBase.DbAdd() at NinjaTrader.NinjaScript.StrategyBase.DbUpdate() at NinjaTrader.Cbi.DB.DBThread()

9/25/2017 13:55 Order Order='63fd625e9c4c4ba48a320bea3dcef0c0/Sim101' Name='STOPSHORT' New state='Cancelled' Instrument='EURUSD' Action='Buy' Limit price=1.19169 Stop price=1.18919 Quantity=100,000 Type='Stop Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='No error' Native error=''

9/25/2017 13:55 Default NTliveps - Message sent successfully

9/25/2017 13:55 DB Error on executing DB command: System.Data.SqlServerCe.SqlCeException (0x80004005): A foreign key value cannot be inserted because a corresponding primary key value does not exist. [ Foreign key constraint name = FK__Strategy2Order__0000000000000403 ] at System.Data.SqlServerCe.SqlCeCommand.ProcessResult s(Int32 hr) at System.Data.SqlServerCe.SqlCeCommand.ExecuteComman dText(IntPtr& pCursor, Boolean& isBaseTableCursor) at System.Data.SqlServerCe.SqlCeCommand.ExecuteComman d(CommandBehavior behavior, String method, ResultSetOptions options) at System.Data.SqlServerCe.SqlCeCommand.ExecuteNonQue ry() at NinjaTrader.NinjaScript.StrategyBase.DbAdd() at NinjaTrader.NinjaScript.StrategyBase.DbUpdate() at NinjaTrader.Cbi.DB.DBThread()

So, here the questions:

1. Why dows NT disable the Strategy if I've set previously:
RealtimeErrorHandling = RealtimeErrorHandling.IgnoreAllErrors; ?

2. It says that the Strategy is been disabled but in reality, I continue to see it active and working in the Ninjascript Output, so how to explain this ?

3. I don't understand the others errors coming from DB, something wrong with the Database ?

Looking forward
Thanks

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

In this case, after changing the ignore rejections setting, have you completely removed all instances of the strategy from the control center and then reapplied it? It could be if you are using an old instance after the change, it is not been fully reloaded.

Regarding the database related errors, could you check that you have removed and re-added the strategy then retest? If this still occurs I may suggest doing a database repair or reset to see if this continues.

I look forward to being of further assistance.

[pstrusi]

In this case, after changing the ignore rejections setting, have you completely removed all instances of the strategy from the control center and then reapplied it? It could be if you are using an old instance after the change, it is not been fully reloaded.

I'm not sure what you mean by "all instances of the strategy from the control center and then reapplied it", what I've done after changes and compiling, it's removing the Strategy from the control center, reset the sim101 account, and then open a new Strategy again. If I'm wrong please indicate what you really mean.

Regarding the database related errors, could you check that you have removed and re-added the strategy then retest? If this still occurs I may suggest doing a database repair or reset to see if this continues.

Yes, as written above, I've removed and re-added the strategy again for a new trial. I imagine that I cannot repair or reset the database while it's working on markets open, right ?

What I can't address is how to prevent that NT disables the Strategy by default behavior when is set NOT to. This is maybe, the most important issue that needs attention.

Another important critical issue here, Limit orders don't execute at prices that are better than actual Bid-Ask, why ?
Working with 1/10 pips, pretty often the buy price is above CurrentAsk, or a sell price is below the CurrentBid, so what I can't understand is, that despite it's pretty obvious that at those prices the Limit order should be processed immediately at the best bid or ask, it gives me an error. This needs attention urgently. The logic of this code processing limit orders under unmanaged was tested for a long time without any issue in NT7, yet in NT8 there must be something that it is preventing a normal behavior.

Looking forward

Thanks

[NinjaTrader_Jesse]

Hello,

Yes by removing all instances from the control center I just mean to remove all of the strategies you see in the control center, by your reply it sounds like you did this.

It wouldn't be expected that the strategy is disabled if you are using IgnoreAllErrors:

if (State == State.Configure)
{
    RealtimeErrorHandling = RealtimeErrorHandling.IgnoreAllErrors;
}

Can you ensure you have this in the same State as listed in the help guide?

https://ninjatrader.com/support/helpGuides/nt8/en-us/realtimeerrorhandling.htm

If you are still seeing this happen, I would suggest after the market hours, repair the database using the instructions below, and restart the platform. After doing so reapply the script and ensure you have complied with the IgnoreAllErrors in the State.Configure. Do you still see the same error?

• Disconnect NinjaTrader from any open connections via Control Center > Connections > Disconnect
• From the NinjaTrader Control Center window select the menu Tools > Database Management
• Click on 'Repair DB'
• Press the "Repair" button
• Restart NinjaTrader

Regarding your other questions, let's try to resolve the initial questions you have asked before branching off to other questions. We can address the orders in a separate question once we have moved along.

I look forward to being of further assistance.

[pstrusi]

Hi Jesse, thanks for the answer, I think that the initial problems have been solved, now I think we might need another post for a major issue, which is described at the final.

Well, I see you've confirmed that this instruction ( RealtimeErrorHandling = RealtimeErrorHandling.IgnoreAllErrors ) should go in State.Configure, I had it in State.SetDefaults , cause in the guide you can read:
Warning: This property should ONLY bet set from the OnStateChange() method during State.SetDefaults or State.Configure , so I imagine that this point should be solved but you should amend the guide and remove State.SetDefaults as valid.

I've repaired the database exactly as you've instructed, and start a new instance, the rejection repeated once again.

Edit: A new important and critical issue was detected during this post, and since it's a different subject that needs good attention, I've moved it to a new post, here:

https://ninjatrader.com/support/forum/showthread.php?t=104584

Looking forward to your findings of it

Thanks