Handling Rejected orders under unmanaged

[NinjaTrader_Jesse]

Hello,

Thank you for confirming this was resolved with changing the state used.

I will report that product management for review.


I look forward to being of further assistance.

[pstrusi]

Well, Jesse I did a couple of trials with the new configuration ( RealtimeErrorHandling = RealtimeErrorHandling.IgnoreAllErrors; under State.Configure ), and unfortunately things are not corrected. I'll describe one of them by steps:

1. I began the Strategy, it was synchronized to 100.000 Short Position and running flawless

2. A buy signal pop up, the limit buy order was submitted, then the first issue, order rejected, as following: Sim101, Limit price can't be greater than current ask. affected Order: Buy 200000 Limit @ 1.17705

3. It gives an error related to email service that I don't know the cause, since my email service is well configured, tested and works most of the time OK. Here the message:
There was an error sending your message: Syntax error, command unrecognized. The server response was: 4.3.2 STOREDRV.ClientSubmit; sender thread limit exceeded <0510df21-53b2-4b1a-9eaf-a7215e96cc98@VI1PR03MB2928.eurprd03.prod.outlook.c om>

4. After all, the email was sent. Here the confirmation message:
NTliveps - Message sent successfully

5. When the order was rejected, I was able to confirm that the procedure to reset the order to null under OnOrderUpdate was sucessful.

6. Despite the new configuration to ignore AllErrors, the Strategy seemed to continue active, but regardless the new signals, the position was unchanged, no new orders was submitted, in the limbo.

So, this left me pretty confused and stressed. Could you take a look of this problem ?

Thanks in advance


Last Update: Since you've suggested a possible workaround to avoid the rejected order situation, perhaps the problem above might be solved. I'll let you know

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

Yes please let me know if you have resolved this as well.

It sounds like this is now getting further into your logic than it was before. Previously you said the script was being disabled, now it is not and your order is being set to null. Have you at this point used Print statements to find why the orders after this point are not being submitted? This is what I would suggest doing next or to print the values you are using in your conditions to see why the orders after this point are not submitted. I am not able to see anything from the details you have provided that would indicate a specific problem at this point.


Regarding the email error, have you at this point tried a different email service to see if this is related to the outlook email specifically? As noted I had seen some errors related to outlook.com specifically that relayed this message.


I look forward to being of further assistance

[pstrusi]

It sounds like this is now getting further into your logic than it was before. Previously you said the script was being disabled, now it is not and your order is being set to null. Have you at this point used Print statements to find why the orders after this point are not being submitted?

The Script was "disabled" for practical purposes because it was left "officially" Active with a position permanently and didn't work anymore, so despite it was shown as Active in the control center and the Output window. The problem never was in reality that the Order wasn't changed to Null, I've confirmed today that this always was done, so the problem is not in the logic, the problem is in NT and it worries me

Regarding the email error, have you at this point tried a different email service to see if this is related to the outlook email specifically? As noted I had seen some errors related to outlook.com specifically that relayed this message.

I solved the email problem by choosing the recipient and sender as the same

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

I am unsure of your expectation surrounding the Ignore Rejections. The script should be left enabled along with any positions that were not exited by the rejected order. The platform wont take any action for the positons you have if the order was rejected, the strategy would just remain enabled and do nothing more until your logic tells it to.

You would need to handle the rejection and then figure out why it was rejected, correct the order and re submit it if that is your goal for this situation. NT will not automatically resubmit your order or do anything, it would be up to your logic to further handle any other orders from that point.

You noted:

The problem never was in reality that the Order wasn't changed to Null, I've confirmed today that this always was done

Are you saying that your logic is not setting the order to null? From your prior posts, you are setting the order to null in your logic, was the condition to set the order to null not true preventing this from happening?

Based on your description it sounds like the Ignore did as it was supposed it and the remainder of your logic had not continued due to some reason. Have you used Prints at this point to identify what part of your logic is failing after the reject? I couldn't comment on any of the other conditions you have as they have not been shown in this thread, the best solution would be to add prints to find where the logic is failing.

I look forward to being of further assistance.

[pstrusi]

I am unsure of your expectation surrounding the Ignore Rejections. The script should be left enabled along with any positions that were not exited by the rejected order. The platform won't take any action for the positons you have if the order was rejected, the strategy would just remain enabled and do nothing more until your logic tells it to.

My expectation logically is that the Strategy continues to be active, working, generating signals and submitting the respective orders. The problem is that the Strategy seems to continue Active, in fact, I can check in the Output windows how it flows the different signals, and despite the rejected order state was set to Null correctly, this kind of orders are no longer submitted, furthermore holding a permanent position. This is the critical point: previous the rejection, with order states set to null, any signal is processed accordingly with no issues, BUT after the rejection, and supposedly the order states were again set to Null, any order of the same kind of the rejected one, is never submitted again, so that's why the Limbo. so, what I don't understand is WHY is not the order submit again?

You would need to handle the rejection and then figure out why it was rejected, correct the order and re submit it if that is your goal for this situation. NT will not automatically resubmit your order or do anything, it would be up to your logic to further handle any other orders from that point.

I handle the rejection as it needs to, resetting its State to Null, and the logic continues to work generating signals and sending out the respective orders. I don't ask NT to do my job, what I really ask NT8 is the most important function: PROCESS THE ORDER THAT IS WELL CONFIGURED, AT PRICES THAT ARE MATCHABLE IMMEDIATELY, but it fails here for some unknown reason, and this is a big issue. Do this test in your very front end and see. Set Calculate to TickByTick, in fact set your chart for better granularity and try under unmanaged approach send any Limit or Stop Limit orders and you'll see. This not an issue of the Strategy's logic, it's within NT simulator filling module, I think.

Are you saying that your is not setting the order to null? From your prior posts, you are setting the order to null in your logic, was the condition to set the order to null not true preventing this from happening?

I think in the previous messages, I've stated clearly that I set the order state to Null correctly, please let's move on.

Based on your description it sounds like the Ignore did as it was supposed it and the remainder of your logic had not continued due to some reason. Have you used Prints at this point to identify what part of your logic is failing after the reject? I couldnt comment on any of the other conditions you have as they have not been shown in this thread, the best solution would be to add prints to find where the logic is failing.

As I've said previously above, I've checked in the NinjaScript Output window, that my Strategy logic continues to work flawlessly, trigerring different signals, BUT for some unknown reason linked to NT simulator, the Order is not submitted again, because IT SEEMS that the order state continues to be different than Null, DESPITE i'VE CONFIRMED BY PRINTS THAT THIS STEP WAS DONE.

We're dealing here with 2 issues, the most important is the failing to fill orders cause non-sense reason, and the other is something happens internally to the Oder State or the Order rejected itself keeps around, I don't know. This why I come to this forum, I really hope to get helped here as always you do.

Looking forward

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

I noted that you mentioned:

My expectation logically is that the Strategy continues to be active, working, generating signals and submitting the respective orders. The problem is that the Strategy seems to continue Active, in fact, I can check in the Output windows how it flows the different signals, and despite the rejected order state was set to Null correctly, this kind of orders are no longer submitted, furthermore holding a permanent position.

As I am unsure of the overall logic you are using, this is something you will still need to review to find the cause. Have you at this point tried to extract the logic that is not happening correctly to form a sample of just the error situation? If you have not, I would suggest doing this next. As our support cannot debug your script for you, you can instead extract the logic performing incorrectly and form a sample of only that. If that does not make it apparent to what the problem is, please provide that sample for further review.

I also noted:

Do this test in your very front end and see. Set Calculate to TickByTick, in fact, set your chart for better granularity and try under unmanaged approach send any Limit or Stop Limit orders and you'll see. This not an issue of the Strategy's logic, it's within NT simulator filling module, I think.

This would go back to my last comment. We would likely need some sample that shows the situation in error along with the steps to cause the problem to further address this. I wouldn't be able to build a script from your description to test this myself, but if you have a sample that demonstrates the problem with the steps to show the problem, you are welcome to post that so we can review the logic being used. Please note, we would not want the full strategy, instead, we would want to see an isolated example of the problematic logic separate from the original script as an importable file that can compile.

Using this approach will allow you to review the logic being used and if it is not apparent why there is a problem this would also highlight the logic used preventing support from needing to debug your entire script.



Please let me know if I may be of further assistance.

[pstrusi]

As I am unsure of the overall logic you are using, this is something you will still need to review to find the cause. Have you at this point tried to extract the logic that is not happening correctly to form a sample of just the error situation? If you have not, I would suggest doing this next. As our support cannot debug your script for you, you can instead extract the logic performing incorrectly and form a sample of only that. If that does not make it apparent to what the problem is, please provide that sample for further review.

I have to put this extract again, use the logic and you tell me:
This is the critical point: previous the rejection, with order states set to null, any signal is processed accordingly with no issues, BUT after the rejection, and supposedly the order states were again set to Null, any order of the same kind of the rejected one, is never submitted again

So thinking logically:
1. Previous the issue: When all order state are set to null, and a trigger signal pops up, the order is submitted correctly.

2. The rejection occurs, afterward, the order state of the rejected order is set again to NULL, so it's like from initial state again.

3. The Algo continues to work, calculating and triggering signals normally, but despite all conditions are set for submitting any new order, for some unknown reason, this doesn't happen again. I've speculated that perhaps is related to the order state, that after I set it to null, something happens that prevent conditions for new orders to be submitted.

.. but if you have a sample that demonstrates the problem with the steps to show the problem, you are welcome to post that so we can review the logic being used. Please note, we would not want the full strategy, instead, we would want to see an isolated example of the problematic logic separate from the original script as an importable file that can compile.

Since you say something reasonable, you don't debug custom script, I don't see much reason to send any code and overload you.

I think I can't express more clearly what's actually going on after all I've written here. There are two major issues, easy to spot for anyone, and I've proved that this not cause the non-sense CME forex restrictions or better stop limit orders than limit orders...etc something major within NT live simulator is not right. In NT7 I've worked with Limit orders flawlessly, but NT8 is becoming a nightmare.

This is a delicate situation for a new software as NT8 that must be solved as soon as possible.

[NinjaTrader_Jesse]

Hello,

Thank you for the additional details.

At this point, I couldn't really say what may be happening without further addressing the logic you are using.

As I don't know the extent of the logic that was used to produce the problem, I can't really confirm if this is the platforms fault or your logic. If you are able to export a sample of the logic in error, I would be happy to continue looking into this item. Otherwise, I could only suggest that you further reduce the code to find the problem.

The Algo continues to work, calculating and triggering signals normally, but despite all conditions are set for submitting any new order, for some unknown reason, this doesn't happen again. I've speculated that perhaps is related to the order state, that after I set it to null, something happens that prevent conditions for new orders to be submitted.

Yes, as noted previously it sounds like something in the logic is holding the future orders up. Perhaps these speculations are correct, you would need to confirm this by debugging the script and adding Prints to confirm this is the case.

I think I can't express more clearly what's actually going on after all I've written here.

Unfortunately, in this case, the description is not becoming more clear to me as you describe more details. At this point, I would very likely need to see an excerpt of what logic is being used to create the error you are having to explain it further. Without that information, it would only be a guess as to the logic you used and why it is working how it is now.

If something is wrong with the platform then we would want to fix it. Our support can assist in providing items for you to try on your own to correct it, but in some cases where the solution is not easily found, we may need to request a simplified example to further review what is happening with the logic used.

We mention that we do not debug scripts as commonly full scripts are sent to support, we would instead ask that you first try to debug the script yourself eliminating code that is not required to see the problem or reduce it to a sample size. From that point, we could look further into the sample to advise what may need to be changed or to report any found bugs to development for review.

Please let me know if I may be of further assistance.

[pstrusi]

It doesn't make sense to me explaining again the rejected order handling problem, yet there's the most critical issue that I don't see any explanation or solution.

UNDER UNAMANAGED, LIMIT ORDERS SUBMITTED IN FOREX ( IB.IDEAL PRO ) AT RIGHT PRICES TO GET FILLED IMMEDIATELY, NOT ONLY ARE LEFT UNFILLED BUT REJECTED BY A NON-SENSE EXPLANATION. THIS OCCURS IN LIVE SIMULATION

This has nothing to do with customers code, but NT itself.

Better address it as soon as possible, cause this is going to escalate to more people for sure.

[pstrusi]

In regards to the rejected order handling problem and if you still care about it, I've found this in the last round:

1. The initial rejected order error

Order='8d18bb30675b4c60914d574db2c97265/Sim101' Name='LONG' New state='Rejected' Instrument='EURUSD' Action='Buy' Limit price=1.17366 Stop price=0 Quantity=200,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='Order rejected' Native error='Limit price can't be greater than current ask.'
Here the Strategy set that order.state from rejected to null

2. Then, the error message from system :

Sim101, Limit price can't be greater than current ask. affected Order: Buy 200000 Limit @ 1.17366

3. And finally this message that it seems out of place

Order='8d18bb30675b4c60914d574db2c97265/Sim101' Name='LONG' New state='Submitted' WHY DOES THIS COME AFTER THE REJECTION ERROR ? Instrument='EURUSD' Action='Buy' Limit price=1.17366 Stop price=0 Quantity=200,000 Type='Limit' Time in force=GTC Oco='' Filled=0 Fill price=0 Error='No error' Native error=''

This perhaps might explain, why despite the logic set the order.state to null, NT didn't submit the new orders coming from new signals.

After all, I knew it that something happened internally that prevented the new orders to trigger, now I know it, and I'll try to correct this NT flaw with some logic.

It's up to you if you search or report the issue where it needs to.

[NinjaTrader_Jesse]

Hello,

I just wanted to provide a quick reply back on this item.

After further review, it appears you have some other threads open relating to some of the items you are explaining in this thread. To avoid duplication of troubleshooting, I just wanted to post back and let you know I am still interested in resolving the problem you are having with your logic failing after the rejection. I would still need to see some form of an example of the logic used to further review the case. If you are able to provide a sample of that situation, I would be happy to continue working with you on this item.

For the other questions, I would suggest continuing to work with the representatives in the other threads on those specific questions.

I look forward to being of further assistance.

[pstrusi]

Hi Jesse, thanks for your assistance again in this situation.

Here you can see the very basic sample of the code logic. Any comment, ideas...etc please let me know. Thanks in advance

protected override void OnBarUpdate()
		{	
			// Here is just the Logic that generates buy or sell signals as psignal=1 or psignal=-1, it doesn't have anything to do with orders.
			
			
			// This is the block that submits the order
			// if buy order and Algo is allowed to trade and it's full connected
			if ( psignal > 0 && stoptrading==0 && dataserver==1 )
			{
				if ( setshort != null )					
				{	
					CancelOrder(setshort);				
				}				
				if ( ( Position.MarketPosition == MarketPosition.Short || Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Long && Qty > Position.Quantity) ) )
				{
					if ( setlong == null )				
					{	
						buyprice = Close[0]+2*TickSize;
						if ( Position.MarketPosition == MarketPosition.Short )
						{
							setlong = SubmitOrderUnmanaged(0, OrderAction.Buy, OrderType.Limit, (Qty+Position.Quantity), buyprice, 0, "", "LONG");
						}
						else if ( Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Long && Qty > Position.Quantity) )
						{
							setlong = SubmitOrderUnmanaged(0, OrderAction.Buy, OrderType.Limit, (Qty-Position.Quantity), buyprice, 0, "", "LONG");
						}
						else
						{
						}						
					}
					else
					{
						if ( buyprice < Close[0] ) 
						{
							buyprice = Close[0]+TickSize;
							if ( Position.MarketPosition == MarketPosition.Short )
							{
								ChangeOrder( setlong, (Qty+Position.Quantity), buyprice, 0);	
							}
							else if ( Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Long && Qty > Position.Quantity) )
							{
								ChangeOrder( setlong, (Qty-Position.Quantity), buyprice, 0);
							}
							else
							{
							}
						}
					}	
				}
			}
			if ( psignal < 0 && stoptrading==0 && dataserver==1 )
			{
				if ( setlong != null )					
				{	
					CancelOrder(setlong);				
				}
				if ( ( Position.MarketPosition == MarketPosition.Long || Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Short && Qty > Position.Quantity) ) )
				{	
					if ( setshort == null )	
					{	
						sellprice = Close[0]-2*TickSize;
						if ( Position.MarketPosition == MarketPosition.Long )
						{
							setshort = SubmitOrderUnmanaged(0, OrderAction.Sell, OrderType.Limit, (Qty+Position.Quantity), sellprice, 0, "", "SHORT");
						}
						else if ( Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Short && Qty > Position.Quantity) )
						{
							setshort = SubmitOrderUnmanaged(0, OrderAction.Sell, OrderType.Limit, (Qty-Position.Quantity), sellprice, 0, "", "SHORT");
						}
						else
						{
						}						
					}
					else
					{
						if ( sellprice > Close[0] )
						{
							sellprice = Close[0];
							if ( Position.MarketPosition == MarketPosition.Long )
							{
								ChangeOrder( setshort, (Qty+Position.Quantity), sellprice, 0);				
							}
							else if ( Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Short && Qty > Position.Quantity) )
							{
								ChangeOrder( setshort, (Qty-Position.Quantity), sellprice, 0);
							}
							else
							{
							}
						}							
					}
				}
			}			
		}
		
		
		protected override void OnOrderUpdate(Order order, double limitPrice, double stopPrice, int quantity, int filled, double averageFillPrice, OrderState orderState, DateTime time, ErrorCode error, string nativeError)
		{
		  	if ( order.Name == "LONG" )
			{
				setlong = order;
			}
      		        if ( setlong != null && setlong == order)				
			{	
				if ( order.OrderState == OrderState.Rejected) 
				{
					setlong = null;
				}
				if (order.OrderState == OrderState.Cancelled || order.OrderState == OrderState.Filled)						
				{	
					setlong = null;		
				}
			}
			
			if ( order.Name == "SHORT" )
			{
				setshort = order;
			}
      		        if ( setshort != null && setshort == order)				
			{		
				if ( order.OrderState == OrderState.Rejected) 
				{
					setshort = null;
				}
				if (order.OrderState == OrderState.Cancelled || order.OrderState == OrderState.Filled)						
				{	
					setshort = null;		
				}
			}
		}