Unmanaged: Limit orders matching Bid or Ask, don't get filled and but get rejected

[pstrusi]

This seems to be an important and critical matter that should be addressed as soon as possible.

I'm doing live simulations of a simple Strategy that runs under unmanaged approach, which eventually submits limit orders with prices that most of the time matches or pass the best Bid or Ask. The chosen instrument to run the live simulations is the EURUSD cash. The data connection is to IB through the latest TWS ( build 967.1f ). So here an example:

I start the Strategy, it's synchronized to a 100.000 Long position, and then a pops up a sell signal that indicates: Reverse Position; so a sell limit order is submitted, here a code extract used:

if ( psignal > 0 )
{
	// Open or Reverse to a Long position
}
if ( psignal < 0 )
{
        // Open or Reverse to a Short position
	// Determines the position and submit the order needed.
	if ( ( Position.MarketPosition == MarketPosition.Long || Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Short && Qty > Position.Quantity) ) )
	{ 
            if ( setshort == null )
	    {	
		sellprice = GetCurrentBid();
		if ( Position.MarketPosition == MarketPosition.Long )
		{
			setshort = SubmitOrderUnmanaged(0, OrderAction.Sell, OrderType.Limit, (Qty+Position.Quantity), sellprice, 0, "", "SHORT");
		}
		else if ( Position.MarketPosition == MarketPosition.Flat || (Position.MarketPosition == MarketPosition.Short && Qty > Position.Quantity) )
		{
			setshort = SubmitOrderUnmanaged(0, OrderAction.Sell, OrderType.Limit, (Qty-Position.Quantity), sellprice, 0, "", "SHORT");
		}
		else
		{
		}						
	    }
	}
}

The order is submitted, accepted and put to work, then, the issue: a rejected state ( that is managed properly by the Strategy ):

Sim101, Limit price can't be smaller than current bid. affected Order: Sell 200000 Limit @ 1.18473

Working with forex, at 1/10 pips, ticks go pretty fast and change constantly, so it's normally expected that whenever an order is submitted at the best Bid or Ask of the moment, milliseconds later, don't match exactly with the new Bid or Ask, most of the time the order price overpass them, under this circumstance they should get filled immediately or pretty fast, and not get rejected because it doesn't match exactly the Bid or Ask. So, this is giving me a hard time cause I don't know if it's only a simulator issue, or even worse, that it would happen as well working with a real broker account.

So, I'm looking forward of your ideas, suggestions, trials, or solution for this issue.

Thanks in advance

[NinjaTrader_AlanP]

Hello pstrusi,

Thank you for your post.

Short Limit orders would need to be above the Bid price. Limit orders are set to enter at the price they are set or better. If you wish to go below the market for your short you would need to use a Stop Market or a Stop Limit order.

Please let me know if you have any questions.

[pstrusi]

Hello pstrusi,

Thank you for your post.

Short Limit orders would need to be above the Bid price. Limit orders are set to enter at the price they are set or better. If you wish to go below the market for your short you would need to use a Stop Market or a Stop Limit order.

Please let me know if you have any questions.

I understand your suggestions as a workaround to this issue that might be a solution and I'll try them; but you must consider that the problem is not a deliberate one because I want to go below market, it's because the Bid-Ask in Forex is so fast and changing that's impossible that an order match a stable Bid-Ask levels. I understand NT8 must can't discriminate Limit Orders according to the instrument, but this problem is unavoidable trading at 1/10 pips.

1. Do you think that this issue would happen as well with the Broker's account live ? I wonder if Limit Oders from the IB-API itself would experience the same.

2. Why can a limit order not adjust itself to a better bid-ask and then get filled?

I'm gonna apply those suggestions, then trials and report once again.

Thanks

[pstrusi]

As suggested, I've substituted the old limit orders by the new ones StopLimit orders, and unfortunately the critical issue pop up once again, I'll describe one of them by steps:

1. I began the Strategy, it was synchronized to 100.000 Long Position and running flawlessly.

2. A sell signal pop up, the Stop limit sell order was submitted, then the first issue, order rejected, as following:
Sim101, Sell stop or sell stop limit orders can't be placed above the market. affected Order: Sell 200000 StopLimit @ 1.17991 x 1.17988. I note here, the very error message is at the same time wrong, cause it states "sell stop limit orders can't be placed above the market" when in reality the Sell order stop price was actually below at 1.17991 vs market 1.17994. This must not be happening.

3.I solved the email problem by choosing the recipient and sender as the same

4. When the order was rejected, I was able to confirm that the procedure to reset the order to null under OnOrderUpdate was successful.

5. Despite the new configuration to ignore AllErrors, the Strategy seemed to continue Active in the control center and the Output window activity, BUT in reality, regardless the new signals, the position continued to be the same permanently, a total limbo, dead for practical purposes.

So, we have 2 CRITICAL ISSUES

1. Despite the type of orders, it continues to be rejected no matter what

2. Despite it's instructed to continue active, the Strategy is not any longer really Active or really disabled but Active-Null, this is a very dangerous situation.

I began to think that this problem should be escalated to developers by now, because it seems a serious bug.

Looking forward special attention to this post.

[koganam]

As suggested, I've substituted the old limit orders by the new ones StopLimit orders, and unfortunately the critical issue pop up once again, I'll describe one of them by steps:

1. I began the Strategy, it was synchronized to 100.000 Long Position and running flawlessly.

2. A sell signal pop up, the Stop limit sell order was submitted, then the first issue, order rejected, as following:
Sim101, Sell stop or sell stop limit orders can't be placed above the market. affected Order: Sell 200000 StopLimit @ 1.17991 x 1.17988. I note here, the very error message is at the same time wrong, cause it states "sell stop limit orders can't be placed above the market" when in reality the Sell order stop price was actually below at 1.17991 vs market 1.17994. This must not be happening.

3.I solved the email problem by choosing the recipient and sender as the same

4. When the order was rejected, I was able to confirm that the procedure to reset the order to null under OnOrderUpdate was successful.

5. Despite the new configuration to ignore AllErrors, the Strategy seemed to continue Active in the control center and the Output window activity, BUT in reality, regardless the new signals, the position continued to be the same permanently, a total limbo, dead for practical purposes.

So, we have 2 CRITICAL ISSUES

1. Despite the type of orders, it continues to be rejected no matter what

2. Despite it's instructed to continue active, the Strategy is not any longer really Active or really disabled but Active-Null, this is a very dangerous situation.

I began to think that this problem should be escalated to developers by now, because it seems a serious bug.

Looking forward special attention to this post.

If you are seeking to reverse, you should be using either a Limit order or a Market order, not a StopLimit order. For the very reason that you are seeing, it is near impossible to use Stop orders when scalping, which if you are reversing in place, after a price is breached, is pretty much a scalping action.

[NinjaTrader_AlanP]

Hello pstrusi,

You can not submit limit offers through the market in FX, this is exchange policy. If this is important to you, I suggest you look into CME FX futures products, as CME allows you to submit limit offers through the market. Or you should use market orders.

If you submit a sell stop, it must be below the market. I suggest adding logic to your code which checks that the sell stop price is beyond x ticks from the current bid to avoid quick movements resulting in this order being rejected.

The order rejections you are getting are expected, this is not a bug.

Please let us know if you need further assistance.

[pstrusi]

If you are seeking to reverse, you should be using Limit order not a StopLimit order. For the very reason that you are seeing, it is near impossible to use Stop orders when scalping, which if you are reversing in place, after a price is breached, is pretty much a scalping action.

I've written several messages about what the Strategy does, even with the error messages, and given the non-sense filling problem, you suggested to me that what I have to do is working with Stop Limit Orders in order to prevent the non-filling issue; and now you tell me that I have to work with Limit orders again, are you kidding me ? Please let's be serious, this is a serious bug, don't waste more time.

Please escalate to NT developers team: WHY LIMIT ORDERS ARE NOT FILLED IN THE SIMULATOR, DESPITE THEY HAVE PRICES THAT MATCHES OR PASS THE BEST BID OR ASK ?

[pstrusi]

You can not submit limit offers through the market in FX, this is exchange policy. If this is important to you, I suggest you look into CME FX futures products, as CME allows you to submit limit offers through the market. Or you should use market orders.

I have many years in the markets, and I work with Interactive Brokers, so the Exchange is IB-IdealPro, here a link:

https://www.interactivebrokers.com/en/index.php?f=759

What you say is non sense, I've worked a lot with these kind of orders in NT7 without any issue whatsoever, so check that info again.

If you submit a sell stop, it must be below the market. I suggest adding logic to your code which checks that the sell stop price is beyond x ticks from the current bid to avoid quick movements resulting in this order being rejected.

I see you're not reading all info that I write. As you can check in the comments below, I put this:

2. A sell signal pop up, the Stop limit sell order was submitted, then the first issue, order rejected, as following:
Sim101, Sell stop or sell stop limit orders can't be placed above the market. affected Order: Sell 200000 StopLimit @ 1.17991 x 1.17988. I note here, the very error message is at the same time wrong, cause it states "sell stop limit orders can't be placed above the market" when in reality the Sell order stop price was actually below at 1.17991 vs market 1.17994. This must not be happening.

And even that sell stop limit order was below, it doesn't get fired either.

So , I can't think of another logic reason that there IS A BIG BUG, and given the delicate situation you should escalate these two issues for immediate evaluations.

I really hope you act as soon as possible, thanks

[NinjaTrader_AlanP]

Hello pstrusi,

What you say is non sense, I've worked a lot with these kind of orders in NT7 without any issue whatsoever, so check that info again.

The error you are receiving trading the Sim101 account, “Limit price can't be smaller than current bid. affected Order:” is expected. The SIM101 account has logic which abides by FX broker FXCM logic, which does not allow you to submit a limit offer through the market.

From Internal Notes:

Limit price can't be greater than current ask/bid.

This error will be received when placing a limit order on the wrong side of the market in simulation.

FXCM does not allow this.

From FXCM,

“A BUY Limit Entry order can only be placed as an order to buy below the current market rate.”
http://help.fxcm.com/markets/Platfor...-BUY-Limit.htm

On the matter of your rejected stop order,

the Sell order stop price was actually below at 1.17991 vs market 1.17994.

You are entering your stop .00003 from the best bid. With your many years in the markets I’m sure you’re aware that you’re putting your stop less than 1 futures tick from the best bid. I suggest you observe the order book for a grasp of how quickly this miniscule change in price can make your stop price invalid.

There is no bug here and the only issue is sloppy coding and a crude understand of markets.

Please let us know if you need further assistance.

[pstrusi]

Hello AlanP,

I've realized now why this fool issue in the platform. It's well known that NT works with several large institutions, each one of them has thousands of clients. As an example, you've cited, is the important Fxcm, but also there are many important ones, such as Interactive.brokers ( and I've said repeatedly that I work with it ), which by the way: has trading rules and liquidity mechanism very different from the narrow and rigid rules of Fxcm, and this is even more important when is about forex, trading at 1/10 pips.. So, having in mind that NT works with many institutions, it should have a simulator with "general trading rules" shared by the most of brokers, not just one of them. So, if NT doesn't change that, obviously the workaround is to have a paper account in IB and work with it.

So, now you know your basic conceptual issue about of markets and brokers rules in your system, is up to you what you decide to do. I think is in your best interest to make this software better.

[NinjaTrader_AlanP]

Hello pstrusi,

I will put a feature request in to remove the limitation with FX limit orders being rejected through the market.

[NinjaTrader_AlanP]

Hello pstrusi,

The feature request which would would allow limit orders to be submitted through the market on the SIM account for FX is being tracked under the following ID,

SFT-2710

[pstrusi]

Hello pstrusi,

The feature request which would would allow limit orders to be submitted through the market on the SIM account for FX is being tracked under the following ID,

SFT-2710

I really hope that this feature can be available in the next upgrade, it's really necessary for real-time trials when a paper account is not possible.

[NinjaTrader_AlanP]

Hello pstrusi,

This feature requests interest is being tracked by votes, I do not have any other information for you.

Please let us know if you need further assistance.

[pstrusi]

Hello pstrusi,

This feature requests interest is being tracked by votes, I do not have any other information for you.

Please let us know if you need further assistance.

With all due respect, it seems to me that for the purpose to make NT8 able to simulate in the most realistic way possible, forex real trading, this feature should not be put through votes but including it as soon as possible, in simple terms: it should be a "must feature" not an optional one. At this time I have to test a "Flatten module" in forex though "unmanaged" limit orders, and I'm not able to do so cause that non-sense restrictions rules. I request to NT's dev. team to consider fix this limiting issue.

Looking forward having a better NT platform for Forex.