Overfill Issue

[Upswing]

Hi,

I just observed some overfill on one of my strategies. The strategie entered a long Position on NQ and sent TP/SL as it should. But then I got a Overfill warning and the strategy closed the position and stopped. I think this is reasonable behaviour after an overfill.

The question I have is why I had an overfill anyways?

Here is the log:

24.09.2014 17:15:00 Entered internal PlaceOrder() method at 24.09.2014 17:15:00: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Entry Long Trend' FromEntrySignal=''
24.09.2014 17:15:00 Entered internal SetStopTarget() method: Type=Target FromEntrySignal='' Mode=Price Value=4076,25 Currency=0 Simulated=False
24.09.2014 17:15:00 Amended target order: Order='0d532013baf049c9a87159fdc3034fb7/U1384937' Name='Profit target' State=PendingSubmit Instrument='NQ 12-14' Action=Sell Limit price=4076,25 Stop price=0 Quantity=1 Strategy='MarketRangerMix' Type=Limit Tif=Gtc Oco='NQ 12-14/0001010c.54220532.01.01-344' Filled=0 Fill price=0 Token='a522e08782fa450b833c4316543a8c8a' Gtd='01.12.2099 00:00:00'
24.09.2014 17:15:00 Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='' Mode=Price Value=4053,5 Currency=0 Simulated=False
24.09.2014 17:15:00 Amended stop order: Order='471c1fde8dcd4b23bdabc41b908c8d37/U1384937' Name='Stop loss' State=PendingSubmit Instrument='NQ 12-14' Action=Sell Limit price=0 Stop price=4053,5 Quantity=1 Strategy='MarketRangerMix' Type=Stop Tif=Gtc Oco='NQ 12-14/0001010c.54220532.01.01-344' Filled=0 Fill price=0 Token='cc3b28c0251e4044b295897a48be28b6' Gtd='01.12.2099 00:00:00'
24.09.2014 17:15:00 Entered internal SetStopTarget() method: Type=Target FromEntrySignal='' Mode=Ticks Value=0 Currency=0 Simulated=False
24.09.2014 17:15:00 Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='' Mode=Ticks Value=0 Currency=0 Simulated=False
**NT** An over fill was detected on order 'Order='1290187400/U1384937' Name='Profit target' State=Filled Instrument='NQ 12-14' Action=Sell Limit price=4076,25 Stop price=0 Quantity=0 Strategy='MarketRangerMix' Type=Limit Tif=Gtc Oco='NQ 12-14/0001010c.54220532.01.01-344' Filled=0 Fill price=0 Token='a522e08782fa450b833c4316543a8c8a' Gtd='01.12.2099 00:00:00'' generated by strategy 'MarketRangerMix/92ff0e1f68ae4dd8ab41451533002d7b' : This strategy will be disabled and NinjaTrader will attempt to cancel/close any strategy generated orders and positions. Please check your account orders and positions and take any appropriate action.
24.09.2014 17:15:01 CancelAllOrders: BarsInProgress=0
24.09.2014 17:15:01 CancelAllOrders: BarsInProgress=1
24.09.2014 17:15:09 Disable() called: strategy disabled
**NT** Disabling NinjaScript strategy 'MarketRangerMix/92ff0e1f68ae4dd8ab41451533002d7b'

And this is what I locked myself in OnExecution which seems to be good.
Market Position
Long
Average Position Price
4062
execution Price
4062
execution avg order Price
4062
TP
4076,25
SL
4053,5

Thanks a lot in advance! appreciate the support in this forum!

[NinjaTrader_ChelseaB]

Hi Upswing,

Overfills happen when an order is attempted to be cancelled that has already filled.

Are you using CancelOrder() in your script?

(There are other actions that can cause an order to be cancelled, but lets start with this)

[Upswing]

Hi Chelsea,

no i didn't use CancelOrder(). The same strategy on the same instrument worked fine yesterday.

I attached a screenshot of these orders which I saw in the Order tab. All orders were placed reasonable but the fill of the stop loss together with the overfill warning keep me confused.

E: I should mention that the fill came instantaneous but the price of NQ wasn't close to TP or SL at that moment.

[NinjaTrader_ChelseaB]

Hi UpSwing(),

Based on what is in the screenshot, it looks like the stop loss filled, which caused NinjaTrader to send out a cancellation, however, the profit target also filled before the cancellation could be done.


From the help guide on Risks of Electronic Trading with NinjaTrader.

"...It is possible (in rare occasions) that order(s) that are part of the OCO group will be filled before the cancellation request has been acknowledged. ..."

http://www.ninjatrader.com/support/h...trading_wi.htm

[Upswing]

Hi Chelsea,

Yes I also think that is what happened but how could a Stop Sell Order with Stopprice 4053.5 can be filled? The price hasn't even been close to that since the position was opened.

[NinjaTrader_ChelseaB]

Hello,

Have you opened a chart to 1 tick bid data to see if the bid prices reached this high?

[Upswing]

Hi Chelsea,

You think we could have had a trade below 4053.5 and in the same instant the price jumped back to 4061.75 where the average fill price for the stoporder was? I hope there is a little more liquidity in the Nasdaq mini Future (NQ 12-14) during regular trading hours .

But to make sure I just looked into the tickchart right now and the bid is at least 7 points away from the stop loss ( 4053.5 ) for the rest of the day since position was opened at 9:15 am CST (5:15 pm german local time).

So the fill makes basicly no sense to me. I will observe it today as well and if I get a trade will let you know.

Thanks,
Upswing

[NinjaTrader_ChelseaB]

Hi Upswing,

(Did you actually set the Price based on parameter to Bid so that you are looking at only bid data?)

I double check this for you if you would like. What time zone are you in?

[Upswing]

Yes I did. Timezone is UTC+1 (Amsterdam,Berlin...).

I got some new strange trades on ES and NQ now . Yesterday ES worked fine. Both instruments run with the same strategy. But now it seems to be something different actually. I am really hoping that we can work this out somehow because I really enjoy using NT but somehow I get a completely different behaviour now that script is running in a real IB account.


Here is Log for NQ:

25.09.2014 15:49:00 Entered internal PlaceOrder() method at 25.09.2014 15:49:00: BarsInProgress=0 Action=SellShort OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Entry Short Trend' FromEntrySignal=''
25.09.2014 15:49:00 Entered internal SetStopTarget() method: Type=Target FromEntrySignal='' Mode=Price Value=4042 Currency=0 Simulated=False
25.09.2014 15:49:00 Amended target order: Order='82eaf2765c624aca8f8bd71930b94ae3/U0000000' Name='Profit target' State=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=4042 Stop price=0 Quantity=1 Strategy='MarketRangerMix' Type=Limit Tif=Gtc Oco='NQ 12-14/0001010c.54234874.01.01-623' Filled=0 Fill price=0 Token='60eaea58a60d4903b0710c3f6115304e' Gtd='01.12.2099 00:00:00'
25.09.2014 15:49:00 Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='' Mode=Price Value=4064,75 Currency=0 Simulated=False
25.09.2014 15:49:00 Amended stop order: Order='15d61edde7384445b3ca25010066737f/U0000000' Name='Stop loss' State=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4064,75 Quantity=1 Strategy='MarketRangerMix' Type=Stop Tif=Gtc Oco='NQ 12-14/0001010c.54234874.01.01-623' Filled=0 Fill price=0 Token='67e7c748beeb4b90a5994f3b92398556' Gtd='01.12.2099 00:00:00'

Profit target should have been at 4042 which seems to be correct in the log.

Log for ES trade:

25.09.2014 15:52:00 Entered internal PlaceOrder() method at 25.09.2014 15:52:00: BarsInProgress=0 Action=SellShort OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Entry Short Trend' FromEntrySignal=''
25.09.2014 15:52:00 Entered internal PlaceOrder() method at 25.09.2014 15:52:00: BarsInProgress=0 Action=SellShort OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Entry Short Trend' FromEntrySignal=''
25.09.2014 15:52:01 Entered internal SetStopTarget() method: Type=Target FromEntrySignal='' Mode=Price Value=1973,25 Currency=0 Simulated=False
25.09.2014 15:52:01 Amended target order: Order='a4b62530534f46f986809e43c4d9ac6e/U0000000' Name='Profit target' State=PendingSubmit Instrument='ES 12-14' Action=BuyToCover Limit price=1973,25 Stop price=0 Quantity=1 Strategy='MarketRangerMix' Type=Limit Tif=Gtc Oco='ES 12-14/0000fe5f.542355b9.01.01-323' Filled=0 Fill price=0 Token='e7b0162996ee4a11a274d172fedc7703' Gtd='01.12.2099 00:00:00'
25.09.2014 15:52:01 Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='' Mode=Price Value=1979,25 Currency=0 Simulated=False
25.09.2014 15:52:01 Amended stop order: Order='f74477741cc848f4b34e7de203be6924/U0000000' Name='Stop loss' State=PendingSubmit Instrument='ES 12-14' Action=BuyToCover Limit price=0 Stop price=1979,25 Quantity=1 Strategy='MarketRangerMix' Type=Stop Tif=Gtc Oco='ES 12-14/0000fe5f.542355b9.01.01-323' Filled=0 Fill price=0 Token='aabdb0c55c204f6bb81b4f61188bf6e6' Gtd='01.12.2099 00:00:00'

This time SL and TP are where transmitted wrong but seem correct in the log.

How can there be a discrepancy between the log and the orderstab?

Attached is what I see in the Orders tab which is actually what was transmitted to IB.

Appreciate the help,

Upswing

[NinjaTrader_ChelseaB]

Hi Upswing,

I'd like to take a look at the full log that shows the accept and working for this profit target on the NQ 12-14.

I am also seeing the profit target has the pending submit as 4042, however, the accepted and working (or any pending changes) are not in this output.

[Upswing]

I just recognized that there is a discrepancy between the things I saw in the Output Window generated by traceorder(what I posted) and the logfiles. Thought those were the same.

In the logfiles I see another StopLoss order send which represents the scenario in the orders tab with a stopprice of 4056.25.

The things i see in output window kind of suggest me that everything works correctly. When I reload the strategy the historic orders for yesterdays trade are correct as well. In live papertrade on Sim101 everything worked fine for weeks.

Any ideas on that?



This is the snippet from the logfile:

25.09.2014 15:49:00|1|32|Order='6a8dd27d59af45d78ba1bd229d7c5 923/U0000000' Name='Entry Short Trend' New state=PendingSubmit Instrument='NQ 12-14' Action=SellShort Limit price=0 Stop price=0 Quantity=1 Type=Market Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|2|4|Order Event Warning:Attribute 'Outside Regular Trading Hours' is ignored based on the order type and destination. PlaceOrder is now being processed. 2109
25.09.2014 15:49:00|1|32|Order='1290187406/U0000000' Name='Entry Short Trend' New state=Accepted Instrument='NQ 12-14' Action=SellShort Limit price=0 Stop price=0 Quantity=1 Type=Market Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187406/U0000000' Name='Entry Short Trend' New state=Working Instrument='NQ 12-14' Action=SellShort Limit price=0 Stop price=0 Quantity=1 Type=Market Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187406/U0000000' Name='Entry Short Trend' New state=Filled Instrument='NQ 12-14' Action=SellShort Limit price=0 Stop price=0 Quantity=1 Type=Market Filled=1 Fill price=4056,25 Error=NoError Native error=''
25.09.2014 15:49:00|1|16|Execution='NQ 12-14/0001010c.54234874.01.01' Instrument='NQ 12-14' Account='U0000000' Exchange=Globex Price=4056,25 Quantity=1 Market position=Short Operation=Insert Order='1290187406' Time='25.09.2014 15:49:00'
25.09.2014 15:49:00|1|32|Order='15d61edde7384445b3ca250100667 37f/U0000000' Name='Stop loss' New state=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4056,25 Quantity=1 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='82eaf2765c624aca8f8bd71930b94 ae3/U0000000' Name='Profit target' New state=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=4056,25 Stop price=0 Quantity=1 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187407/U0000000' Name='Stop loss' New state=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4056,25 Quantity=1 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187407/U0000000' Name='Stop loss' New state=Accepted Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4056,25 Quantity=1 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187407/U0000000' Name='Stop loss' New state=PendingChange Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4064,75 Quantity=1 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|2|4|Order Event Warning:Attribute 'Outside Regular Trading Hours' is ignored based on the order type and destination. PlaceOrder is now being processed. 2109
25.09.2014 15:49:00|1|32|Order='1290187407/U0000000' Name='Stop loss' New state=Accepted Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4064,75 Quantity=1 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187407/U0000000' Name='Stop loss' New state=Cancelled Instrument='NQ 12-14' Action=BuyToCover Limit price=0 Stop price=4064,75 Quantity=1 Type=Stop Filled=0 Fill price=0 Error=NoError Native error='Order Canceled - reason: (202)'
25.09.2014 15:49:00|1|32|Order='1290187408/U0000000' Name='Profit target' New state=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=4056,25 Stop price=0 Quantity=1 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187408/U0000000' Name='Profit target' New state=Accepted Instrument='NQ 12-14' Action=BuyToCover Limit price=4056,25 Stop price=0 Quantity=1 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187408/U0000000' Name='Profit target' New state=Working Instrument='NQ 12-14' Action=BuyToCover Limit price=4056,25 Stop price=0 Quantity=1 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
25.09.2014 15:49:00|1|32|Order='1290187408/U0000000' Name='Profit target' New state=Filled Instrument='NQ 12-14' Action=BuyToCover Limit price=4056,25 Stop price=0 Quantity=1 Type=Limit Filled=1 Fill price=4056,25 Error=NoError Native error=''
25.09.2014 15:49:00|1|16|Execution='NQ 12-14/0001010c.54234875.01.01' Instrument='NQ 12-14' Account='U0000000' Exchange=Globex Price=4056,25 Quantity=1 Market position=Long Operation=Insert Order='1290187408' Time='25.09.2014 15:49:00'

[NinjaTrader_ChelseaB]

Hello Upswing,

Just to confirm, you did not post your log in post #9?

I am seeing the following:
25.09.2014 15:49:00|1|32|Order='82eaf2765c624aca8f8bd71930b94 ae3/U0000000' Name='Profit target' New state=PendingSubmit Instrument='NQ 12-14' Action=BuyToCover Limit price=4056,25 Stop price=0 Quantity=1 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''

This indicates the order was submitted to 4056,25.

May I have a reduced version of your script that demonstrates this issue so that I may test on my end and get a sense of what the strategy is actually doing?

[Upswing]

Hi,

yeah as I said I just recognized that there is a difference between the log and the things posted in the output window.

But how can the output produced by trace order be different?

The same thing just happened one hour ago on NQ again and almost at the same time a trade with same strategy on ES worked .

Sure I will send you a reduced version.

Appreciate the help,

Upswing

[xTrader1]

Another overfill isuue

Hi UpSwing(),

From the help guide on Risks of Electronic Trading with NinjaTrader.

"...It is possible (in rare occasions) that order(s) that are part of the OCO group will be filled before the cancellation request has been acknowledged. ..."

http://www.ninjatrader.com/support/h...trading_wi.htm

I'm just joing this thread, because it's a fresh one. I developed a strategy placing market orders on 1-tick data. I'm running the same strategy on several instruments, every instrument in a separate chart. I payed attention that an overfill occurs, when the instruments are placing orders simultaneously. I hear sometimes „order,order,order filled”, which I think means that the orders were placed so closely in time, that the full message is not heared. Sometimes it terminates succesfully, e.g. all the orders are filled, sometimes I hear a “thrash” sound, an overfill message box appears and the strategy for one or more instruments is disabled.

My scenario, according to your explanation of the overfill event, as appears above is, that because of sudden orders load, the market orders are queued for filling, but cannot be filled during a life time of current bar (which is 1-tick, may be much less than a second). Then, because the market order is out of scope, the order is canceled, but then its time to be filled arrives and the order, already canceled is filled.

This is my data, this is my strategy, this is my instrument period.
What could be done? It would be useless to postpone my order from the point of view of my strategy, even if it's possible programatically.
I'm working in managed mode, if there is no choice, I'll port my code to unmanaged mode, and set IgnoreOverFill = true; . In this case must I provide my overfill handler? Could it be just an empty handler? What will happen in this case, will my orders be filled?

Thanks.

[NinjaTrader_CalH]

xTrader,

You will want to take a look at the link below on how to handle the IgnoreOverfill with the Unmanaged approach. Read the last section -
http://www.ninjatrader.com/support/h...d_approach.htm

Additionally, we do not have any samples available for this kind of handling.