Order stuck in "CancelPending" state

[saltminer]

Hi,
in a strategy that uses a bracket of stop orders to enter, in NT8 in unmanaged mode, an order got stuck in "CancelPending". This was in Market Replay.

Below is an excerpt from the Output Window which shows a lot of the strategy's actions and also the trace orders information. The order "SE" successfully transitions to realtime as per the recommendations. Then it is cancelled but only enters "CancelPending" and gets stuck there.The strategy tries a new setup each bar, but cannot proceed because it is waiting for the order to reach a final state. (i.e. cancelled)

Thanks in advance,
saltminer.

UpdatingOrdersToRealtime()

transition orderId='NT-00141-134' account='Playback101' name='LE' orderState=Working instrument='GC 12-17' orderAction=Buy orderType='Limit' limitPrice=1339.2 stopPrice=1339.2 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2017-09-04 22:01:03' gtd='2099-12-01' statementDate='2017-10-21'

orderId='f025eed26331413a82818597bd45220b' account='Playback101' name='LE' orderState=Rejected instrument='GC 12-17' orderAction=Buy orderType='Limit' limitPrice=1339.2 stopPrice=1339.2 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5766 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

transition orderId='NT-00142-134' account='Playback101' name='SE' orderState=Working instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2017-09-04 22:01:03' gtd='2099-12-01' statementDate='2017-10-21'

orderId='d5ec6ff4c6334ef68ca7868fa861f344' account='Playback101' name='SE' orderState=CancelPending instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5767 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

transition orderId='NT-00128-134' account='Playback101' name='LT' orderState=Cancelled instrument='GC 12-17' orderAction=Sell orderType='Limit' limitPrice=1341.7 stopPrice=1341.7 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2017-09-04 03:50:49' gtd='2099-12-01' statementDate='2017-10-21'

Transition returned null

orderId='NT-00128-134' account='Playback101' name='LT' orderState=Cancelled instrument='GC 12-17' orderAction=Sell orderType='Limit' limitPrice=1341.7 stopPrice=1341.7 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2017-09-04 03:50:49' gtd='2099-12-01' statementDate='2017-10-21'

transition orderId='NT-00140-134' account='Playback101' name='ST' orderState=Filled instrument='GC 12-17' orderAction=BuyToCover orderType='Limit' limitPrice=1341.5 stopPrice=1341.5 quantity=1 tif=Gtc oco='' filled=1 averageFillPrice=1341.4 onBehalfOf='' id=-1 time='2017-09-04 21:18:18' gtd='2099-12-01' statementDate='2017-10-21'

Transition returned null

orderId='NT-00140-134' account='Playback101' name='ST' orderState=Filled instrument='GC 12-17' orderAction=BuyToCover orderType='Limit' limitPrice=1341.5 stopPrice=1341.5 quantity=1 tif=Gtc oco='' filled=1 averageFillPrice=1341.4 onBehalfOf='' id=-1 time='2017-09-04 21:18:18' gtd='2099-12-01' statementDate='2017-10-21'

mkVolumeProfileSystem 80-100 myHistorical achieves Realtime data on bar 222 of 223

9/5/2017 2:37:32 AM Strategy 'mkVolumeProfileSystem 80-100 /119804486': Cancelled custom managed order: orderId='d5ec6ff4c6334ef68ca7868fa861f344' account='Playback101' name='SE' orderState=CancelPending instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5767 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

9/5/2017 2:37:31 AM Inverted (strangle) trade, Long layer 2 @ 1336.6 Target=1337.0 Stoploss=1332.4, Short layer 1 @ 1342.5 Target=1342.1 Stoploss=1346.7

9/5/2017 2:37:31 AM arriving at GetFlat(), gettingFlat=False, doBarStartOrdersWhenFlat=True Position is Flat, AllOrdersInactive=False

9/5/2017 2:37:32 AM Strategy 'mkVolumeProfileSystem 80-100 /119804486': Cancelled custom managed order: orderId='d5ec6ff4c6334ef68ca7868fa861f344' account='Playback101' name='SE' orderState=CancelPending instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5767 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

9/5/2017 3:25:12 AM Strategy 'mkVolumeProfileSystem 80-100 /119804486': Cancelled custom managed order: orderId='d5ec6ff4c6334ef68ca7868fa861f344' account='Playback101' name='SE' orderState=CancelPending instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5767 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

9/5/2017 3:25:11 AM Inverted (strangle) trade, Long layer 1 @ 1334.8 Target=1335.3 Stoploss=1330.5, Short layer 1 @ 1338.6 Target=1338.1 Stoploss=1342.9

9/5/2017 3:25:11 AM arriving at GetFlat(), gettingFlat=True, doBarStartOrdersWhenFlat=True Position is Flat, AllOrdersInactive=False

9/5/2017 3:25:12 AM Strategy 'mkVolumeProfileSystem 80-100 /119804486': Cancelled custom managed order: orderId='d5ec6ff4c6334ef68ca7868fa861f344' account='Playback101' name='SE' orderState=CancelPending instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5767 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

9/5/2017 5:21:32 AM Strategy 'mkVolumeProfileSystem 80-100 /119804486': Cancelled custom managed order: orderId='d5ec6ff4c6334ef68ca7868fa861f344' account='Playback101' name='SE' orderState=CancelPending instrument='GC 12-17' orderAction=SellShort orderType='Limit' limitPrice=1343 stopPrice=1343 quantity=1 tif=Gtc oco='89371b3cb0e94fe5bb615919592add54' filled=0 averageFillPrice=0 onBehalfOf='' id=5767 time='2017-09-04 23:00:00' gtd='2099-12-01' statementDate='2017-09-04'

[NinjaTrader_Jim]

Hello saltminer,

Thanks for the report.

So we can review this matter further, could you give us answers to the following questions?

Is this constantly reproducible whenever you enable this strategy in Market Replay, or could we enable this strategy at a specific time period using Market Replay data to reproduce the CancelPending order? It would be important to verify that transitioning happens correctly sometimes or the order gets stuck under certain conditions.

Is this strategy complex? -Could it be reduced to a simple example that can reproduce the occurrence taking as few steps as possible? Once we have a test case that can show repeat the issue, we can submit something solid for the development team to review. Any help you could provide with such an example would help us to improve the software.

I look forward to being of further assistance.

[saltminer]

The incident happened on a customer's machine, not my own, so I don't have my own data to provide you. This customer will be in touch with you to get advice about how to remove this stuck order from his database.
This is just a single incident of the issue. I don't know if it is reproducible. (I guess we'll all find out together.)
The strategy is quite complex, and uses a proprietary indicator in a DLL, so it is not easily forwardable to your tech team.

Cheers,
saltminer

[fx.practic]

I have the same issue on my PC: historical replay and OrderState.CancelPending.
Any solutions?

[NinjaTrader_Kate]

Hello fx.practic,

Thank you for your post.

Would you be able to provide answers to the questions Jim posed to saltminer above?

So we can review this matter further, could you give us answers to the following questions?

Is this constantly reproducible whenever you enable this strategy in Market Replay, or could we enable this strategy at a specific time period using Market Replay data to reproduce the CancelPending order? It would be important to verify that transitioning happens correctly sometimes or the order gets stuck under certain conditions.

Is this strategy complex? -Could it be reduced to a simple example that can reproduce the occurrence taking as few steps as possible? Once we have a test case that can show repeat the issue, we can submit something solid for the development team to review. Any help you could provide with such an example would help us to improve the software.

Thanks in advance; I look forward to assisting you further.

[NinjaTrader_Kate]

Hello fx.practic,

I'm following up as I haven't heard back from you on your issues with historical replay and OrderState.CancelPending. Are you still experiencing issues, or have you resolved this?

Thanks in advance; I look forward to assisting you further.

[fx.practic]

Hello, Kate.
Sorry for the late answer.
Yes, I experienced it on regular basis and had to create each-tick-maintenance for all orders like this:

if( Order.IsInTerminalState( myOrder ) myOrder = null; 

This issue and other similar OrderState-related issues happened at switching State from Historical to RealTime if live order exists.
Enabling strategy on Playback with market moving significantly increase probability of issues.

It can be very helpful to have access to all alive orders, generated by current strategy instance (to save milliseconds on Orders collection filtering).
Also, I'd like to have ability to add own sting notes to Order object (just something like UserNotes field with ability to get and set).

Let me request these two features.

Thank you.

[NinjaTrader_Kate]

Hello fx.practic,

Thank you for your reply.

To clarify for our developers, when you say you would like access to all live orders generated by the strategy instance, how would you like to see that structured and be able to access orders?

Thanks in advance; I look forward to assisting you further.

[fx.practic]

Thank you for the question.
If you are ask me - any enumerable collection will be great.
Right now orders are stored in collection:

System.Collections.ObjectModel.Collection<NinjaTrader.Cbi.Order>

I'd wish to have 2 separate (Historical and Realtime orders) collections of orders in non-terminal state.

[fx.practic]

And I have another feature request: add new OrderState.Ignored and to trigger OnOrderUpdate() method when order ignored.
Thank you.

[NinjaTrader_Kate]

Hello fx.practic,

Thank you for your suggestion.

I have submitted feature requests for all the above requests for you. These are being tracked as SFT-4365 for the Ignored orders items, SFT-4366 for adding notes to an order object, and SFT-4367 for the separate collections of orders.

As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted in the Release Notes page of the Help Guide.

Release Notes — https://ninjatrader.com/support/help...ease_notes.htm

Please let us know if we may be of further assistance to you.

[fx.practic]

Thanks a lot.
NT support is great as usual!