GetRealtimeOrder Returns Null Order Reference

[ChazJ]

Hi,

I am using an unmanaged strategy. I am now ready to test live (but in a demo account). I have read about how to handle the transition from historical to live when the strategy is enabled on real-time data.

Here is my problem: when I call GetRealtimeOrder with a historical order reference, it returns a null value:

if (myorder != null)
myorder = GetRealtimeOrder(myorder);
// myorder is now NULL!!

No exception is thrown at the time I call GetRealtimeOrder. Then in subsequent code execution I am getting a null reference exception in referencing myorder.

Why is this happening? Under what circumstances does this function return a null reference? What can I do about this?

Thank you,

Charlie

[NinjaTrader_ChelseaB]

Hello Charlie,

Is the order still in a working or accepted state on the last historical bar?

Is the StartBehavior .AdoptAccountPosition or .ImmediatelySubmit?

[ChazJ]

Hi Chelsea,

The order is actually a stop order to close a position (Long, Qty=1) when the transition happens. I have the StartBehavior set to 'WaitUntilFlat'.

I have all the documentation (many times over), and maybe I just do not understand how this is supposed to work. Before the transition there may have been multiple orders to open and close positions. If I were connected to a real, live trading account, how on earth would that work? Does NT not actually submit these historical orders to the exchange? I would hope so!

I have tried canceling the outstanding historical order, and it will eventually execute, but then leave me in a short position on the real-time live account as the documentation warns.

What am I missing here?

Thanks,

Charlie

[NinjaTrader_ChelseaB]

Hello Charlie,

With WaitUntilFlat, there won't be any real-time orders allowed until the position is flat.
So likely, there won't be any orders to transition to real-time because that position will be closed before any real-time orders are allowed.
If the position is closed, I would not expect there to be orders working to close that position.

Historical orders are not submitted live to brokerage. GetRealtimeOrder() would cause a working historical order to be submitted to the brokerage.

If you are in a short position on the real-time account, was that from an order submitted by the strategy that filled?

[ChazJ]

Hi Chelsea,

Yes, the short position was from an order that filled to the exchange (demo account) when I did not have a long position established with the exchange. The long position was in the historical context.

I tried changing the StartBehavior to 'ImmediatelySubmit, Synchronize Account' and I got a more reasonable result. The historical position was opened on the exchange and then GetRealTimeOrder() worked correctly and submitted the stop order. So it left me in the same exact state I was in before the transition, at least as viewed on the chart.

So that is progress! However, what I would really like is to start the live session completely flat, so when I enable the strategy after the RTH session has already started, I can start off in a flat position. How can I achieve that?

[NinjaTrader_ChelseaB]

Hello Charlie,

If the position is closed and there is a working order, would this be an entry order for a new position?

Was this order placed and working while the previous position was open?
(Meaning the order was in the same direction as the open position so as not to violate the internal order handling rules?
https://ninjatrader.com/support/help...antedPositions)

Are you using prints and TraceOrders to see that the strategy position was flat and the order that filled was one that was placed in historical data, transitioned to real-time, and then filled in real-time to take the strategy into a position?
May I see the output from those prints to confirm?

[ChazJ]

I am using prints to trace what is happening. Unfortunately, I cannot reporduce the exact conditions again, as the market conditions have changed when it transitions now.

With the StartBehavior set to 'ImmediatelySubmit, Synchronize Account', it is leaving me in the exact same state after the transition as before the transition. The position I had and the stop loss order I had before the transiton were both carried over to the real-time environment. I guess this OK, but not what I am looking for.

I want to start to start in a flat position when the strategy is enabled. But the StartBehavior setting of 'WaitUntilFlat' seem to do nothing, I guess, until the outstanding stop order is filled. Frankly, I did not have the patience to wait for it to be filled. I guess I could try waiting and maybe then I would be in a flat state like I am looking for.

Is there any way to detect just before the transition occurs, so I can flatten my position on the historical data manually within my code?

[NinjaTrader_ChelseaB]

Hello ChazJ,

If the instrument is a future or forex instrument, try and download market replay data and playback that time.

https://ninjatrader.com/support/helpGuides/nt8/playback_connection.htm

I'm curious if you have a working order.
The output from the output window could let us know when the state is changing, when the order is submitted, and when the order changes states from submitted, to working, to filled or cancelled or lost due to not being transitioned.

Typically, if the strategy is wait until flat, there won't be any working orders to transition. Those would come after the first real-time order..

https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/104242-how-to-reset-sim-on-ninjatrader-8-can-t-close-open-position?p=811541#post811541

If the start behavior is wait until flat, the position will be flat when the strategy starts real-time trading.
You could also choose to not allow the strategy to process historically.
Or you can exit any open position on the last historical bar if you want.

[ChazJ]

Chelsea,

I did as you suggested and downloaded todays RTH data for MES micro future which is what I am working with. I loaded the data to the point where the incident occurred this morning. My StartStrategy set to 'WaitUntilFlat'

Here is my Print output. I output the time and Bar number on each Print:

10/22/2019 3:23:37 PM, Bar# 2349: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2349: Last Bar Close = 3013.25
10/22/2019 3:23:37 PM, Bar# 2350: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2350: On Bar Update. Current STC Stop at 3004.75
10/22/2019 3:23:37 PM, Bar# 2350: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2350: Last Bar Close = 3013.5
10/22/2019 3:23:37 PM, Bar# 2350: Changing STC Stop Order Price to 3005
10/22/2019 3:23:37 PM, Bar# 2350: On Order Update. Order = STC
10/22/2019 3:23:37 PM, Bar# 2350: STC Order Update. limitprice = 0, StopPrice = 3005, filled = 0, orderState = ChangePending
10/22/2019 3:23:37 PM, Bar# 2350: On Order Update. Order = STC
10/22/2019 3:23:37 PM, Bar# 2350: STC Order Update. limitprice = 0, StopPrice = 3005, filled = 0, orderState = ChangeSubmitted
10/22/2019 3:23:37 PM, Bar# 2350: On Order Update. Order = STC
10/22/2019 3:23:37 PM, Bar# 2350: STC Order Update. limitprice = 0, StopPrice = 3005, filled = 0, orderState = Accepted
10/22/2019 3:23:37 PM, Bar# 2350: On Order Update. Order = STC
10/22/2019 3:23:37 PM, Bar# 2350: STC Order Update. limitprice = 0, StopPrice = 3005, filled = 0, orderState = Working
10/22/2019 3:23:37 PM, Bar# 2351: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2351: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2351: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2351: Last Bar Close = 3013
10/22/2019 3:23:37 PM, Bar# 2352: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2420: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2420: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2420: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2420: Last Bar Close = 3010.75
10/22/2019 3:23:37 PM, Bar# 2421: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2421: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2421: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2421: Last Bar Close = 3011
10/22/2019 3:23:37 PM, Bar# 2422: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2422: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2422: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2422: Last Bar Close = 3011.75
10/22/2019 3:23:37 PM, Bar# 2423: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2423: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2423: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2423: Last Bar Close = 3011.5
10/22/2019 3:23:37 PM, Bar# 2424: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2424: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2424: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2424: Last Bar Close = 3011.75
10/22/2019 3:23:37 PM, Bar# 2425: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2425: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2425: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2425: Last Bar Close = 3011.5
10/22/2019 3:23:37 PM, Bar# 2426: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2426: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2426: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2426: Last Bar Close = 3011
10/22/2019 3:23:37 PM, Bar# 2427: On Bar Update. Position = Long
10/22/2019 3:23:37 PM, Bar# 2427: On Bar Update. Current STC Stop at 3005
10/22/2019 3:23:37 PM, Bar# 2427: Current Position Quantity: 1
10/22/2019 3:23:37 PM, Bar# 2427: Last Bar Close = 3011.25
Enabling NinjaScript strategy 'MyStrategy/173662198' : On starting a real-time strategy - StartBehavior=WaitUntilFlat Position=MES 12-19 1L EntryHandling=All entries EntriesPerDirection=2 StopTargetHandling=Per entry execution ErrorHandling=Stop strategy, cancel orders, close positions ExitOnSessionClose=True / triggering 30 seconds before close SetOrderQuantityBy=Strategy ConnectionLossHandling=Recalculate DisconnectDelaySeconds=10 CancelEntriesOnStrategyDisable=True CancelExitsOnStrategyDisable=True Calculate=On bar close IsUnmanaged=True MaxRestarts=4 in 5 minutes
10/22/2019 3:23:37 PM, Bar# 2427: State transitioned to REAL-TIME!
10/22/2019 3:23:37 PM, Bar# 2427: STC Order Active. Getting RealTime Order
10/22/2019 3:23:37 PM, Bar# 2427: RealTime STC Order returned NULL!!
10/22/2019 3:23:37 PM, Bar# 2427: Connection Status Has Changed.
10/22/2019 3:23:37 PM, Bar# 2427: Previous Status: Disconnected
10/22/2019 3:23:37 PM, Bar# 2427: Current Status: Connecting
10/22/2019 3:23:37 PM, Bar# 2427: Previous Price Status: Disconnected
10/22/2019 3:23:37 PM, Bar# 2427: Current Price Status: Connecting
10/22/2019 3:23:37 PM, Bar# 2427: Connection Status Has Changed.
10/22/2019 3:23:37 PM, Bar# 2427: Previous Status: Connecting
10/22/2019 3:23:37 PM, Bar# 2427: Current Status: Connected
10/22/2019 3:23:37 PM, Bar# 2427: Previous Price Status: Connecting
10/22/2019 3:23:37 PM, Bar# 2427: Current Price Status: Connected
10/22/2019 3:24:28 PM, Bar# 2428: On Bar Update. Position = Long
10/22/2019 3:24:28 PM, Bar# 2428: Current Position Quantity: 1
10/22/2019 3:24:28 PM, Bar# 2428: Last Bar Close = 3011
10/22/2019 3:24:28 PM, Bar# 2428: On Bar Update, STCOrder == null - SHOULD NEVER REACH HERE!!!
10/22/2019 3:24:28 PM, Bar# 2428: Submitting STC Stop Order at 2996.5
10/22/2019 3:24:28 PM, Bar# 2428: On Order Update. Order = STC
10/22/2019 3:24:28 PM, Bar# 2428: STC Order Update. limitprice = 0, StopPrice = 2996.5, filled = 0, orderState = Submitted
10/22/2019 3:24:28 PM, Bar# 2428: On Order Update. Order = STC
10/22/2019 3:24:28 PM, Bar# 2428: STC Order Update. limitprice = 0, StopPrice = 2996.5, filled = 0, orderState = Accepted
10/22/2019 3:24:28 PM, Bar# 2428: On Order Update. Order = STC
10/22/2019 3:24:28 PM, Bar# 2428: STC Order Update. limitprice = 0, StopPrice = 2996.5, filled = 0, orderState = Working


STCOrder is a reference to a "Sell to Close" stop loss order that is in a Working state when the transition occurs. Right afterwards, I attempt to convert the order with GetRealTimeOrder and it returns a NULL. Later, I get an OnBarUpdate callback and you can see that my reference to that order is null and I Print an error message. Below is the section of code that outputs this message:

if (Position.MarketPosition == MarketPosition.Long)
{
if (STCOrder != null)
{
(Code here to re-set my trailing stop price)
}
}
else
{
string msg = "On Bar Update, STCOrder == null - SHOULD NEVER REACH HERE!!!";
DebugPrint(msg);
Log(msg, LogLevel.Alert);
mSTCTargetPrice = mEntryPrice - StopLoss;
DebugPrint("Submitting STC Stop Order at " + mSTCTargetPrice.ToString() );
STCOrder = SubmitOrderUnmanaged(0, OrderAction.Sell , OrderType.StopMarket , Position.Quantity , 0, mSTCTargetPrice,"","STC");
}

Hope this helps.

Charlie

[NinjaTrader_ChelseaB]

Hello Charlie,

To save the output from the output window, right-click the output window then select Save As.

To print the order object from OnOrderUpdate():

protected override void OnOrderUpdate(Order order, double limitPrice, double stopPrice, int quantity, int filled, double averageFillPrice, OrderState orderState, DateTime time, ErrorCode error, string comment)
{
    Print(order.ToString());
}

To print the State:

protected override void OnStateChange()
{
    Print("State: " + State.ToString());
}

To print the position:

protected override void OnPositionUpdate(Position position, double averagePrice, int quantity, MarketPosition marketPosition)
{
    Print("MarketPosition: " + marketPosition);
}

So looking at the output you have, is the issue that the strategy is not waiting to be flat? I'm not seeing that the position is flat at any point.

Without trace orders its hard to tell when orders are being submitted.

But it looks like the issue is possibly an order which may have a signal name of STC did not fill and close the historical position before the strategy began placing real-time trades, is this correct?

[ChazJ]

No, I think we are straying into the weeds here. The issue is very simple. Prior to transitioning, I have working order (STC) that I try to convert to real-time using GetRealTimeOrder, and that function is returning NULL. So my reference to the STC order is now null which causes a null reference exception later on.

[NinjaTrader_ChelseaB]

Hello ChazJ,

I'm trying to understand why an order would be transitioned when using wait until flat as the start behavior and there is a historical position that has to be flattened first.

I am seeing from the output the position is not becoming flat. Maybe this is why you think the order should still be working?

If you would like I can make a short example that places a historical order that does not fill in historical data so there is no historical position, and then gets transitioned to real-time and fills on the second real-time bar.

Let me know if you would like this.

[ChazJ]

Yes, that might be helpful, but what I am trying to achieve is to cancel this historical order when it transitions, not fill it. But I get the exception when I try place the cancel order, because GetRealTImeOrder returned a null reference. I want to start flat after the transition.

[NinjaTrader_ChelseaB]

Hello ChazJ,

A historical order that is not transitioned does not need to be cancelled. If it is not transitioned it will disappear completely. You don't have to do anything.

Below is a link to an example that does not transition historical orders when using adopt account position.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/1068697-why-quantity-can-not-be-set-for-exitlongstopmarket-more-than-stratetgy-position?p=1068829#post1068829

I also not certain the order is actually a working order when you are using GetRealtimeOrder() on it.

Attached is an example of an order submitted historically that remains working until real-time and that is transitioned from historical to realt-time with GetRealtimeOrder().

[ChazJ]

Yes! I see in the example that you are using GetRealTimeOrder in the Transition state. I was not even aware of such a state. I was trying to do it after the transition in the real-time state. Now it is working as expected.

Thanks so much for your help (and patience)

Charlie