OnOrderUpdate using Interactive Brokers API

[NinjaTrader_Jim]

Hello Kobi,

Are you stating that when you submit an order and disconnect, you are not receiving OrderUpdates after the connection is re-established?

Account positions (PositionAccount, Account.Positions) are updated on PositionUpdate events (updated as early as OnPositionUpdate.) The aim of the UnmanagedOnOrderUpdate strategy is to break away from order/execution/position ordering and to provide a way to generate a strategy position (not an account position) based on OrderUpdates instead of ExecutionUpdates. The strategy position can only account for trades made from that strategy alone. If you need full account level access, you would be back to relying on Order/Execution/Position ordering in your strategy, because you would need Position updates to drive your strategy logic and account position.

PositionAccount - https://ninjatrader.com/support/help...ionaccount.htm

Account.Positions - https://ninjatrader.com/support/help...ns_account.htm

With Interactive Brokers, I had previously recommended to build a strategy that follows 1. OnOrderUpdate 2. OnPositionUpdate to handle things that would most often be done following a SampleOnOrderUpdate style 1. OrderUpdate 2. ExecutionUpdate approach. Since account positions update as early as OnPositionUpdate, this approach may be worth a check if you can use it for your purposes. My old suggestion is attached in this post.

When this was suggested, Interactive Brokers was reliably giving out PositionUpdates after OrderUpdates (ExecutionUpdates were coming out of our normally expected order.) We are staying away from advising this incase Order/Execution/Position ordering changes again. This has been the case with Rithmic (we no longer expect an order with Rithmic,) and we also had to update the way the NinjaTrader Control Center handles PositionUpdates from IB when it updates positions on the Strategies tab of the Control Center.

You may see if following 1. Order 2. Position updates works for your purposes with Interactive Brokers. I would not try that ordering with Rithmic, however.

We look forward to assisting.

[levikNT]

Hello Jim,

1.

Are you stating that when you submit an order and disconnect, you are not receiving OrderUpdates after the connection is re-established?

What I meant is in case the order has been filled while NT was disconnected ,IB will send OnOrderUpdate as a response of the filled order.
But because NT is disconnected than it would not receive this event.
When NT connection with IB is re-established then it would be late , because the event has been all ready sent while NT was disconnected.

The same can be accrued with other reports or event from IB.

So in order to handle with connection lost situation I need a full control and access to my account.

I do not want to relay only on IB reports when there are changes in my IB account\position status,
I would like the ability to initiate queries regarding to my account. it's in addition to events that come from IB in case of account updated.
I overviewed IB API and they support these queries.

2. Does the unmanage approach orders work with OnPosition\OnExecution ?(generic question)


Sincerely, Kobi

[NinjaTrader_Jim]

Hello Kobi,

I just wanted to say I have not forgotten about you. I am trying to get a demonstration video together for #1, but due to timing and being able to sever my internet connection at the right moment, I am having difficulty.

I'll come back to #1 once I have been able to make a successful demonstration, but I can say that the communication between IB and NinjaTrader would be dictated in what the NinjaTrader Connection Adapter offers. We can read information from the account with Account.Get, and you can also subscribe to account level order/execution/position updates. These same event updates are what feed a NinjaScript strategy's OnOrderUpdate, OnExecutionUpdate and OnPositionUpdate methods, although the strategy filters only what is relevant for the strategy.

There is no other means to utilize API features directly as these are all handled in the Connection Adapter which is outside of the scope of a NinjaScript strategy. Connection Adapters are also closed source/proprietary. They cannot be modified by the end user.

Regarding #1, I do need to clarify here. NinjaTrader will try to recover orders upon reconnect and if the order can be mapped to a strategy order submission, the strategy will see it on reconnect. IB does not offer an avenue for us to do this through their API so we would not be have the orders seen by the strategy after connection is re-established.

Regarding #2, Unmanaged and Managed both work with these methods. The only thing we really need to understand is that Managed has internal rules and is tied to the internal Strategy Position. Unmanaged does not have internal rules and is not tied to the internal Strategy Position.

[sidlercom80]

Hello Kobi,

Regarding #2, Unmanaged and Managed both work with these methods. The only thing we really need to understand is that Managed has internal rules and is tied to the internal Strategy Position. Unmanaged does not have internal rules and is not tied to the internal Strategy Position.

Hi _Jim, "Unmanaged does not have internal rules and is not tied to the internal Strategy Position." if i understand this correctly, the synchronization in unmanaged mode has no relevance for the strategy and there is no synchronization, right? the strategy can also send orders without being synchronized (position vs. account position). then the strategy as shown on the picture (not synchronous) can still send orders, right? the orders should all be maintained via the OrderUpdates (), because I use Rhitmic.

[NinjaTrader_Jim]

Hello slidercom80,

Correct, you do not have to worry about the strategy position stating it is out of sync because the strategy is not following the internal strategy position. It would be following the Position object we created within the strategy, that is updated with OrderUpdate events.

[FutureDragon]

Jim,

thanks for posting the UnmanagedOnOrderUpdate sample and I am updating my strategy based on it. I have a couple of questions:

1) I have an account with NinjaTrader (NinjaTrader Continuum) and multiple accounts with prop firms which all use Rithmic connections. Can I use the same code base for all data providers? Is it worthwhile to maintain two code bases?

2) What's the perf impacts of UnmanagedOnOrderUpdate? I noticed it adds a tick bar, and would like to understand the perf implication to strategy optimization and back testing.

3) In the sample code, calculate is set to Calculate.OnBarClose. Would set it to OnPriceChange make it more responsive to price changes, or OnBarClose is actually better (or required) since the tick bar will make the OnBarUpdate be called for each tick?

[NinjaTrader_Emily]

Hello FutureDragon,

Thank you for your note.

If you have developed scripts that are meant to work with the Rithmic connections, there is no need to create a second script that achieves the same outcome for a NinjaTrader Continuum account. You can use the same code for all data providers as long as it does not rely on the order of Position/Execution/Order updates due to the way the Rithmic (and Interactive Brokers) are not guaranteed to send those updates in any particular order.

Adding a 1-tick data series can actually be very useful when backtesting and using the Strategy Analyzer because it offers a way to get more granular order fills compared to using a larger timeframe for order fills. We even have a reference sample that goes over adding a 1-tick series for more granular fills here:

https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

Adding a 1-tick series does mean that NinjaTrader has to process additional historical data and it can result in performance impacts such as taking a longer time to complete a backtest. This would also occur if you did not add the series in the script and instead chose to use the High Order Fill Resolution in the backtest settings:

https://ninjatrader.com/support/helpGuides/nt8/index.html?understanding_historical_fill_.htm#UnderstandingOrderFillResolution

There are other items that can affect optimization performance, such as the Best # of results kept, IncludeTradeHistoryInBacktest, IsInstantiatedOnEachOptimizationIteration, the number of iterations or range of optimization, and whether the data is already cached from your provider or not.

Setting Calculate to OnBarClose would call OnBarUpdate() for each tick, as the 1-tick bars would close after every 1 tick. That being said, we have some good tips for the different calculate methods in the help guide here:

https://ninjatrader.com/support/helpGuides/nt8/calculate.htm

As well as the reference sample that is mentioned which demonstrates how to separate logic between bar close and each tick here:

https://ninjatrader.com/support/forum/forum/ninjascript-educational-resources/reference-samples/19874-strategy-separating-logic-to-either-calculate-once-on-bar-close-or-on-every-tick?t=19387

Changing the Calculate to OnPriceChange instead of OnBarClose wouldn't make a different unless you also have logic based on the primary bars series that the strategy is added to rather than just calculating logic based on the added 1-tick data series.

I hope this information helps to clarify the answers to your questions. Please let me know if I may be of further assistance.

[FutureDragon]

Emily,

Thanks for the detailed information. The 1-tick series makes my strategy optimization many time slower, minutes work becomes days. So I am settling with 1 second series for now.

[thetradeengine]

Hello Kobi,

Thanks for your question.

You could consider checking the order name from Execution.Order.Name.

When designing a robust strategy that will work with Rithmic and Interactive Brokers, you can consider designing a strategy that uses OnOrderUpdate only, and this will work around the obscure event ordering for Rithmic and IB.

Please note that strategy positions (which the Managed Approach uses internally) are updated on Execution events (OnExecutionUpdate.) If you would like to track your own strategy position that is updated on OrderUpdates, this would be possible, but requires we use the Unmanaged Approach.

I have attached an example Unmanage strategy that uses OnOrderUpdate only, and creates its own Position object to track the strategy position. The strategy is built to be similar to our SampleOnOrderUpdate strategy.

Ref.

Execution - https://ninjatrader.com/support/help.../execution.htm

SampleOnOrderUpdate - https://ninjatrader.com/support/help...and_onexec.htm

Unmanaged Approach - https://ninjatrader.com/support/help...d_approach.htm

We look forward to assisting.

Hi Jim,

After looking at different posts regarding work arounds for Rithmic, your sample code here worked for me, but I have been running into an issue. I have been seeing about a 50% success rate of trades being taken in realtime when a trade is triggered. No errors appear in the logs, and I've tried logging out many things from within the strategy. The strategy logs that it does not think it is in a position, and that all conditions for a trade are triggered, but no trade is taken, and if I refresh Ninjascript on the chart afterwards the positions of the trades appear historically on the chart. I wanted to share my code to see if there is something obvious that I'm missing.

Thank you

[NinjaTrader_Emily]

Hi Jim,

After looking at different posts regarding work arounds for Rithmic, your sample code here worked for me, but I have been running into an issue. I have been seeing about a 50% success rate of trades being taken in realtime when a trade is triggered. No errors appear in the logs, and I've tried logging out many things from within the strategy. The strategy logs that it does not think it is in a position, and that all conditions for a trade are triggered, but no trade is taken, and if I refresh Ninjascript on the chart afterwards the positions of the trades appear historically on the chart. I wanted to share my code to see if there is something obvious that I'm missing.

Thank you

Hello thetradeengine,

Although your message was directed to Jim, I would be glad to assist.

It sounds like your strategy position and account position may not be in sync; when your strategy is not taking trades at times when you expect it to, please check the color of the strategy name on the Strategies tab of the Control Center. If it is orange, this means that the strategy is waiting to become flat (based on its historical executions) before starting because of the start behavior selected for the strategy.

For more information about the Strategy position vs. the Account position as well as syncing account positions, please see the following links:

• https://ninjatrader.com/support/help..._account_p.htm
• https://ninjatrader.com/support/help..._positions.htm

If you are experiencing this behavior where the strategy is not placing trades and the name is green in the Strategies tab of the Control Center, you may need to add debugging prints to better understand your strategy's behavior:

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working#post791121

Please let us know if we may be of further assistance.

[thetradeengine]

Hello thetradeengine,

Although your message was directed to Jim, I would be glad to assist.

It sounds like your strategy position and account position may not be in sync; when your strategy is not taking trades at times when you expect it to, please check the color of the strategy name on the Strategies tab of the Control Center. If it is orange, this means that the strategy is waiting to become flat (based on its historical executions) before starting because of the start behavior selected for the strategy.

For more information about the Strategy position vs. the Account position as well as syncing account positions, please see the following links:

• https://ninjatrader.com/support/help..._account_p.htm
• https://ninjatrader.com/support/help..._positions.htm

If you are experiencing this behavior where the strategy is not placing trades and the name is green in the Strategies tab of the Control Center, you may need to add debugging prints to better understand your strategy's behavior:
https://ninjatrader.com/support/foru...ing#post791121

Please let us know if we may be of further assistance.

Hello Emily,

The strategy always becomes green after it is enabled.

I have tried different routes to debugging the strategy. I add this code to change the background color according to the internally calculated position, and the background correctly shows that the position is flat:

if (_CalculatedPosition.MarketPosition == MarketPosition.Long)
    BackBrush = Brushes.DarkGreen;
else if (_CalculatedPosition.MarketPosition == MarketPosition.Short)
    BackBrush = Brushes.DarkRed;

I have also tried these prints in the order logic, and the output shows that the entry conditions are met, the daily PnL filters return true, the calculated market position is flat, and the running quantity is 0:

if (longEntry == null && longEntryConditions)
{
    longEntryToken = true;
    Print(Time[0] + "Long entry == null and Long entry conditions");
    Print(Time[0] + "Daily PnL: " + dailyPNL);
    Print(Time[0] + "Loss Limit Filter: " + lossLimitFilter);
    Print(Time[0] + "Daily Profit Target Filter: " + dailyProfitTargetFilter);
    Print(Time[0] + "Calculated Market position while in long entry logic is flat: " + calculatedFlat);
    Print(Time[0] + _runningQuantity.ToString());
}​;

[NinjaTrader_Emily]

Hello thetradeengine,

Thank you for that information.

I would be glad to take a look at your diagnostic files to further investigate. Please follow the steps below to manually attach your log and trace files to a support email for me to review:

• Open the Windows File Explorer (hotkey = Windows key + E) Then navigate to, "Documents" > 'NinjaTrader 8' folder
• Right-click on the 'log' and 'trace' folders and select Send To> Compressed (zipped) Folder.
• Send the 2 compressed folders as attachments to an email to support[AT]ninjatrader[DOT]com
  • Please include "ATTN Emily C" in the subject line
• Once complete, you can delete these compressed folders.

Thank you for your patience. I look forward to hearing from you.

[ChrisR]

Hello Kobi,

Thanks for your question.

You could consider checking the order name from Execution.Order.Name.

When designing a robust strategy that will work with Rithmic and Interactive Brokers, you can consider designing a strategy that uses OnOrderUpdate only, and this will work around the obscure event ordering for Rithmic and IB.

Please note that strategy positions (which the Managed Approach uses internally) are updated on Execution events (OnExecutionUpdate.) If you would like to track your own strategy position that is updated on OrderUpdates, this would be possible, but requires we use the Unmanaged Approach.

I have attached an example Unmanage strategy that uses OnOrderUpdate only, and creates its own Position object to track the strategy position. The strategy is built to be similar to our SampleOnOrderUpdate strategy.

Ref.

Execution - https://ninjatrader.com/support/help.../execution.htm

SampleOnOrderUpdate - https://ninjatrader.com/support/help...and_onexec.htm

Unmanaged Approach - https://ninjatrader.com/support/help...d_approach.htm

We look forward to assisting.

I have imported and loaded the UnmanagedTemplate and is does not place any orders. Not getting anything usefull from the output, just the initial enabled string. What am I missing here? I am using Rithmic data and trying to find a solution since OnExecutionUpdate is not an option.

[NinjaTrader_Emily]

Hello ChrisR,

Thank you for that info.

Which template are you using? You mentioned UnmanagedTemplate - do you have the link to the post where you downloaded it from? Jim has also posted a .zip containing Rithmic and IB friendly examples for both the managed and the unmanaged approach here:

• https://ninjatrader.com/support/foru...72#post1206372

Additionally, after the strategy is enabled what color is the strategy name on the Strategies tab of the Control Center? If the name is orange, that means it will not submit any live orders until the strategy position is flat so it may sync up with your account. The name will change to green once the strategy is synced and ready to submit live orders. For more information about the strategy position vs. the account position and syncing positions, please see the following pages:

• https://ninjatrader.com/support/help..._account_p.htm
• https://ninjatrader.com/support/help..._positions.htm

Please let me know if I may be of further assistance.

[ChrisR]

Emily,

Thanks for the reply. I was able to figure it out, but also decided to go the managed route instead as my strategy doesnt really require that level of manipulation.