Manage orders and strategy after disconnections

[algoroman]

Hello i have develop a strategy that is doing a qind of swing trading with an average of 10 positions par year and an average time in the market of 3 days and it's the important thing during these 3 days it have to stay connected or to handle the orders correctly to be sure everything works fine when there is disconnections.


My algo is made to work on CFDs with the FXCM broker.
I have read the NT / Fxcm disclaimer about the states of the orders when there is disconnections.
I have salso see in the logs that there is some disconnections on my server and that it's written :Strategy 'myStrategy' lost price connection but will keeep running and the same with lost order connection.


As i want my strategy to be fully automated i have few questions about how to manage these cases.



In my strategy, the orders are submited with a tag to identify them like that :

EnterLongLimit(2, true, positionSize, C_target_price, "BoX_ORDER");
SetStopLoss("BoX_ORDER", CalculationMode.Pips, C_SL_Size , false);
SetProfitTarget("BoX_ORDER", CalculationMode.Pips, (C_SL_Size * riskReward));


Then In the function OnOrderUpdate() i retrieve the order with it's name and it's state (submited or working and filled) and assign it to a variable.
Then i use this variable (the order) in the function OnBarUpdate() to manage the active order.

For exemple I use myOrder.Time.AddHours(96) to set an expiration and to cancel the order with CancelOrder(myOrder) after 4 days if it have not been filled.
And also use in a variable an integer with the trade bar index when the order is filled to check conditions with the active order from this bar index.

When certain conditions are meeted i will use SL order modification like this :
SetStopLoss("BoX_ORDER", CalculationMode.Price, entryOrder.AverageFillPrice, false);



Everything works fine on backtests as there is no disconnections, but i want to prevent the problems that can happends in real time.
For that i would like to ask the community :


In the case of a disconnection and a re connection, will the algo will be able to identify the order by it's name (the tag) to manage it correctly ?


It seems that in some cases the strategy is restarted automaticly after long disconnections, in this case will the algo will be able to identify the orders to manage them and did the variable that was stored with the integer of the trade bar index at the moment that the order is filled will logically be lost (as restart meens variable get re intitialized)?


Also, is there is a way to write in the code instructions to disconnect then reconnect if the connection is no more active since a certain time, because it happends a few times that the connection status stay on red for long time ?


Also, is it a good idea to store the tradeId given by the broker to identify the active orders after disconnections, or to store this id in a flie with C# to retreive the orders after disconnections ?



Thanks a lot for the help

Roman

[NinjaTrader_Jim]

Hello Roman,

Thanks for your post.

In the case of a disconnection and a re connection, will the algo will be able to identify the order by it's name (the tag) to manage it correctly ?

To handle brief connection losses where you do not have to restart the strategy, you can set Connection Loss Handling to Keep Running (Set in the Control Center's Tools > Options > Strategies menu)

To handle a situation where the strategy needs to be restarted, you could consider using the ImmediatelySubmit start behavior. When this start behavior is used, the strategy processes historical data to see what the Strategy Position will be as well as any orders that should be active from the strategy. The orders that should be active are compared to any live working orders. IF the orders match, the strategy will take control of the live working orders. If the live working orders do not match what is calculated from historical data, the live orders will be cancelled and replaced with those calculated from historical data.

Having a strategy that produces backtest results closer to realtime results can be helpful to make this transition smoother. This would especially be the case if you submit target and stop based on the fill prices of orders. Writing a strategy that functions OnBarClose and submits orders to a single tick data series can help to make the backtest results closer to live.

Discrepancies between realtime and backtest - https://ninjatrader.com/support/help...ime_vs_bac.htm

Backtesting with intrabar granularity (submitting orders to a single tick data series) - https://ninjatrader.com/support/help...ipt_strate.htm

Syncing account positions (start behaviors) - https://ninjatrader.com/support/help..._positions.htm

It seems that in some cases the strategy is restarted automaticly after long disconnections, in this case will the algo will be able to identify the orders to manage them and did the variable that was stored with the integer of the trade bar index at the moment that the order is filled will logically be lost (as restart meens variable get re intitialized)?

If the strategy is restarted, variables will be reinitialized, and the strategy will process the data in the data series as if it were historical data (like a backtest.) Your variables would then be expected to be what they would come up with as if you backtested that data.

If the strategy is left running with KeepRunning, and returns functioning after there is a reconnection, the strategy would not be restarted, and the variables would remain as they were before the disconnection.

Also, is there is a way to write in the code instructions to disconnect then reconnect if the connection is no more active since a certain time, because it happends a few times that the connection status stay on red for long time ?

There is some functionality in AddOns that could be used to make a connection through NinjaScript, but for a case of a strategy, this would not help to re-enable a strategy automatically, because there is not any means to start strategies programmatically.

Also, is it a good idea to store the tradeId given by the broker to identify the active orders after disconnections, or to store this id in a flie with C# to retreive the orders after disconnections ?

This really would not be necessary, NinjaTrader logs executions it sees and fetches executions of the day after connecting/reconnecting. Executions are stored in the NinjaTrader database and are paired together to form trades (not the same as the broker's account statement, but serves well to review trade performance.) Past orders/trades and executions can be reviewed from a Trade Performance window.

Using Trade Performance - https://ninjatrader.com/support/help...erformance.htm

Let us know if you have any additional questions.

[algoroman]

Hello Jim


Thanks a lot for your answer. It's great to see such a quick and precise answer nowadays

I'm at the point where i want to enable my strategy on a small account to see if everything works fine before enabling it on the big account.



Thanks for the precisions about the granularity.

My strategy is attached to a 60mn chart and to have correct backtest i have read the article about the granularity and the algo takes action on a 1mn time frame and like that it seems to be accurate as I have backtested and optimize it for 8 years of data and I believe to go down to the tick is not necessary for this swing trading strategy, tell me if I'm wrong ?



When you say about granularity : "This would especially be the case if you submit target and stop based on the fill prices of orders. "

Do you mean that when I submit an order with SL and target at the same time like actually :

EnterLongLimit(2, true, positionSize, C_target_price, "BoX_ORDER");
SetStopLoss("BoX_ORDER", CalculationMode.Pips, C_SL_Size , false);
SetProfitTarget("BoX_ORDER", CalculationMode.Pips, (C_SL_Size * riskReward));

These (SL and target) are not executed when the order is filled ?
And that there can be a difference between submitted price and filled price and so it's better to wait for the order to be filled and get the order entry price and set SL and target with CalculationMode.price based on the real fill price of the order ?




To handle disconnections I use on purpose a poor internet connection to create this case that might happends.


I have understand that by default the keepRunning option is seleted in the strategy options and that it will handle well the case of brief disconnections.


In a perfect world I would like the strategy to be never restared and to be completly autonomous.
But i'm affraid that the strategy will need to be restarted after after long disconnections.

Like today with the screen capture i send in this message or after the week end as the broker disconnect the quotes for a long moment every week end and i have see that on monday morning (like today) the connection status on the bottom left of the NT main window was in red. And i had to go to connection disconnect and do a reconnect to have the quotes and the green status again.
Then i have see that the strategy restarted.


I would like this case to never occurs, but when this occurs, is there another way than to manually disable and re enable the strategy ? Can I do it programmaticly ?
In all cases when this occurs if the strategy is restarted (but I would prefered that this didn't happend) it's sure it needs the good setting for the startBehavior.



Thanks a lot for your help.



Roman

[NinjaTrader_Jim]

Hello Roman,

Set methods using a CalculationMode other than Price would have the Target/Stop distanced from the average entry price of the order. Without using High Order Fill Resolution, the Set methods will be based off the entry's fill price which would be simulated with the OHLC values of the data series when the strategy backtests (or when we re-enable it and the trades are calculated from historical data.)

If we re-enabled the strategy with ImmediatelySubmit, and without High Order Fill Resolution, the stop and target will likely be calculated with a different price level, so the live stop/target orders would be cancelled and replaced with the target/stop orders calculated from historical data.

If you want to have stop/target resume with the same prices, or as close as possible, High Order Fill Resolution may be used (or submitting orders to a single tick data series.)

More information on how orders are filled with historical data can be found here - https://ninjatrader.com/support/help...ical_fill_.htm

8 years of tick data may be overkill for a backtest. Using a 1 minute data series for the fill series may be more sane, but we would have to keep in mind that the fill prices would not be as accurate as using single tick data for the fill series.

Set methods prep NinjaTrader for submitting target and stop orders once the execution for the entry order is seen. (Therefore it is advised to call the Set methods, before the entry order is submitted. In some cases where the Set method is called after the entry order, a previous Stop/Target level may be used which could get rejected.)

You can programmatically disable a strategy with CloseStrategy() or SetState(State.Terminated). Strategies cannot be programmatically enabled, and can only be enabled manually on a Chart or from the Strategies tab of the Control Center.

CloseStrategy() - https://ninjatrader.com/support/help...sestrategy.htm

SetState - https://ninjatrader.com/support/help...l?setstate.htm

I believe this should answer all of your questions. If anything else is withstanding or if there is something you would like me to clarify, please let me know.

[algoroman]

Hello Jim,


Thanks for your answers.

I would like to have possibility to set the order fill resolution to high in the backtests but as my strategy use multi data series to take decisions (daily to get informations, 60mn to find the setups, and 1mn chart to trade for backtests fiability) it doesn't work as NT refuse it.


Thanks for the precision to set the SL and target with the mode.Price, it seems more logical to set everything in this mode and I see a little difference in the backtests that should be more accurate.

In important point in my strategy is that the orders prices are calculated from the highs and lows of the last 48 hours on the 60mn chart and it's programmed to wait some time with some conditions to submit the orders, so I believe that order prices will remain the same if using live data or if the strategy has to be restarted and use historical datas so it would match in most cases and retrieve and manage correctly the orders if I use the startBehavior ImmedialtySubmit.



For the case of disconnections, I see different possibility that I could use together


First and more simple would be to change general settings for strategies connection loss handling would use keepRunning but probably use a larger number of restart attempts.

Another solution use together would be to use the Connection class to manage long disconnections to do something like if price and order feeds are disconnected from a to long time disconnect then reconnect. If I do that prgrammaticly, do you think it will restart the strategy (like manually when I disconnect, I get a promp warning for strategiy running, and when I reconnect it seems to restart the strategy as there is the last bars missing due to disconnection ) ?




Thanks a lot

[NinjaTrader_Jim]

Hello algoroman,

There is no supported means to programmatically restart a strategy. Strategies will disable on disconnection and will not re-enable upon connection, so you will have to re-enable them manually.

If you want to achieve more granular order fills with historical data, but are using a multi time frame strategy or Tick Replay, you can add a single tick data series (or a minute data series if you want order fills of minute granularity) and submit orders to that data series. This requires using Enter/Exit methods or Unmanaged method overloads that allow submitting to a desired BarsInProgress index. (See example from post #2.)

[algoroman]

Thanks a lot Jim

Have a nice day


Roman