Again with the disconnections

[JoseFRS]

Hello again,

I am writing to you again because I am already quite desperate with the issue of disconnections and I don't know what else to do.

I know that:
1.- Ninja has no control over disconnections and that it is the data provider that performs the disconnection and Ninja simply notifies it to me.
2.- I must deactivate the power saving of the network card.
3.- I must contemplate possible outages from my ISP (My computer is a cloud server contracted with a company with redundant internet connections).
4.- I must understand the different start behaviors of the strategy (I think I already understand all of them and from what I see, they only make sense On "Connecion Lost" and not On "Disconnected". In any case, I already have established it in "Kepp Running").

My problem usually occurs on weekends when my data provider (NinjaTrader Continuum) makes disconnections. Said this, I need you to clarify several things to me:

1.- It seemed to me to understand in some previous answer that in this case, Ninja stays disconnected and there is no way to reconnect automatically. That's right?
2.- I can understand that when the disconnection occurs, the strategies stop. But I don't understand why, when I restore the connection (manually of course) and the charts correctly reflect the purchased positions and the ProfitTarget and StopLoss set, when I re-enable the strategies, the ProfitTarget and StopLoss are cancelled and disappear.
3.- Do you ALWAYS go through the CONNECTION LOST state before reaching a DISCONNECTED state? Is there a way to get to DISCONNECTED without first going through STOPLOSS?
4.- What exactly does the "Disconnected Delay Seconds" parameter do? I have read the help many times, but it is not very clear:

4.1.- Is it the time that elapses from when a ConnectionLost is detected until it becomes Disconnected? If so, setting that parameter to "99999999" would never go DISCONNECTED after a weekend. However, it does happen (it happened to me this past weekend).
4.2.- What reconnection attempts are then made with the "Number of restart attempts" and "Restarts within x minutes" parameters? If in the end it is passed to DISCONNECTED and it is true that there is no way to automatically reconnect, then, what connection retries do both parameters perform?

I need to understand all this very well.

My strategies shouldn't be affected by the weekend disconnections (or any other). In other words. If the last tick received is on Saturday at 01:30 am and another one is not received until Sunday night, then perfect. But everything must continue to function as if nothing had happened. If there have been disconnections in between, the ideal is that the system reconnects itself. And in the worst case, if that were not possible (something that seems unusual to me on a platform of this level and that allows programming strategies) at least when I do the manual reconnection and manually activate my strategies again, the data (Positions, ProfitTargets and StopLoss) are maintained as if nothing had happened.

Please excuse my way of expressing myself, but I'm already quite desperate with all this. I have spent more than 1 year programming my strategies with Market Replay (where obviously there are no disconnections) and now I have found all this.

I need you to please explain to me ONE BY ONE each of the above points and better yet, tell me how I should configure Ninja and/or my strategies to get what I need.

Thank you and I apologize again for my desperate way of expressing myself.

All the best.

[NinjaTrader_Jim]

Hello JoseFRS,

Thanks for your post.

Reconnections are not attempted when we get disconnected for weekend server maintenance.

You will need to re-enable your strategies at least weekly.

When you re-enable the strategy, historical data is processed to determine the strategy position and any orders that should be active. What happens with the live working orders then is determined by your Start Behavior.

Synching Account Positions (Start Behaviors) - https://ninjatrader.com/support/help..._positions.htm

Disconnect Delay Seconds determines how long we should wait before a disconnect is detected by the Connection Loss Handling logic. This is relevant for the strategy, not the connection itself. More information on these properties can be found below:

https://ninjatrader.com/support/helpGuides/nt8/index.html?options_strategies.htm#UnderstandingninjascriptProperties

My suggestions:

If you want to keep a strategy running through disconnections, use Keep Running, and set Disconnect Delay Seconds to a value high enough to withstand the disconnection. Note: this does not help with server maintenance disconnects, which are expected on the weekends.

When weekend maintenance occurs, you will need to re-enable the strategy. If you use ImmediatelySubmit, the strategy will try and compare working orders that were calculated from historical data to live working orders. If they match , they will be resumed, if they do not match, live orders are cancelled and replaced with the historical orders. (If live orders are simply cancelled when using ImmediatelySubmit and no orders were submitted to replace them, it would mean the strategy did not calculate any orders when it processed historical data. Debugging the strategy logic can then tell more why a historical order was not calculated.)

We look forward to assisting.

[JoseFRS]

Hello again.

First of all, thank you for your quick response.

I forgot to tell you that I already have "ImmediatelySubmit" as the start behavior of my strategy. That said, and applying everything you've told me, I still don't understand why the following two things happen to me when I run the following test:

PROOF:

1.- I open Connection in NinjaTrader Continuum.
2.- I start my strategy on a market.
3.- At a given moment, my strategy opens a position (long or short, it doesn't matter) and sets a ProfitTarget and a StopLoss
4.- Immediately after, I manually close the NinjaTrader Continuum connection => My strategy stops
5.- I immediately re-establish a connection with NinjaTrader Continuum => On the chart (with ChartTrader activated), the open positions, the ProfitTarget and the StopLoss drawn can be perfectly observed.
6.- I start my strategy (with "ImmediatelySubmit" start behavior) => The open position is maintained, but the ProfitTarget and the StopLoss are cancelled.

FIRST QUESTION:

If I disconnect to NinjaTrader Continuum, connect to NinjaTrader Continuum and activate the strategy immediately one after the other (perhaps in the market there will have been 1 tick or at most 2 ticks in the meantime), how is it possible that the "orders do not match" to keep the Profit and StopLoss fixed? The open position is maintained. I do not get it.

SECOND QUESTION:

I have tried to assume that said ProfitTarget and StopLoss always cancel. In that case, and thanks to the fact that when I fix them I save their value in a database of my own, I make my strategy, when restarted, order again to establish a ProfitTarget and a StopLoss with the stored values. BUT IN THE ORDERS TAB, BOTH ORDERS ARE OBSERVED AS CANCELED!! I do not get it.

I beg you please help me with this.

Thanks in advance.

[NinjaTrader_Jim]

Hello JoseFRS,

I had left a note in my last post:

If live orders are simply cancelled when using ImmediatelySubmit and no orders were submitted to replace them, it would mean the strategy did not calculate any orders when it processed historical data. Debugging the strategy logic can then tell more why a historical order was not calculated.

Checking your strategy logic when enabling the strategy to see why it did not calculate previous orders would explain why ImmediatelySubmit cancelled the orders, but did not have any orders to replace.

For example, if you skip processing historical data with if (State == State.Historical) return; the strategy would not be able to calculate if it should be in a position and if it should have any active orders, because that part of the strategy processing is skipped.

I have included some pre written text to further describe how the start behaviors work.

To give a definite answer why your strategy did not come up with any orders that should be active when it processes historical data, debugging steps should be taken to se how the strategy executes in State.Historical.

Debugging Tips - https://ninjatrader.com/support/help...script_cod.htm

To answer your second question, WaitUntilFlat always cancels orders and assumes your account is flat, so the first live entry from the strategy is an entry order. ImmediatelySubmit assumes the account position is where it should be. Live orders are compared to those calculated from processing historical data. Matched orders resume, Unmatched orders are cancelled and replaced.

Start Behavior Educational information

When a strategy is enabled, it processes historical data to determine trades that the strategy would have made on the data that is already on the PC/chart and to determine what position the strategy is in. (Strategy positions are separate from actual Account positions.)

Wait Until Flat will wait until this virtual/historical position is closed before live orders can be submitted. It ensures that the strategy starts trading live from a flat position. If you enable the strategy when the account is flat, the strategy will waiting until this position calculated from historical data is closed so it is logically making trades starting from an entry signal.

Immediately Submit automatically submits working orders from when the strategy processed historical data, and assumes the strategy position and account position are where you want it when you enable the strategy. This is typically used to have a strategy resume a position after disabling/enabling. If the strategy already had live orders running, the orders will resume with the new enablement of the strategy if they match the historically calculated orders. If the orders calculated from historical data do not match the live working orders, the live working orders will be cancelled and replaced by those calculated from historical data.

Sync Account Positions is an additional option that has NinjaTrader submit an order to sync the account position to the position calculated by the strategy. (Not the other way around.)

Adopt Account Position would be used if you want the strategy to inherit the Account Position on enablement. This requires additional programming.

If you do not want the strategy to calculate a position from processing historical data: Simply add if (State == State.Historical) return; to the top of your strategy logic so historical processing is skipped. The strategy will then always start from a flat position because it has not calculated any orders.

Strategy vs. Account Position — https://ninjatrader.com/support/help..._account_p.htm

Start Behaviors — https://ninjatrader.com/support/help..._positions.htm

[JoseFRS]

Hello again.

Thanks for your reply.

I think I understood what you are telling me. It is true that my strategy did not deal with historical data.

As I seem to understand you, when starting, my strategy must process all the historical data so that it can regenerate the same orders that it generated in real time before the disconnection and thus remain in the same state as my account. Is that right?

In that case, I just have a big doubt that scares me.

My strategy is only executed OnEachTick and when IsFirstTickOfBar=True (Only on opening bar).

However, it seemed to me that when historical data is processed, it is always processed as OnBarClose since in that case the bars do not have the Ticks.

I am afraid that this may cause my strategy to generate different orders when dealing with historical data than when it was working with such data in real time before the disconnection, so it is possible that even so, the ProfitTarget and StopLoss set before the disconnection will not coincide with those calculated during the treatment of historical data, also causing their cancellation when restarting my strategy after having raised the connection again. Is it so?

Thanks in advance.

[NinjaTrader_Jim]

Hello JoseFRS,

You understand correctly.

Using Calculate.OnEachTick/OnPriceChange does impose some challenges to get historical and realtime aligned.

Before we get further, we should have a good understand of the discrepancies below.

Discrepancies between realtime and backtest - https://ninjatrader.com/support/help...ime_vs_bac.htm

Understanding the above, there would be some paths forward:

1. Try using Tick Replay and submitting orders to a single tick data series with Unmanaged orders or Managed Enter/Exit methods (Set methods could not be used in this case for intrabar order fills.) This would allow for intrabar actions, and intrabar order fills with historical processing.
2. Try modifying the strategy to follow Calculate.OnBarClose logic, which more gracefully can be simulated with historical processing
3. Continue moving forward without processing historical data, but use AdoptAccountPosition, so you can protect the inherited account position once the strategy reaches State.Realtime.

I suspect that #3 would be the easiest path to take, and I have attached some examples that can demonstrate how you could write a strategy that skips processing historical data, and can protect an inherited account position.

[JoseFRS]

Hello again,

Thanks for everything. At least I feel understood now, .

Just 2 hours ago I discovered and tried AdoptAccountPosition, but it didn't work as expected.

At least in theory it seems to me that it is just what I need. That is, when reconnecting the strategy, I want the open positions on that instrument in the account, as well as the ProfitTarget and StopLoss set in it, to be adopted by the strategy as its orders (In fact, they always will be. I will never trade manually and the strategy is designed so that there would never be any disconnections).

I'm going to see the examples you sent me and try again.

I will give you news shortly.

Thanks for everything.

All the best.

[NinjaTrader_Jim]

Hello JoseFRS,

As you may see in the Help Guide, AdoptAccountPosition would not inherit the orders, just the position of the account. Otherwise the behavior is much like ImmediatelySubmit.

My examples above:

1. Skip processing historical data, so no historical orders are calculated (all active live orders previously submitted by the strategy would be cancelled)
2. In State.Realtime, the position is protected with new orders

So there is brief moment where orders are cancelled and then replaced with the new orders from State.Realtime, which should be trivial.

If you absolutely need the orders resumed and not cancelled/replaced with minor differences to the orders, programming the strategy to come up with the same historical orders when using ImmediatelySubmit would be necessary.

[JoseFRS]

Hello again,

I'm desperate with this and I need a solution, please.

I have three strategies running at the same time, one on the M6E market, one on M6B and one on M6A.

I have managed to perfectly treat the CONNECTION_LOST events thanks to setting their treatment to "Keep Running" and setting the "Disconnect delay Seconds" parameter to a high value.

But the problem occurs with DISCONNECTED events (especially on weekends).

I've been testing by doing disconnects myself in Ninja and it seemed like I found the solution:

• I have performed the tests with a single market, and with the parameters IsAdoptAccountPositionAware = true and StartBehavior = StartBehavior.AdoptAccountPosition.
• With the connection established, I start the strategy and wait for it to open a position and set a Profit Target and a StopLoss.
• Once position is open, I close the connection manually (See OnConnectionStatusUpdate how it goes to Disconnected), which stops my strategy.
• Afterwards, I establish the connection again. Obviously, my strategy is still disabled, but when I enable it, it can be seen how the strategy acquires the account positions correctly, having to re-set the Profit Target and the StopLoss since these have been lost after re-enabling it. .
• For this, I had already taken care of saving both values and that the strategy set them again in said startup.
• Therefore, everything finally seemed to work correctly in said test.

However, this weekend I have put the 3 strategies to run to see how they behave with a DISCONNECTED.

When the DISCONNECTED occurred (02/05/2022 09:32:39), the three strategies had two open positions each in their market and with their corresponding Profits Targets and Stop Loss established.

Today (02/06/2022 11:03:10), I have opened the connection again without problems. BUT WHEN I TRY TO ENABLE THE STRATEGIES, THE SYSTEM DOES NOTHING. IT IS NOT ABLE TO ENABLE THEM AND DOES NOT INDICATE ANYTHING IN THE LOG TAB!!!

The behavior I expect is the same as when I did the tests. In other words, each strategy acquires the positions that the account has open in each corresponding market, and each one of them will also be in charge of reestablishing its corresponding Profits Targets and StopLoss.

Please help me. I'm already desperate with this.

Thanks in advance.

All the best.

[NinjaTrader_Jim]

Hello JoseFRS,

An issue was discovered with AdoptAccountAware when there are preexisting orders on the account/instrument when the strategy is enabled. The strategy should not enable in this case, but we are seeing an issue where the expected error is not coming up when enabling from the Control Center, and the strategy is showing an error, but still enabling when it is enabled from a chart. (We are investigating this internally with the ticket ID QA-5743, I'll update this post as new information arrives.)

The expected behavior is noted here:

The account and instrument the strategy is started on must not have any working orders which were submitted outside of the strategy, or by another instance of the same strategy. If an order is detected, the strategy can not be started until these orders have been manually managed.

https://ninjatrader.com/support/helpGuides/nt8/index.html?syncing_account_positions.htm#AdoptAccountPosition

To be able to work around this specific case with AdoptAccountPosition, you can cancel the pending orders on the account before you re-enable the strategy. If your strategy has behavior to place new target/stop orders to protect the position in State.Realtime when the strategy is enabled, your position would be then protected again after the strategy is enabled.

Otherwise, you may want to go back to using ImmediatelySubmit, where the strategy would process historical data, compare live working orders with the working orders calculated from historical data. (Orders that match are resumed, and orders that do not match are cancelled and replaced.)