Running strategies with connection loss handling set to 'recalculate' causing problem

[adamus]

When NT7 was in early beta, I tried and decided against using the connection loss handling 'recalculate' option.

However now that I have experienced some other issues caused by connection loss, I decided to try it again, hoping that NT's recalculate functionality might offer a more robust solution.

Foolishly I didn't do any testing to make sure my hopes would be realised. Instead I set my live NT7 to do 'recalculate' handling on Wednesday and today I have decided to switch back.

The documentation and the forum provides some ambiguous info about what exactly the recalculate option does.

I wanted to see NT7 submitting delayed orders after a period of connection loss. However in 72 hours of trading over 20 strategies, I didn't see this happen but what I did see happen was a lot of orders being 'orphaned' by their strategies - exit orders that should have been cancelled when the positions were closed.

I also saw the Strategy tab display show a lot of strategies having open positions with average entry prices and unrealized profits after the positions had been closed, regardless of the fact that they also showed the realized profit. This gave me the strong feeling that somewhere inside NT7's strategy control, its left hand didn't know what its right hand was doing.

The final nail in the coffin as far as I'm concerned was today on GBP/JPY, where at about 12:00PM GMT, the strategy reversed its position but failed to enter the stop order. On manual restart later when I realised NT was displaying incorrect info on the Strategies tab, the reversal had disappeared.

I read elsewhere on the forum that 'automatic recalculation' does not reload missing data after the connection loss, so that is probably the biggest factor here. It seems then that recalculation with each connection loss is not a robust approach. When the latest bar is different from the real bar, then the recalculation of the strategy can easily send the strategy in the wrong direction as I've just seen.

This is my main point - take from it what you will. As far as I can tell, it's akin to offering me a gun, but without any sights to aim with.

I could upload the logs and traces if you want to check what was going on, but I don't think you'll come to any better conclusion.

[NinjaTrader_RyanM1]

Adamus,

Thank you for the feedback. Correct - the recalculate option does not automatically reload historical data. It recalculates the strategy, running it against historical data, and comparing the historical run of the strategy to the actual one. If you're running with calculate on bar close = false, these will never match and could lead to orders being canceled, and account / strategy position mismatch.

[drolles]

NT support,

Is there a means (method or sets of methods) in the standard API that could request a reload of the data?

This logic appears flawed to me; when you say the strategy is being recalculated – recalculated against what data? If the historical data isn’t there to be recalculated against how is the strategy recalculating?

Thanks and regards,

drolles

[NinjaTrader_Austin]

Drolles, thanks for your response. I will have someone get back to you on Monday.

[adamus]

Thank you for the feedback. Correct - the recalculate option does not automatically reload historical data. It recalculates the strategy, running it against historical data, and comparing the historical run of the strategy to the actual one. If you're running with calculate on bar close = false, these will never match and could lead to orders being canceled, and account / strategy position mismatch.

Hi,

I'm not using CalculateOnBarClose=false, it's always set to true.

It's not just a situation for CalculateOnBarClose=false, as drolles pointed out, it makes no sense to recalculate the strategy position if one doesn't also load the data from the period where the connection is down.

[NinjaTrader_RyanM1]

Thank you both for your suggestions. Unfortunately there is no way to force an historical data reload. If you reload historical after a disconnection then the chance of syncing is even less likely. The only way for strategies to recognize prior orders and positions is if the historical run of the strategy matches exactly with a prior, actual one.

There are additional disconnect handling methods available if you prefer not to recalculate:

http://www.ninjatrader.com/support/helpGuides/nt7/strategies_tab.htm

[adamus]

Ryan, first thanks for the link to the disconnect handling stuff, but re sync'ing and reloading data: either I am misunderstanding what you wrote or I just plain can't see the sense of what you say. After a disconnection, the missing data has to be loaded. How can a strategy trade successfully if it is missing data from a period of disconnection?

[NinjaTrader_RyanM1]

Yes, it is a good suggestion to reload historical data and have the most recent data available for sure. With the current system though, it would mean that any recalculate efforts will fail in picking up prior strategy placed positions and orders. This feature is dependent on a 100% match between the new historical run and the prior, actual run. Once you reload data, the data sets available between these runs are different, and part of the syncing process means it will cancel any of the strategy placed orders when these don't match.

My post is mainly an observation that an automatic reload historical data with the setting "recalculate" would almost always result in a mismatch between the new historical run and the prior, actual run.

[adamus]

That's not my experience. I run 20 systems on 60 mins bars and I find that restarting my strategy only results in different positions about 1 in 100 occurrences. Different stop and limit prices are more common but they don't represent a problem.

If I have a connection loss over the end of the hour, then my close is going to be wrong, so I have to reload at some point to save the correct data to my HDM.

[Sameer]

Running strategies with connection loss handling set to 'Recalculate'.

Thank you both for your suggestions. Unfortunately there is no way to force an historical data reload. If you reload historical after a disconnection then the chance of syncing is even less likely. The only way for strategies to recognize prior orders and positions is if the historical run of the strategy matches exactly with a prior, actual one.

There are additional disconnect handling methods available if you prefer not to recalculate:
http://www.ninjatrader.com/support/h...tegies_tab.htm

Hi,

But now this problem has resolved or not ? I am using NinjaTrader 64 bit 7.0.1000.22

Thanks
Sameer.

[NinjaTrader_Bertrand]

Hi Sameer, there have not been any changes in this regard, we would expect the data to not be reloaded automatically in this case. We do have an enhancement suggestion tracked though for our next major platform update in this regard, I will make sure to include your vote as well on it.

Thanks,