Different backtest results after NinjaTrader upgrade (Trailing Stop bug)

[NinjaTrader_ChrisL]

Hi John,

If you are unsure of the orders placed during the backtest please print out the values of the orders as they are being processed. We have an example here that monitors stop loss and profit target orders:

https://ninjatrader.com/support/helpGuides/nt8/?monitoring_stop-loss_and_profi.htm

It would also make it more clear what is happening if you use the Playback connection to test your strategy.

Kind regards,
-ChrisL

[John_Paodo]

Hello

I did what you advice and here is the result (Printscreen: 2021.09.21 - 02 - NinjaScript Output)

As you can see, the Initial Trailing Stop order is sent with a value 1.3075 where the Entry Price is 1.30223.

Since the Initial Trailing Stop price is way above the Entry Price, the trade is immediately exited.

My guess is that, in backtesting, NT calculates the Initial Trailing Stop price based on the high of the Entry bar (1.32071 x 0.99 = 1.3075) (Printscreen 2021.09.14 - 04 - Trailing 1% - Extreme Trade - Chart 60 Minute + Printscreen: 2021.09.14 - 05 - Trailing 1% - Extreme Trade - Chart 1 Minute)

This does not make any sense since this high (1.32071) is yet to come at the Entry of the Trade.

Best regards





NinjaScript Output:

ENTRY: orderId='NT-00000-1770' account='Backtest' name='LongEntry1' orderState=Submitted instrument='EURUSD' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=92 500 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

ENTRY: orderId='NT-00000-1770' account='Backtest' name='LongEntry1' orderState=Accepted instrument='EURUSD' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=92 500 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

ENTRY: orderId='NT-00000-1770' account='Backtest' name='LongEntry1' orderState=Working instrument='EURUSD' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=92 500 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

STOP: orderId='NT-00001-1770' account='Backtest' name='Trail stop' orderState=Submitted instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.3075 quantity=92 500 tif=Gtc oco='NT-00000-1770' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

STOP: orderId='NT-00001-1770' account='Backtest' name='Trail stop' orderState=Accepted instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.3075 quantity=92 500 tif=Gtc oco='NT-00000-1770' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

STOP: orderId='NT-00001-1770' account='Backtest' name='Trail stop' orderState=Working instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.3075 quantity=92 500 tif=Gtc oco='NT-00000-1770' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

ENTRY: orderId='NT-00000-1770' account='Backtest' name='LongEntry1' orderState=Filled instrument='EURUSD' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=92 500 tif=Gtc oco='' filled=92500 averageFillPrice=1.30223 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

STOP: orderId='NT-00001-1770' account='Backtest' name='Trail stop' orderState=Filled instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.3075 quantity=92 500 tif=Gtc oco='NT-00000-1770' filled=92500 averageFillPrice=1.30223 onBehalfOf='' id=-1 time='2013-07-11 00:00:00' gtd='2099-12-01' statementDate='2021-09-21'

[NinjaTrader_ChrisL]

Hi John, thanks for your reply.

If you would like a more detailed trailing stop in backtesting you would need to use an Exit method and submit it on a 1 tick series as opposed to SetTrailStop which will always be submitted on the primary series of the chart. Note the BarsInProgressIndex you can use to target a 1 tick series.

ExitLongStopLimit()
ExitLongStopMarket()

Kind regards,
-ChrisL

[John_Paodo]

Chris,

I don’t need extra granularity. I need that NT realizes there might be something wrong about Trailing Stop in backtesting. Exiting a (winning) Trade directly after entering it … This is just so wrong, whatever the granularity.

I bought a multi-broker lifetime licence (1650$) and this one of my first post.

It has been 3 weeks now that I am trying to make NT understand there might be something wrong. But all I get is « push back answers » to meet the SLA without really considering I might be right.

To be honest, I am disappointed.

Nevertheless, I did some extra research (this bug is blocking me).

As you advised, I used the Playback Connection on the strategy you provided. I did the following modifications: This strategy is only entering Long at 12:00 with amarket order and a 0.8% trailing stop. As you advised, I added monitoring of the order via NinjaScript Output (Strategy used: 2021.09.26 - 01 - TestTrailStopMonitor2020)

As you can see on the printscreen 2021.09.26 - 03 - NT 8.0.23.3 - Crash, the Strategy crashes in Playback with the Message « Stop price can't be changed above the market »

I ran the exact same Strategy on the exact same data but on an older version of NT (8.0.21.1) and the issue is not there.

I checked the NinjaScript Output in both environments, see print screens:

2021.09.26 - 02 - NT 8.0.21.1 - OK
2021.09.26 - 03 - NT 8.0.23.3 - Crash



Line 11 is different between both environments:


NT 8.0.21.1 - NinjaScript Output

STOP: orderId='43051abfcf9c4d31807934b08ca0a343' account='Playback101' name='Trail stop' orderState=ChangeSubmitted instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.06973 quantity=92 500 tif=Gtc oco='3f2a06f10b0944be96d394a5aa3392e9' filled=0 averageFillPrice=0 onBehalfOf='' id=21 time='2020-03-19 12:00:04' gtd='2099-12-01' statementDate='2020-03-19'


NT 8.0.23.3 - NinjaScript Output

STOP: orderId='fd3b1481cbba445b9a8ad2fd89287f18' account='Playback101' name='Trail stop' orderState=ChangeSubmitted instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.10258 quantity=92 500 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=598 time='2020-03-19 12:00:02' gtd='2099-12-01' statementDate='2020-03-19'

Strategy 'TestTrailStopMonitor2020/202937792' submitted an order that generated the following error 'Unable to change order'. Strategy has sent cancel requests, attempted to close the position and terminated itself.




For whatever reason in NT 8.0.24.3 (not in 8.0.21.1) the Trailing Stop is changed to a value way above the market, stopping the Strategy.

Hope this help investigate the issue.

[NinjaTrader_ChrisL]

Hi John, thanks for your reply.

We have already established that on older versions, the SetTrailStop method is influenced by the extra series which is not correct. We have a warning about this rejected order scenario in the SetTrailStop help guide page:

Trail stop orders are modified based on the strategies 'Calculate' settings. In the case of 'Calculate' on bar close, when the bar closes the trail stop order modification will occur using the lowest/highest price of the bar as the reference price to apply the trail offset. Subsequently, if the open price of the next bar is significantly higher or lower than this price then there is a possibility that the calculated trail stop price is now an invalid stop price. This is a risk with modifying any stop order closer to the current market price since any modification above/below the current price would be rejected.

I attached an example that adds a 1 tick series and is able to trail intrabar historically and in playback/live mode. Please test this out and study the code through the NinjaScript editor. It is a modification of the example here:
https://ninjatrader.com/support/help...and_onexec.htm


Kind regards,
-ChrisL

[John_Paodo]

Chris,

The modification of the stop price we see in « 2021.09.26 - 03 - NT 8.0.23.3 - Crash » is not in line with your explanation.

Can you explain where the 1.10258 stop price is coming from ?

The Trailing Stop is 0.8% (=0.992)
Entry Price = 1.07833

High of the Entry bar: 1.08057 x 0.992 = 1.07193
High of the Previous bar: 1.08268 x 0.992 = 1.07402
High of the Next bar: 1.08401 x 0.992 = 1.07534

Regards

[NinjaTrader_ChrisL]

Hi John, thanks for your reply.

I have been trying to reproduce on the replay data that I have available (9/19-9/21). Can you find some playback data from the last 90 days where it is reproducible with the test script you posted?

Kind regards,
-ChrisL

[John_Paodo]

Hello Chris,

As requested, here is a recent example (10th of August 2021): 2021.10.05 - 07 - Recent Example

The Trailing Stop is 0.8% (=0.992)
Entry Price = 1.17209
Initial Stop = 1.17209 * 0.992 = 1.16271 -> This is correct
But then the Stop « jumps » to 1.17419, generating the Crash

I loaded the data from FXCM (Bid/Ask/Last Tick).

I attached the script used in order to reproduce it (slightly modified to remove the date & time filter).

Regards

[NinjaTrader_ChrisL]

Hi John,

I am not able to reproduce anything on playback for August 10th running an EURUSD 60 minute chart. Please provide the bar type, period, and a specific time where I can see this.

Kind regards.