Strategy is executing differently depending on the start date I choose.

[Danilod]

This happened to me using the strategy analyzer backtest and the strategy on the chart itself. When doing a backtest of my strategy I noticed that if I enter "from date: 03/01/2021" instead of "02/26/2021", not only the first entry is affected but also the execution of it's exit condition which is within both ranges. In one case 1 order is replaced with another 8 orders.

In both cases I am running the same strategy with the same parameters. The only difference is the from date.

I am not allowed to share the entire strategy as this is a work for someone else but just for the sake of helping you find the issue, let me tell you that my strategy involves an exit condition based on RSI value in the 60min timeframe calculated on the las 14 bars. Could that be the cause of the problem?

In this case the difference in profit between both backtest was only 300$ on a total Profit of $5000. My results are still valid for sake of validating the strategy but I am afraid an issue like this could have a bigger effect next time.

Let me know what you think and if there is any workaround to avoid these edge cases.
Thanks,

[bltdavid]

Are you connected when using the Strategy Analyzer?

Is all the historical data needed by the strategy available?

Have you tried downloading the historical data ahead of time?

[tgn55]

In my experience there could be several reasons...

• Is the Bars series on your chart IDENTICAL to the series selected in backtest? Including settings like break at EOD?
• Does your strategy include any indicators that might need more time to settle? Things like EMA have a long memory... the value can in theory depend on data prior to your BarsRequiredToTrade setting... but if you make that generous, the effect should be minimal. Bottom line is... make sure any/all data has settled to a static state before you use it.

Hope it helps,
T.

[NinjaTrader_Jesse]

Hello Danilod,

Welcome to the NinjaTrader support forums.

What you described sounds like something which can happen based on the changing data. As tgn55 mentioned indicators can be affected by the changing data, if new data is included or less data is included that could ultimately cause differences down the line in how your conditions are evaluating.

You mentioned you have used the RSI so a good starting point at seeing what may be happening would be to look at the indicator values between the two tests. In test A you can make note of the indicator values around the fills and then compare that to test B to see if the values match. If they don't match that's very likely going to be a starting point where you see differences happening. Generally prints can be used to compare tests like this, a simple example would be:

Print(Time[0] + " " + RSI(12)[0]);

We should most often not expect two tests to match identically when the data changes. Without investing some time in exploring what specifically changed between those two tests it would be very hard to say if they should or should not match.


I look forward to being of further assistance.

[Danilod]

Are you connected when using the Strategy Analyzer?

Is all the historical data needed by the strategy available?

Have you tried downloading the historical data ahead of time?

Yes, I'm always connected using continuum connection.
I've tried to download all the year of data but it doesn't seem to make any difference. Do I need to do anything else to make use of this historical data in analyzer or it will automatically use it if it finds it in memory?
Thanks.

[NinjaTrader_Jesse]

Hello Danilod,

The historical data you download will be able to be used by the platform automatically.

You can see and manage the historical data you have by using the Historical Data window: https://ninjatrader.com/support/help...ta_manager.htm

You can also open a chart for the timeframe you want to request and Right click -> Reload historical data. You can then visually see the data that has been loaded to confirm its present.


I look forward to being of further assistance.

[Danilod]

Thanks for all your messages!

In my experience there could be several reasons...

• Is the Bars series on your chart IDENTICAL to the series selected in backtest? Including settings like break at EOD?
  
  Yes, in fact. The same error appears when I just use two Strategy analyzer with to different Start Dates.
  • Evaluating from 03/01/2021 makes that the first exit in the month is on 03/01 12:05pm.
  • But if I evaluate from 02/25, the first exit on march is on 03/04 on a stop loss. The Exit condition based on RSI data that must have happened on 03/01 12:05pm is not triggered.
• Does your strategy include any indicators that might need more time to settle? Things like EMA have a long memory... the value can in theory depend on data prior to your BarsRequiredToTrade setting... but if you make that generous, the effect should be minimal. Bottom line is... make sure any/all data has settled to a static state before you use it.
  
  mmmm, I think something like this might be happening, not sure what. I am using in one of my conditions:

   RSI(<Primary> (60 Minute), 14, 3).Avg[0]

  It seems that when I don't give space in the past in the range, ninja trader is mistakenly triggering the condition.

Hope it helps,
T.

[Danilod]

Hello Danilod,

The historical data you download will be able to be used by the platform automatically.

You can see and manage the historical data you have by using the Historical Data window: https://ninjatrader.com/support/help...ta_manager.htm

You can also open a chart for the timeframe you want to request and Right click -> Reload historical data. You can then visually see the data that has been loaded to confirm its present.


I look forward to being of further assistance.

Thanks for you help too!
I did all this. I have all the data loaded.

[Danilod]

Hello Danilod,

Welcome to the NinjaTrader support forums.

What you described sounds like something which can happen based on the changing data. As tgn55 mentioned indicators can be affected by the changing data, if new data is included or less data is included that could ultimately cause differences down the line in how your conditions are evaluating.

You mentioned you have used the RSI so a good starting point at seeing what may be happening would be to look at the indicator values between the two tests. In test A you can make note of the indicator values around the fills and then compare that to test B to see if the values match. If they don't match that's very likely going to be a starting point where you see differences happening. Generally prints can be used to compare tests like this, a simple example would be:

Print(Time[0] + " " + RSI(12)[0]);

We should most often not expect two tests to match identically when the data changes. Without investing some time in exploring what specifically changed between those two tests it would be very hard to say if they should or should not match.


I look forward to being of further assistance.

I tried printing RSI values and the value in both backtest runs are the same around the time of the RSI Target Trigger 12:05pm. The odd thing is that the RSI value should have triggered the exit multiple times according to this but if I see the chart, RSI Avg doesn't seem to be greater than the value I specified to trigger.

Thanks for your help on all this.

[NinjaTrader_Jesse]

Hello Danilod,

A next step would be to check if the condition is true, the Print could be placed inside your condition along with the entry. If you don't see the print then something is happening surrounding the entry condition. If you do see the print you might be hitting a managed order handling rule which is ignoring the order or possible the EntriesPerDirection depending on the situation. You can enable TraceOrders and then have the output window open during the test to check for order handing errors.

https://ninjatrader.com/support/helpGuides/nt8/traceorders2.htm?zoom_highlightsub=traceorders

I look forward to being of further assistance.

[Danilod]

Hello Danilod,

A next step would be to check if the condition is true, the Print could be placed inside your condition along with the entry. If you don't see the print then something is happening surrounding the entry condition. If you do see the print you might be hitting a managed order handling rule which is ignoring the order or possible the EntriesPerDirection depending on the situation. You can enable TraceOrders and then have the output window open during the test to check for order handing errors.

https://ninjatrader.com/support/helpGuides/nt8/traceorders2.htm?zoom_highlightsub=traceorders

I look forward to being of further assistance.

Thanks Jesse,
Yes, I did print both executions as you said and the one with start date = 02/25 is matching the first exit condition on 03/07; whereas the backtest with start date = 03/01 is matching the first exit condition on 03/01. The same results are appearing when using TraceOrders. It show not reason why it was ignored; it just doesn't appear. The only way of making both backtest give the same orders are if I add many more "Bars Require to Trade". Eg: 400 instead of 20.

Here is the output I am receiving in the one that is executing the exit 12:05pm:

 7/27/2021 3:05:39 PM Strategy 'RSIBollingerU/-1': Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='' Mode=Ticks Value=400 IsSimulatedStop=False IsMarketIfTouched=False
7/27/2021 3:05:39 PM Strategy 'RSIBollingerU/-1': Entered internal SetStopTarget() method: Type=Target FromEntrySignal='' Mode=Ticks Value=600 IsSimulatedStop=False IsMarketIfTouched=False
3/1/2021 12:00:00 AM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 12:00:00 AM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='EntryLong' FromEntrySignal=''
3/1/2021 6:30:00 AM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 6:30:00 AM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='EntryLong' FromEntrySignal=''
3/1/2021 6:30:00 AM Strategy 'RSIBollingerU/-1': Ignored SubmitOrderManaged() method at 3/1/2021 6:30:00 AM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='EntryLong' FromEntrySignal='' Reason='Exceeded entry signals limit based on EntryHandling and EntriesPerDirection properties'
3/1/2021 8:10:00 AM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 8:10:00 AM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='EntryLong' FromEntrySignal=''
3/1/2021 8:10:00 AM Strategy 'RSIBollingerU/-1': Ignored SubmitOrderManaged() method at 3/1/2021 8:10:00 AM: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='EntryLong' FromEntrySignal='' Reason='Exceeded entry signals limit based on EntryHandling and EntriesPerDirection properties'
3/1/2021 12:00:00 PM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 12:00:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal=''
3/1/2021 12:00:00 PM-76.137991129318
3/1/2021 12:00:00 PM Strategy 'RSIBollingerU/-1: Cancelled pending exit order, since associated position is closed, orderId='NT-00002-2332' account='Backtest' name='Profit target' orderState=Working instrument='YM 09-21' orderAction=Sell orderType='Limit' limitPrice=31498 stopPrice=0 quantity=1 tif=Gtc oco='NT-00000-2332' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2021-03-01 00:00:00' gtd='2099-12-01' statementDate='2021-07-27'
3/1/2021 12:00:00 PM Strategy 'RSIBollingerU/-1: Cancelled pending exit order, since associated position is closed, orderId='NT-00001-2332' account='Backtest' name='Stop loss' orderState=Working instrument='YM 09-21' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=30498 quantity=1 tif=Gtc oco='NT-00000-2332' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2021-03-01 00:00:00' gtd='2099-12-01' statementDate='2021-07-27'
3/1/2021 12:05:00 PM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 12:05:00 PM: BarsInProgress=0 Ac [ATTACH=JSON]{"data-align":"none","data-size":"large","data-attachmentid":1165390}[/ATTACH][ATTACH=JSON]{"data-align":"none","data-size":"large","data-attachmentid":1165391}[/ATTACH] tion=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal=''
3/1/2021 12:05:00 PM Strategy 'RSIBollingerU/-1': Ignored SubmitOrderManaged() method at 3/1/2021 12:05:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal='' Reason='This was an exit order but no position exists to exit'
3/1/2021 12:05:00 PM-76.137991129318
3/1/2021 12:10:00 PM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 12:10:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal=''
3/1/2021 12:10:00 PM Strategy 'RSIBollingerU/-1': Ignored SubmitOrderManaged() method at 3/1/2021 12:10:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal='' Reason='This was an exit order but no position exists to exit'
3/1/2021 12:10:00 PM-76.137991129318
3/1/2021 12:15:00 PM Strategy 'RSIBollingerU/-1': Entered internal SubmitOrderManaged() method at 3/1/2021 12:15:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal=''
3/1/2021 12:15:00 PM Strategy 'RSIBollingerU/-1': Ignored SubmitOrderManaged() method at 3/1/2021 12:15:00 PM: BarsInProgress=0 Action=Sell OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='Exit_Long_RSI' FromEntrySignal='' Reason='This was an exit order but no position exists to exit'

Attached the orders

[NinjaTrader_Jesse]

Hello Danilod,

It looks like you are getting some ignored orders and there are some reasons listed along with that. Here are the reasons isolated:

Reason='Exceeded entry signals limit based on EntryHandling and EntriesPerDirection properties'

Reason='This was an exit order but no position exists to exit'

The first reason is the EntriesPerDirection or EntryHandling property. If you are trying to enter in the same direction more than once you need to either increase the EntriesPerDirection to a number higher than 1 or change the EntryHandling to EntryHandling.Unique.

The second reason would be if you called an Exit order which specifies an entry that does not exist. Based on the first reason that would make sense, if the entry was not submitted but your exit logic is true then we should see the warning.



I look forward to being of further assistance.

[Danilod]

Hello Danilod,

It looks like you are getting some ignored orders and there are some reasons listed along with that. Here are the reasons isolated:



The first reason is the EntriesPerDirection or EntryHandling property. If you are trying to enter in the same direction more than once you need to either increase the EntriesPerDirection to a number higher than 1 or change the EntryHandling to EntryHandling.Unique.

The second reason would be if you called an Exit order which specifies an entry that does not exist. Based on the first reason that would make sense, if the entry was not submitted but your exit logic is true then we should see the warning.



I look forward to being of further assistance.

Thanks for you reply. But I am still confused, the exit that was placed at 12:05 was wrong, not following my rules. When I change the start date to 02/25 it doesn't trigger the mistaken exit. I don't see the relation with what you are mentioning.

Sorry if I am missing something obvious. I am pretty new to NinjaScript.
Thanks.

[NinjaTrader_Jesse]

Hello Danilod,

If the exit was placed at the wrong time you could use prints to identify why your condition did that. The entry had otherwise been ignored and that is why the entry did not show up. The message about the exit is simply because your logic tried to exit a position which does not exist.

I look forward to being of further assistance.

[tgn55]

Hey Danilod,
as Jesse noted in post #12, you should first resolve the issue re EntriesPerDirection exceeded on 3/1 at 6:30 AM.

This could be simply increasing that value... but, since we don't know the details of your strategy, please confirm that your strategy really is expected to open a second position before the first position is closed... If the answer is "yes"... then increasing EntriesPerDirection above 1 should fix it. If the answer is "No"... then maybe your strategy logic has a problem.

Next, your code is trying to exit a position when there is no position... so either you are flat, or, I think this error could also be triggered if you used method signatures with signal names... I see your exit orders have a signal name... do you also provide signal names with your entries? And if so do they always match up? By which I mean, you are requesting to exit a position with a matching entry signal. This seems a long shot... as sometimes trades happen... and sometimes not, so I'm guessing this code is kosher. But.. worth confirming.

Which just leaves the question... can your code execute the exit logic when you are flat? Obviously... it shouldn't!

Cheers,
T.