Backtesting Discrepancies - Obvious Example

[markkm]

I noticed that too. It's ok. Not the main subject of this post.
Let's get to the point, starting a strategy from the tab window VS Chart.
Why does it yield different results ?

[markkm]

I tested adding to the Strategies tab of the Control Center and I am finding that 'SystemPerformance.AllTrades[SystemPerformance.AllTrades.Count - 1].ProfitCurrency' is always 0.

Sorry I just saw this.
Please take your time and let me know.
Thanks a lot!

[NinjaTrader_ChelseaB]

Hello markkm,

I've found the issue is related to IncludeTradeHistoryInBacktest. This must be set to true for a strategy added directly to the Strategies tab to have orders affect the SystemPerformance collection.
If IncludeTradeHistoryInBacktest this will resolve the issue.

I've reported the behavior to our development for clarification on why this affects real-time orders. Once I have more information I will post in this thread.

[markkm]

So, I just want to make sure:

1. Setting a strategy on a chart seems to be more consistent with backtest results, and I think I saw a while ago one of your threads referring to always "Reload historical Data" on a chart prior to launching a strategy. Is it correct?

2. If add IncludeTradeHistoryInBacktest = true, in if (State == State.SetDefaults), would that fix the problem (daily limit stop loss) if I set my strategy is set on the strategy tab ?

3. This is only 1x problem I found using strategy tab VS Chart. Since I have MultiTimeframe strategies, so moving forward, what is the best approach to making sure that backtest algorithm will have similar behavior to live trading?


Thank you,
Mark

[NinjaTrader_ChelseaB]

Hello Mark,

I do not recall advising to always reload historical data. Generally this is advised if the PC Clock is adjusted.

If you provide a link to the forum post in question I may be able to provide context.

Yes, setting IncludeTradeHistoryInBacktest = true; in State.SetDefaults and then removing the instance of the strategy and adding a new instance (so the defaults are pulled) will correct the behavior when adding a strategy to the strategies tab of the Control Center.

Below is a public link to a forum post on intra-bar granularity for getting historical orders to match real-time orders as closely as possible.

https://ninjatrader.com/support/forum/showthread.php?p=491297#post491297

[markkm]

I do not recall advising to always reload historical data. Generally this is advised if the PC Clock is adjusted.

https://ninjatrader.com/support/forum/showthread.php?t=102504

Can you please answer this?
3. This is only 1x problem I found using strategy tab VS Chart. Since I have MultiTimeframe strategies, so moving forward, what is the best approach to making sure that backtest algorithm will have similar behavior to live trading?

[NinjaTrader_ChelseaB]

Hello Mark,

If you are specifically looking for what has caused found differences between real-time and backtest, one of those differences might be the time stamps of historical data.
If you know there are differences you might check the prints to see if the data is correct. If the data is different I would recommend syncing the PC Clock and then reloading historical data.

The best way to ensure a backtest will perform as similarly as possible to real-time is to add intra-bar granularity.

[markkm]

That's exactly what I am doing -- by adding more timeframe -- 1 tick -- I want to ensure that my algorithm will run every single tick.

Now, adding 1 tick "intra-bar granularity" will make my strategy MultiTimeFrame strategy (added data), similar to multi timeframe / instrument (my 1 tick added data could be 1440 minutes, then what's the point of running it in 1 minute chart?) .

Again, my question is; This post refers to only 1x problem I found using strategy tab VS Chart. Since I have MultiTimeframe strategies, so moving forward, what is the best approach to making sure that backtest algorithm will have similar behavior to live trading?

[markkm]

Would the 1 tick added "intra-bar granularity" fix the problem posted here ? I don't think so.

I am refering to
'SystemPerformance.AllTrades[SystemPerformance.AllTrades.Count - 1].ProfitCurrency' is always 0.

Please help with my question:

Again, my question is; This post refers to only 1x problem I found using strategy tab VS Chart. Since I have MultiTimeframe strategies, so moving forward, what is the best approach to making sure that backtest algorithm will have similar behavior to live trading?

[markkm]

I might've been not clear enough, but let's say I have added data fo 1, 15, 60 minutes, and 1 tick.
What's the best approach to launch strategies? On a chart? If yes, which timeframe?

On the tab window, how do I make sure that this discrepancy doesn't happen again? It was limited to 1x problem I found, how do I know there would be more? I
Now all I am asking what is the best approach to launch strategies?
I really hope you can help me.

[NinjaTrader_ChelseaB]

Hello markkm,

The IncludeTradeHistoryInBacktest is false by default when a strategy is not added to a chart.

This is mentioned in the help guide.

https://ninjatrader.com/support/helpGuides/nt8/en-us/includetradehistoryinbacktest.htm

I'm not aware of any other settings that default differently when a script is not added to a chart.

However, your question does not appear to be related to this.

what is the best approach to making sure that backtest algorithm will have similar behavior to live trading?

Since your question is not about chart vs strategies tab and instead about real-time vs backtest, my answer is the same.

Intra-bar granularity is what is needed for the real-time results to match backtest results.


Your new inquiry

what is the best approach to launch strategies?

I would say ensure IncludeTradeHistoryInBacktest is true, and add a new instance to the Strategies tab of the Control Center or add a new instance of the strategy to a chart.
Use the amount of historical data you would like. Set the settings how you would like.

[NinjaTrader_ChelseaB]

Hello markkm,

I have confirmed with development that IncludeTradeHistoryInBacktest must be true for the SystemPerformance collection to be populated.

Below is a public link to where this is mentioned in the Notes of the SystemPerformance page of the help guide.

https://ninjatrader.com/support/helpGuides/nt8/en-us/?systemperformance.htm