Backtesting Issues

[Ram011558]

Hi,

I am seeing significant differences between Live SIM and backtesting. At times, backtesting is adding 30 percent to my Live SIM daily profit and daily loss based on a single entry and exit on a given day. I am using 2 minute candles to help reduce this (am assuming that this would help). I am aware of slippage but this seems excessive. A couple of questions:

1.) Would going to 1 minute candles help with the gap?

2.) In general, is there something that stands out that I may be doing wrong?

2.) I am using IB as my data feed and I know they do not supply tick historical data. If I were to use a data feed that supports ticks, would the Ninjatrader backtesting use the tick data? If so, would this likely reduce this gap that I am seeing?

Appreciate the help.

[sledge]

. At times, backtesting is adding 30 percent to my Live SIM daily profit and daily loss based on a single entry and exit on a given day.

Is this the exact same trade? Same entries/exits? What's different about them that is causing the 30%?

Anyways, There are many posts here on the subject about differences.

http://ninjatrader.com/support/forum/showthread.php?t=64943&highlight=differences+live+trading+backtesting

Have you tried market replay?

[Ram011558]

Thanks for the response. Yes, exact same strategy for both Live SIM and back testing. I am trading a volatile stock so I am wondering if that might be coming into play.

Any chance on the thoughts for the 3 questions?

[Ram011558]

I have not tried market replay. What are you thinking about with it? I am fairly new to this.....does this store and use tick data for backtesting? Can data going back 2 years be retrieved and stored and used for backtesting? Thanks again.

[NinjaTrader_ZacharyG]

Hello Ram011558,

Please take a look at this help guide post for discrepancies when running a strategy real-time (through a live account, sim data feed, Market Replay, etc.) vs. a backtest: https://ninjatrader.com/support/help...ime_vs_bac.htm

For a backtest to utilize tick data, you will need to add a Tick data series to your strategy. Please take a look at this forum post for information about backtesting NinjaScript strategies with an intrabar granularity: http://ninjatrader.com/support/forum...ead.php?t=6652

Please, let us know if we may be of further assistance.

[Ram011558]

Thank you for the response. If I add this script as you suggest, would this work with Interactive Brokers? It is my understanding that their historical data is not tick by tick.

If I were to add this script and used tick by tick historical data, would this help close the gap that I am seeing?

Thank you.

[NinjaTrader_ZacharyG]

Hello Ram011558,

You will need to have historical tick data available for your strategy to utilize it during a backtest.

Because of the differences between doing a backtest and running your strategy live, there would be no guarantee that the results will match. If your strategy submits orders on a tick by tick basis rather than at the close of a bar when running real-time, then adding an intrabar granularity for backtesting and submitting your orders to the more granular time frame may help.

[Ram011558]

Hi. My strategy runs at close of bar both in real time and backtesting. Hence, the issue. Would backtesting better represent real time if I ran the strategy with tick to tick data for both real time and backtesting? Versus now where both run at close of bar.

[NinjaTrader_PatrickH]

Hello Ram011558,

Thank you for your response.

For the executions if you were to run an additional Tick series in the code to submit orders to the executions would be more similar to those in real-time even if you calculate on the close of the bar in real-time. This is because executions in real-time are intra-bar and in backtesting they are on the close of the bar.

Please take a look at our reference sample at the following link: http://ninjatrader.com/support/forum...ead.php?t=6652

[Ram011558]

Found The Issue

I spent some time on this and it looks like the main issue is the huge spread of the instrument I am working with. So figured that out.

However, I did find something odd when checking back testing numbers against live SIM. This isn't the first time I've seen this. In live SIM, the strategy traded 5 times while in back testing, it entered short at 9:40 AM ish, went long just before 11:00 AM then closed just before 4:00 PM. I reloaded the live SIM chart later in the evening and the new chart now replicated the back testing chart. The newly created chart is attached and shows the extra trades generated through SIM. It is as though the live SIM slow moving average was out of location (too high on the chart) and the crossovers were generating the extra trades. I'm guessing that this isn't the first time that this type of situation has arisen so any ideas?

[NinjaTrader_PatrickH]

Hello Ram011558,

Thank you for your update.

The new item is strange to see. I would think there is a difference between live and historical data. However, such a large difference is very strange. Can you send us over your log and trace files?
You can do this by going to the Control Center-> Help-> Mail to Platform Support.
Please place 'ATTN: Patrick H - 1482129' in the subject line.

[Ram011558]

Follow Up

This has been sent. Thank you for the support.