Discrepancies: Real-Time vs Backtest

<< Click to Display Table of Contents >>

Navigation:  Operations > Strategy Analyzer >

Discrepancies: Real-Time vs Backtest

Previous page Return to chapter overview Next page

You should expect that a strategy running real-time (live brokerage account, live market simulation, Playback connection etc...) will produce different results than the performance results generated during a backtest. This difference may be more easily seen on certain Bars types (e.g. Point and Figure) than others due to their inherent nature in bar formation.


Getting Filled on an Order

Fills are determined based on 4 data points, OHLC of a bar since that is the only information that is known during a backtest.
During simulation using real-time live market data or Playback, the fill algorithm is dynamic in that it uses incoming market data (both price and volume) to determine if an order should be filled or not.
During real-time live brokerage trading, orders are filled according to market dynamics.


As you can see, there are three distinctly different models for how and when an order may be filled. This is why you may see orders NOT fill in real-time that you may otherwise expect to see filled based on your backtesting results.


The Fill Price of Orders

During a backtest assumptions are made on the fill price of an order is based on the OHLC of a bar and the price of the order itself. You can also have differences depending on which fill algorithm you choose.
During simulation using real-time market data or Playback, the fill price is based on incoming market data and volume, you may receive better or worse fill prices dependant on where the bid or ask price is and what volume is available at this market prices.
During real-time brokerage trading, orders are filled according to market dynamics.


As you can see, there are three different models on what price an order can be filled at.


Running a Strategy at the Close of a Bar or Tick by Tick

During backtest, strategies can ONLY be processed at the close of each bar.
During real-time operation, you have a choice to run a strategy tick by tick (Calculate set to 'On Each Tick') which can produce different results. This is because you can have a signal that executes an order at the close of a bar but when running tick by tick, while in a bar a signal condition can be true although its false at the close of the same bar.


Differences in chart data

If you run a strategy in real-time on DAY1 and then DAY2, you are now backtesting your strategy on DAY1 data instead of processing like it did in real-time so there could be differences. You should understand how chart bars are built.
If using tick based charts, all it takes is a single tick difference between real-time and historical data to generate completely different looking charts. This in turn would impact the calculations of your strategy should the data sets be different.