Market Replay Filled Order At Ask Price That Did Not Exist

[Matheyas5]

Hi NT,

I ran the strategy shown below (full source code is in attached ReplayExample.zip) in Market Replay mode over your ES 03-16 Replay data for 2/17/2016.

Attached 1.jpg shows the settings I had the Simulator on while running the Replay.

Attached 2.jpg shows a chart from the replay. Here is how to read the chart:
Blue Dots= 1 Tick Bid Bars
Red dots= 1 Tick Last Bars
Green Dots= 1 Tick Ask Bars
Area circled in Blue= Shows the exact entry and exit times for the two orders displayed on the chart.

Anyway, you'll see in 2.jpg that the short exit called "TakeProfitExit" was filled at 1884.00, but no Ask bars on the chart ever dropped to 1884.00.

Any idea why this happened?

Thanks you.

         {
        #region Variables
        private double stopLoss = 2; // Default setting for StopLoss
        private double takeProfit = 1; // Default setting for TakeProfit
        #endregion

        protected override void Initialize()
        {	
	      Add("ES 03-16",PeriodType.Tick, 1, MarketDataType.Bid);
	      Add("ES 03-16",PeriodType.Tick, 1, MarketDataType.Ask);
	      Add("ES 03-16",PeriodType.Tick, 1, MarketDataType.Last);
	  
              CalculateOnBarClose = true;
        }

        protected override void OnBarUpdate()
        {
	      if (BarsInProgress == 2)
	      {				
		  if (Position.MarketPosition == MarketPosition.Short)
		  {
		      ExitShortLimit(2,true,1,Position.AvgPrice - TakeProfit * TickSize,
                      "TakeProfitExit","ShortEntry1");
			
                      ExitShortStop(2,true,1,Position.AvgPrice + StopLoss   
                     *TickSize,"StopLossExit","ShortEntry1");
	       	  }		
	      }	

		if (BarsInProgress == 3)
		{
		    if  (Position.MarketPosition == MarketPosition.Flat
			 && CrossBelow(MACD(BarsArray[3], 12, 26, 9).Avg, MACD(BarsArray  
                         [3], 12, 26, 9),1))
	          {		
			EnterShort(1, 1, "ShortEntry1");
	          }
	      }
         }

[sledge]

Hi NT,


Any idea why this happened?

I believe your answer is in #2 of this post:

http://ninjatrader.com/support/forum/showpost.php?p=440579&postcount=5

[Matheyas5]

Thanks sledge.

NT, is what sledge said your official answer to this as well?

What sad news if so

[NinjaTrader_JessicaP]

Hello Matthayas5,

NinjaTrader_Brett is discussing the Tick Replay feature of NinjaTrader 8 in the thread that sledge posted, not Market Replay in NT7. I am attempting to reproduce your results now. I will be able to get back to you tomorrow with my discoveries.

Here is another thread where our PM team discusses this issue in NT7.

http://ninjatrader.com/support/forum/showthread.php?t=50508

In this thread, they mention this,

Hello,

In order to read the historical data values for bid/ask you would have to make a multi series script which added the correct marketDataType
See the 3rd overload here: http://www.ninjatrader.com/support/h....html?add3.htm

For more on multi series scripts: http://www.ninjatrader.com/support/h...nstruments.htm

Quote:
2) Assume I have 3 txt-files: MSFT.Ask.txt, MSFT.Bid.txt, and MSFT.Last.txt, all tick data. I have to import each of those files in order to run the aforementioned indicator, correct?
Correct, you would need to have the historical data on your system to use it. Some of our data providers do provide historical bid/ask data: http://www.ninjatrader.com/support/h...rical_data.htm

Let me know if I can further assist.

It does appear in the code sample that you provided that you are using multiple data series correctly. I would recommend viewing that entire thread, as it pertains directly to the issue at hand.

I appreciate your patience, I will return to this thread when I have more information.

[Matheyas5]

Thanks Jessica.

FYI: quoted below is the part of the link sledge posted where I believe NinjaTrader_Brett mentions Market Replay in NT7. However, I'm not 100% sure if this quote is referring to NT7's Market Replay or NT7's Strategy Analyzer backtest.

Even with NT7 we did not use bid/ask data for the fill of orders.

[Matheyas5]

. . . hey Jessica, i also want to mention that the timestamps circled in blue in 2.jpg of post 1 of this thread are EST.

Thnks again

[Matheyas5]

Hi Jessica. This post (post #7 of this thread) is regarding a backtest accuracy issue in NT7's Strategy Analyzer. It is not regarding a backtest accuracy issue in NT7's Market Replay (like in posts #1-#6 of this thread).
I am including post #7 in this thread (rather making a new thread for it) because it also involves the ReplayExample.zip strategy attached to post #1 and I figure the accuracy issues I'm bringing up in post #7 and posts #1-#6 may all be related.

Anyway, I ran the ReplayExample.zip strategy (from post #1) in NT7's Strategy Analyzer over the historical market data in this link http://bit.ly/22ASuce using the settings in attached 1.jpg

The results show a short entry happened on 2/18/2016 at 1:36:44 AM EST (as shown in attached 2.jpg). However, this entry does not make any sense because the Last price did not change at 1:36:44 AM EST (as shown in attached 3.jpg)

Any idea why this happened?

[NinjaTrader_JessicaP]

Hello again,

I noticed in your 1.JPG that you selected the "Liberal" fill type. This would coincide with your fill happening at the projected time of 44 seconds, as your price trended from 1926.00 at 42 seconds to 1925.75 at 59 seconds. I am going to provide a quote from the help guide page covering discrepancies in real time v backtest data which I believe covers both these cases :

Getting Filled on an Order
•During a backtest you can select conservative or liberal fill algorithms which will produce different results. Fills are determined based on 4 data points, OHLC of a bar since that is the only information that is known during a backtest.
The Fill Price of Orders
•During a backtest assumptions are made on the fill price of an order is based on the OHLC of a bar and the price of the order itself. You can also have differences depending on which fill algorithm you choose.
Running a Strategy at the Close of a Bar or Tick by Tick
•During backtest, strategies can ONLY be processed at the close of each bar

I believe when we put these pieces of information together, we can understand that the strategy analyzer is placing trades at projected, rather than actual, times and price values. The trades would actually be placed at the close of a bar, at which an appropriate point in time during the bar would be selected for the trade to occur.

To prevent this from occurring in the future, I would recommend selecting the "Conservative" fill type when backtesting to restrict fills to actual data points. Please let us know if this does not work, or if there is any other way we can help.

[Matheyas5]

Hi Jessica,

My strategy is coded to fill exit orders on 1 tick Ask bars (not 1 tick Last bars). The "1926.00 at 42 seconds to 1925.75 at 59 seconds" gap you mentioned in your post #8 is regarding a 1 tick Last bars, so I don't thinks its applicable.

You'll see in the Ask tick data I uploaded in post #7 that the ask Price received a tick at 42 seconds, but no new ticks till 44 seconds. After re-thinking things I'm guuessing this is why the fill happened at 1:36:44 AM rather then 1:36:42 AM?

Also, post #8 addresses post #7, but not the question brought up in post 1#-6#.
Any thoughts on posts 1#-6# ?

Thanks

[NinjaTrader_JessicaP]

I have prepared the following script which I will be able to run over known data, which will reveal to us exactly how NinjaTrader places trades at Bid, Ask, and Last prices with historical data. I will return with my findings using this script. I would like to invite you to attempt to use this script with your data as well.

[FONT=Courier New]#region Using declarations
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Data;
using NinjaTrader.Indicator;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Strategy;
#endregion

// This namespace holds all strategies and is required. Do not change it.
namespace NinjaTrader.Strategy
{
    /// <summary>
    /// Places trades at several known data points
    /// </summary>
    [Description("Places trades at several known data points")]
    public class BackTestAnalyzer : Strategy
    {
        #region Variables
        // Wizard generated variables
        private double askPriceToTest = 1000.000; // Default setting for AskPriceToTest
        private double bidPriceToTest = 1000.000; // Default setting for BidPriceToTest
        private double lastPriceToTest = 1000.000; // Default setting for LastPriceToTest
        // User defined variables (add any user defined variables below)
        #endregion

        /// <summary>
        /// This method is used to configure the strategy and is called once before any strategy method is called.
        /// </summary>
        protected override void Initialize()
        {
            CalculateOnBarClose = true;
        }

        /// <summary>
        /// Called on each bar update event (incoming tick)
        /// </summary>
        protected override void OnBarUpdate()
        {
            // Condition set 1
            if (GetCurrentAsk() == AskPriceToTest)
            {
                Log("Placing a trade at ask price", LogLevel.Information);
                EnterLongLimit(DefaultQuantity, AskPriceToTest, "PlacingAsk");
            }

            // Condition set 2
            if (GetCurrentBid() == BidPriceToTest)
            {
                Log("Placing a trade at bid price", LogLevel.Information);[/FONT]
[FONT=Courier New][FONT=Courier New]
                EnterLongLimit(DefaultQuantity, BidPriceToTest, "PlacingBid");[/FONT]             }

            // Condition set 3
            if (Close[0] == LastPriceToTest)
            {
                Log("Placing trade at last price", LogLevel.Information);[/FONT][FONT=Courier New][FONT=Courier New]
                EnterLongLimit(DefaultQuantity, LastPriceToTest, "PlacingLast");[/FONT]
            }
        }

        #region Properties
        [Description("Trades will be placed when the ask price is at this amount")]
        [GridCategory("Parameters")]
        public double AskPriceToTest
        {
            get { return askPriceToTest; }
            set { askPriceToTest = Math.Max(1, value); }
        }

        [Description("Trades will be placed when the bid price is at this amount")]
        [GridCategory("Parameters")]
        public double BidPriceToTest
        {
            get { return bidPriceToTest; }
            set { bidPriceToTest = Math.Max(1, value); }
        }

        [Description("Trades will be placed when the last price is at this amount")]
        [GridCategory("Parameters")]
        public double LastPriceToTest
        {
            get { return lastPriceToTest; }
            set { lastPriceToTest = Math.Max(1, value); }
        }
        #endregion
    }
}

[/FONT]

[Matheyas5]

Cool.

Also, please ignore post #9.
Post #9 is talking about Ask Tick data in regards to a Short Entry, which I now realize makes no sense because Short Entries are based on Bid Tick data.

Apologies for the mistake. And again, please ignore post #9.

[NinjaTrader_JessicaP]

As a test environment, I have prepared three files for the E-Mini S&P Historical's data, for April 15th, modified as follows :

• Ask :
  • Every price at 12:00:00 is 1000.00
• Bid :
  • Every price at 12:59:59 is 999.50
  • Every price at 13:00:01 is 1000:50
  • Half the prices at 13:00:00 are 999.50 and half are 1000.50
• Last :
  • Every price at 13:59:59 is 999.50
  • Every price at 14:00:00 is 1000.00
  • Every price at 14:00:01 is 1001.00

This is all fake data and does not reflect actual market conditions.

[NinjaTrader_JessicaP]

My results :

At 1000, 1000, 1000 : No trades, no log messages, Graph reads 1000.5 at noon, 1 PM, and 2 PM

At 1000.5, 1000.5, 1000.5 : No trades, these log messages, Graph reads 1000.5 at noon, 1 PM, and 2 PM

4/20/2016 1:39:11 PM|1|4|Placing a trade at ask price
4/20/2016 1:39:11 PM|1|4|Placing a trade at bid price
4/20/2016 1:39:11 PM|1|4|Placing trade at last price

I tried several settings, such as modifying liberal <-> default, minimum bars required, tick <-> minute, CABC = true or false, etc, and received the same results.

Conclusions : NinjaTrader uses a single price point for Ask, Bid, and Last prices with historical data, that does not necessarily reflect actual tick values or times.

[Matheyas5]

Hi Jessica,
Can you please elaborate a little more on what you mean by this quote?

Conclusions : NinjaTrader uses a single price point for Ask, Bid, and Last prices with historical data, that does not necessarily reflect actual tick values or times.

Thanks

[NinjaTrader_JessicaP]

I would be happy to.

So with one of the sets of data, all of the prices were 1000 exactly. With another set of data, none of the prices were at 1000, and with a third, the majority of the prices were 1000.5 with 999.5 and 1000.5 . In the strategy analyzer, on the other hand, on the chart, all of these prices were treated as if they were 1000.5 . None of the data sets overlapped in time.

However, since log messages could only show up in code when an ask, bid, or last price was exactly at the level of the user defined input, and all three log messages showed up at 1000.5 , we can conclude that the Ask, Bid, and Last prices visible to the strategy were all 1000.5 , and thus prices that NinjaTrader calculated.