Attention: Forex bid-last discrepancy

[pstrusi]

After studying discrepancy in backtesting results, I've realized Forex tick data, for Bid and Last are not the same when it should. I went to the database for a close look, and basically this is what I've found:

For every trading price, Last ticks outnumber Bid ticks always by one tick during that "level" of trading price.

This discrepancy of one tick shouldn't make any difference in the 98% of Algos, but unfortunately my Script is always evaluating Bid and Last behavior for calculations, and this obviously create a distortion cause difference in quantities. So now I wonder what's is really the genuine trading action, Last or Bid ? My logic says Last, but still I'd like to know why this. I'm pretty sure that has to do with some kind of database's internal rule registering data, where probably it states that Last data should be at least one tick more than Bid in order to reflect an execution or trade.

It's not really an issue, but there's something to have in Mind when working with Algos at this level. It seems easily addressed changing the internal logic of the Algo, cause that directive in Forex that Bid data is exactly equal to Last data is not strictly true.


Thanks

[reach4thelasers]

As well as this, the data that is available on "NinjaTrader Historical Servers" is not the same as "FXCM Historical Servers". I have been in touch with platform support about this. I don't know whether its related to bid/last confusions or what, but With FXCM's data, the close of one candle is almost always the open of the next. Ninja's historical data does not show that, and in some cases there is up to a one-pip gap between opens and closes on Forex EURUSD during the London Trading session.

Thats the most liquid financial instrument at the world on the most liquid market and the most liquid time of day. The European session accounts for 40% of total trading volume on Forex - about $1.5 Trillion - a one pip gap - or any gap at that time of day on that instrument is unfathomable.

You can see this in the attached screen shots, I've drawn lines on the candles where they don't match up. And also pointed out a candle structure for comparison between Tradingstation and Ninjatrader.

[pstrusi]

As well as this, the data that is available on "..... With FXCM's data, the close of one candle is almost always the open of the next. Ninja's historical data does not show that, .....

That situation might depend on the type of data chat. For example in a Range Bar chart the open of the new bar will be always up or down regarding to Last close bar. So it's important to know exactly every info detail of the environment that you're working on.

[reach4thelasers]

Well that's the most obvious thing that has ever been stated. Obviously if I'm doing a comparison between Ninja charts and FXCMs I'm going to look at the same chart - do you think i'm comparing a 3 minute chart to a range chart?

[pstrusi]

I'm sorry that I'haven't seen closely the charts, otherwise I have not had that comment.
It seems that the difference might come from some different internal protocols, or even the data quantity flow registered in platforms.

[NinjaTrader_ZacharyG]

Hello pstrusi,

Thank you for your inquiry.

With forex, you will need to focus on the Bid price. Forex does not have a true Last price.

Unfortunately with backtesting on forex, only the Bid price is used for all executions done by the strategy, which would result in inaccurate results as, in reality, all sells are filled at the Bid and all buys are filled at the Ask.

[pstrusi]

Hello pstrusi,

Thank you for your inquiry.

With forex, you will need to focus on the Bid price. Forex does not have a true Last price.

Unfortunately with backtesting on forex, only the Bid price is used for all executions done by the strategy, which would result in inaccurate results as, in reality, all sells are filled at the Bid and all buys are filled at the Ask.

Thanks for the answer Zachary, which now makes me wonder another questions :

1. When you need to focus on the Bid price, is it for Algos's internal calculation or for execution purposes?

2. Do you mean that the genuine market action is what Bid does, either realtime or backtesting?

3. If Bid is the real variable to watch, why doesn't NT have the same data for the Last? Why is Last always one tick more?

Looking forward
Thanks

[NinjaTrader_ZacharyG]

Hello pstrusi,

1. The Last and Bid prices are equivalent to each other in the platform. Forex pairs do not have a true Last price. NinjaTrader assigns the Last price as the Bid price.
2. Yes, you will want to be watching the Bid price.
3. I am not clear about what you mean here. As stated, forex pairs do not have a true Last price and therefore the Last would not be something to use. There should be no difference between Last and Bid. Can you provide a screenshot of what you are witnessing?

[pstrusi]

Thanks for your answers Zak.
What you said about the Bid as the critical variable is all what I needed to know.

I appreciate your help

[reach4thelasers]

I've been told that the Last price as displayed in Ninjatrader is supposed to be the same as the bid price - but I agree there is discrepancies.

[NinjaTrader_ZacharyG]

There will be discrepancies, yes, between backtesting and live trading forex as only the Bid price is used for all executions done by the strategy, which would result in inaccurate results. In reality, all sells are filled at the Bid and all buys are filled at the Ask.

[pstrusi]

Given the confusion , I decided for a closer view on tick data and what it seems to me is a database directive that whenever there's a new price level ( up or down ), the very first tick of that bar, is registered twice. I imagine that this behaviour could be a smart move from developers for a trading or backtesting purposes, under CalculateOnBarClose=True and taking action at the very same price. For the vast majority of Strategies like ours working as low-latency scripts, this should have no impact whatsoever.

One consideration: please let's us know if any change on database directives that might mean different behaviors in Algos.

[NinjaTrader_ZacharyG]

Hello pstrusi,

I have not seen the first tick of a bar registering twice before.

Can you provide a script that demonstrates this behavior through Print statements?

[pstrusi]

Never mind Zach, maybe it's just an IB's thing, nothing to worry about.

Thanks