Entry/Exit price discrepancy

[joesandyego]

I noticed an anomaly in Strategy Analyzer that I am sure has a simple explanation that is eluding me....

I have 2 computers (ComputerA and ComputerB). Each pulls data from IQFeed, When I run the same strategy with the same parameters on each computer in Strategy Analyzer, I get different results. The difference is related to the fact that although both systems show trades at the exact same time/bar, the entry and exit prices are different for most trades. I am trading on 1 minute bars. When I looked at the Historical Data in the Historical Data Manager, both computers show the exact same data. So they should show the exact same entry/exit prices on the same bars.

ComputerB is clearly showing entry and exits on bar open prices. I'm not sure what ComputerA is using/showing. It does not appear to be Open, High, Low or Close Could this be a rounding error on ComputerA since there are splits involved? I'm not sure why one computer would have a rounding error and the other not. Slippage is set to 0 for both.

Ideas appreciated.

,,,,,,,,,
Strategy Analyzer,Instrument,Quantity,Entry price,Exit price,Entry time,Exit time,Profit,Cum. profit,Bars
Computer A,VXX,2600,378.88,432.64,2/7/2012 11:31 AM,2/10/2012 9:31 AM,-0.141918285,-0.141918285,1051
Computer B,VXX,2600,379.2,432.48,2/7/2012 11:31 AM,2/10/2012 9:31 AM,-0.140506329,-0.140506329,1051
,,,,,,,,,
Historical Data,Open,High,Low,Close,,,,,
Computer A,379.2,379.36,378.88,379.2,2/7/2012 11:31 AM,,,,
Computer A,432.48,433.76,432.32,433.28,2/10/2012 9:31 AM,,,,
,,,,,,,,,
Computer B,379.2,379.36,378.88,379.2,2/7/2012 11:31 AM,,,,
Computer B,432.48,433.76,432.32,433.28,2/10/2012 9:31 AM,,,,

[NinjaTrader_PatrickH]

Hello joesandyego,

Thank you for your post.

Please compare the following settings in the Strategy Analyzer on both PCs:

• Price Based On
• Fill Type
• Slippage

[joesandyego]

Hi Patrick,
Both systems have identical settings for these parameters. However, I have discovered something curious.

1) Both systems have identical historical data
2) ComputerA was connected to IQFeed when I ran Strategy Analyzer.
3) I have both computers set to "Get data from server", which means that ComputerA was pulling data directly from IQFeed when it ran Strategy Analyzer (right?)

This appears to be the difference in setup between the two systems.

The obvious question is the following: Both of these computers get data from IQFeed. How is it that the data that is saved as historical is different than what is pulled of of the IQFeed server in real-time? When I reloaded the historical data on ComputerA from IQFeed then the historical data was updated and was different that the previous historical data and matched the IQFeed server data. The timeframe that I am looking at is before any data could have been saved in realtime (e.g. I was not trading or looking at this symbol back then).

[NinjaTrader_PatrickH]

Hello joesandyego,

Thank you for your response.

Historical data providers can update the data on their servers that is available to download. You can try disconnecting and running the test on both PCs if the historical data matches.

[joesandyego]

Thank you.

The sorts of changes to data that I see have material impact on backtesting. Is there a data service that is known to not back-adjust data and get it right the first time? I'm only trading equities.

[joesandyego]

I noticed a possible source of some of the anomalies that I am seeing. It seems to be reproduceable as follows:

1) System: IQFeed, NT 7.0.1000.20 64bit, Windows 7 Ultimate. IQFeed connnected
2) Strategy Analyzer: ^VIX, VXX based strategy. Nothing else open/running in NT
3) Set timeframe of strategy range: 1/1/08 - 1/27/14. Note that this is before VXX existed
4) Run strategy (I am assuming that it can be any strategy with these symbols)
5) Observe Cum Profit
6) Use Historical Data Manager and Reload historical data
7) Disconnect IQFeed
8) Run strategy again. In my case the results are different.

If I repeat the above with a timeframe start date in which the data actually exists, such as 1/1/10 - 1/27/14, then the results are identical between IQFeed connected and the Reloaded historical data with IQFeed disconnected.

This makes me believe that NT may not be handling my request to run a strategy or reload data before the symbol existed gracefully. Why this is important is that if I were to have reloaded my data with a start date that was too early and not realized it, then all subsequent back testing using that historical data may be wrong - even if my back testing started with a proper date.

[NinjaTrader_PatrickH]

Hello joesandyego,

Thank you for your response.

It would not be recommended to run a strategy on a time period where the instrument did not exist. However, I notice differences when testing while connected and while not connected. This is the difference between the historical data already stored in your database versus requesting that data again and then running the strategy. There is no guarantee the data will be the same from the data providers server as on your PC (downloaded from earlier).

[joesandyego]

Thanks Patrick. It was not really the answer that I was hoping to hear since it really means that I must continually re-backtest my strategies to make sure that there was not erroneous data from my provider. It's the age old "clean data" problem.

I am sure that NT cannot answer this question and so I put it out to the members: Is there a data provider that is better than the others at providing historical data right the first time and not changing historical data randomly?