Historical Bid / Ask values don't seem right

[daqu40]

Hello Forum. I've already seen several threads dealing with issues of getting accurate Bid and Ask data for use in Backtest/Optimize of an NT8 strategy.

This one (https://ninjatrader.com/support/foru...ad.php?t=93168) in particular was very helpful.

My observation in using a 1 Tick data series is the Bid/Ask values this data series is providing are often too many ticks apart to seem real. I'm testing against CL 03-18 and often find the delta between the bid and ask at over 10 ticks. My strategy is also running in TickReplay as I have an indicator that requires it.

I can understand this happening during the Oil Inventories announcement, but I'm seeing this at times of the day where there is no news related reason for a jump in volatility and this big of a gap.

Does anyone have an explanation for what I'm seeing?

Thanks!

[NinjaTrader_AlanP]

Hello daqu40,

Thank you for your note.

Have you downloaded market replay data during the time in question, and replayed the data with a superdom open to confirm whether its a data issue or a coding issue? Here is a video on Market Replay,

https://www.youtube.com/watch?v=wvoc16oJhk8

If its not a data issue as confirmed by the test above, would you be able to provide a sample script which demonstrates this discrepancy between the bid and answer during a backtest?

Would you also please include the time and date for which you are seeing such wide/bid and ask?

I look forward to your reply.

[daqu40]

Thanks for the reply Alan,

Attached is an image of the output of a strategy showing a sizable gap between the bid and ask and a DOM showing no such gap.

The strategy I used to generate this is also attached.

Thanks!

[NinjaTrader_Jim]

Hello daqu40,

I am responding on behalf of Alan who is out of the office today.

Tick Replay uses the best bid and the best ask for a given bar index and would not have every bid and ask as you would see with Realtime or Market Replay. Tick Replay also cannot be used with MarketDataType.Bid and MarketDataTyple.Ask data series. Since we are not seeing every bid and every ask like we would on the DOM we could expect to see some gaps if we were to use OnMarketData() with Tick Replay.

I may suggest to add the data series for the Bid and Ask and use OnBarUpdate() to retrieve these values as opposed to OnMarketData().

Using Historical Bid/Ask Series - https://ninjatrader.com/support/help..._ask_serie.htm

Please let me know if I can be of further assistance.

[daqu40]

Thanks for the reply Jim,

Doesn't adding the Bid and Ask data series conflict with TickReplay?

I have an indicator that requires TickReplay.

[NinjaTrader_Jim]

Yes, this is correct. In such a case I would recommend applying the NinjaScripts that require Tick Replay on another chart than indicators that would use added Bid/Ask series.

If this is part of your strategy to use both of these, it would be possible to create an AddOn that has static variables along with methods for the strategies and indicator to read/write from.

NinjaTrader_Chelsea has an example here: https://ninjatrader.com/support/foru...104#post457104

I'm happy to be of any additional help.

[daqu40]

I'll look at Chelsea's and see if that will meet my needs and post back here when I'm sure I can meet all the requirements of the strategy I'm trying to build.

[daqu40]

Hi Jim/Alan,

I've looked at Chelsea's example. Nice work! Unfortunately, I don't think this will help with the strategy I'm building. Chelsea's architecture will help with the case of running in a live market with the indicator on a separate chart in TickReplay mode and the strategy not on TickReplay.

However, I need to be able to run in a live market and use the Strategy Analyzer to optimize the parameter set.

I don't see how this helps with backtesting since I would need to pass a TickReplay data series to the indicator. From other research, I'm not seeing how to have a strategy running on a non-TickReplay data series and then reflect an indicator to use internally that is on TickReplay.

Or even another option, how could I add a 1 tick dataseries to the strategy that isn't TickReplay?

Thanks

[NinjaTrader_Jim]

Hello daqu40,

Thanks for reaching back.

I do not have any workarounds to overcome the Bid/Ask limitation with TickReplay so your strategy can be optimized in the Strategy Analyzer. We would need to submit a feature request to expand TickReplay's functionality.

If it is possible, I would recommend avoiding TickReplay when creating a strategy that uses a Bid and Ask series so you can effectively backtest and optimize it. You could use Market Replay to test the strategy's performance after devising a workaround based on the above example, but this would be far less convenient to optimize.

I may suggest recording the indicator's values ahead of time so it can be read from the strategy in the backtest instead of calculating them with TickReplay. You could use code to detect that the indicator is added to a Tick Replay data series, and if it is not, it can read values previous plot values from a file.

IsTickReplay - https://ninjatrader.com/support/help...tickreplay.htm

I'll be happy to submit a feature request to expand TickReplay on your behalf. If that is the case, please let me know.

[daqu40]

Thanks for the quick follow up Jim.

Please submit the enhancement request. I'm busy trying to figure out a workaround.

The real issue is trying to get Backtest/Optimize order fills to more accurately reflect what comes out of Replay testing.

The documentation is quite clear that TickReplay is not meant for improving fill accuracy so I keep working to find a way to reconcile the two.

The strategy I'm building currently has near perfect alignment between Backtest and Replay in terms of when a trade opportunity is recognized and entered. Half the battle is won there. However, with the fills only matching about 25% of the time there is no way to really tell how well a set of parameters will perform in Replay.

Looking at Bid and Ask values and forcing the entry to those prices improved the match between Backtest and Replay, but it's still well under 50%.

Any further wisdom you have to pass along for achieving my goal with an indicator that requires TickReplay will be very appreciated.

[NinjaTrader_Jim]

Hello daqu40,

The feature request has been submitted. I'll update this thread with a ticket ID when it becomes available.

Ticket ID: SFT-1531

While you have probably been over these items to bring historical backtests closer to Market Replay and Realtime results, I will include a link for an indepth set of videos Chelsea has also put together to better approach these discrepancies.

As another rule of thumb, if you run the backtest to depend more on slippage, then you can expect the strategy to get better results with Market Replay and live data. It can sometimes be more acceptable to see better results than matching results when comparing performance.

Comparing Historical and Backtest - https://ninjatrader.com/support/foru...d.php?t=102504

If there is anything else we can do to help, please let us know.