Strategy Anlyzer vs Playback REALTIME Discrepancies

[jortuzar]

I have for weeks been debugging and tracing differences between my results in backtesting and playback (realtime). I only use Limit Orders and let's assume at this stage I'm familiar with defining Fill Resolution by using another time series (1tick). I'm doing the fill resolution programatically since I also use tick replay and you can't have both settings in Strategy Analyzer (SA) concurrently.

My problem is that there is great inconsistency between Playback in REALTIME vs SA or Playback HISTORICAL; the last two act the same. It doesn't matter which one is better or worse; just looking to get the more realistic and best settings that will generate consistent results good or bad. My conclusion right now is that it is due to the fill criteria (independent of the resolution) utilized on both scenarios.

When not using Fill On Touch, the Strategy Analyzer will generate more profits then it should. From the image below (1 tick range chart) you can see that on the left there is a winning trade that produced 1tick more profit then it should since the limit order was set at the green target (SEE IMAGE 1). This scenario is with high fill resolution. All losing trades however are stopped on the limit price. This on the other hand will not happen on PlayBack. PlayBack will set the winning trade to where the limit target was originally with or without using Fill On Touch. This criteria seems more realistic.

The problem is that if I switch to true the Fill on Touch setting. SA will fill profit targets on touch whereas Playback will not fill the profit targets on touch. Playback will still hit the stop loss targets on touch (SEE IMAGE 2 & 3) and apply a stricter criteria to the profit targets.

It seems playback is giving a Worst Case scenario.Which is not necessarily a bad thing especially if you are trading bigger time frames. When trading on 1 tick sensibility strategies, it makes all the difference in the world.

If I could in some way get Strategy Analyzer to act the same way as Playback, then I could optimize my strategy, but as is I don't see a way to emulate playback criteria on SA.

This is what I see in summary:

• SA, fill on touch ON -> Applies same criteria for profit and loss, but not very realistic (should require price to go 1 tick above/below, but to keep profit or loss at original target)
• SA, fill on touch OFF -> Hits profit target 1 tick above where it should. Hits stoploss at right place
• PB, fill on touch OFF/ON -> Applies different criteria for profit targets than stoploss targets.

If I could get Strategy Analyzer not to give 1 tick away then that would be great.

If I could get both SA and PB to be strict in criteria (have price pass 1 tick above) but to calculate the profit or loss to the original limit prices than that would be ideal.

Can you please point me in the right direction?

Thanks,

Julio

[NinjaTrader_ChelseaB]

Hello jortuzar,

May I confirm you have reviewed the 'Discrepancies: Real-Time vs Backtest' in the help guide?

Do you have intra-bar granularity added to the script?

Are you ensuring that when testing with the Playback connection that no historical data is loading (as this will change the results) and that the first processed bar in backtest is also the same processed bar in real-time (starting a strategy at different times will change the orders it places)?

Below is a link to a forum post where I give details about intra-bar granularity.

http://ninjatrader.com/support/forum/showthread.php?p=491297#post491297

Having the 'Fill limit orders on touch' enabled causes the limit order to fill at the limit price instead of requiring the price to pass through the order for the order to fill.

[jortuzar]

Thanks NinjaTrader_ChelseaB.

May I confirm you have reviewed the 'Discrepancies: Real-Time vs Backtest' in the help guide?Yes I have reviewed.

Do you have intra-bar granularity added to the script? Yes I do.

Are you ensuring that when testing with the Playback connection that no historical data is loading (as this will change the results) and that the first processed bar in backtest is also the same processed bar in real-time (starting a strategy at different times will change the orders it places)?
Yes.

The point/answer I'm looking for is different... It has to do with the fill criteria more than the fill resolution. I understand with backtesting you only have OHLC info vs tick data en PB realtime.

For simplicity let's just concentrate on IMAGE 3 (Playback realtime with Filled on Touch ON). Could you please explain why the profit targets don't get hit on touch and the stop loss gets hit on touch?

Thanks,

Julio

[NinjaTrader_ChelseaB]

Hello Julio,

I am showing that you have sent this exact same question in an email to platform support.

So that we are not updating you in multiple places, I will continue working with you through email.

Moving forward, please either post on the forums, or feel free to send in an email to platform support, but please refrain from sending the same inquiry multiple times as this will require more resources from our staff.

(Sometimes our staff will direct you on the forums to email support, however, we will continue assisting through email and will no longer update the forum thread)

[jortuzar]

Ok sorry about that. Whichever works for me.

[pen15]

Hello Julio,

I am showing that you have sent this exact same question in an email to platform support.

So that we are not updating you in multiple places, I will continue working with you through email.

Moving forward, please either post on the forums, or feel free to send in an email to platform support, but please refrain from sending the same inquiry multiple times as this will require more resources from our staff.

(Sometimes our staff will direct you on the forums to email support, however, we will continue assisting through email and will no longer update the forum thread)

It would be helpful to resolve these issues where other users can see.

[NinjaTrader_ChelseaB]

Hello pen15,

We have confirmed the logic is to fill the Stop Limit order on touch no matter what. So the logic of the Limit order fills is not applied to the Stop Limit orders. We have put through a feature request for the Stop Limit orders to reflect the Limit Orders fill logic. The internal tracking number for this feature request is SFT-2321. Please reference this internal tracking number if you ever have questions regarding this feature request.