• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Strategy Anlyzer vs Playback REALTIME Discrepancies

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Strategy Anlyzer vs Playback REALTIME Discrepancies

    I have for weeks been debugging and tracing differences between my results in backtesting and playback (realtime). I only use Limit Orders and let's assume at this stage I'm familiar with defining Fill Resolution by using another time series (1tick). I'm doing the fill resolution programatically since I also use tick replay and you can't have both settings in Strategy Analyzer (SA) concurrently.

    My problem is that there is great inconsistency between Playback in REALTIME vs SA or Playback HISTORICAL; the last two act the same. It doesn't matter which one is better or worse; just looking to get the more realistic and best settings that will generate consistent results good or bad. My conclusion right now is that it is due to the fill criteria (independent of the resolution) utilized on both scenarios.

    When not using Fill On Touch, the Strategy Analyzer will generate more profits then it should. From the image below (1 tick range chart) you can see that on the left there is a winning trade that produced 1tick more profit then it should since the limit order was set at the green target (SEE IMAGE 1). This scenario is with high fill resolution. All losing trades however are stopped on the limit price. This on the other hand will not happen on PlayBack. PlayBack will set the winning trade to where the limit target was originally with or without using Fill On Touch. This criteria seems more realistic.

    The problem is that if I switch to true the Fill on Touch setting. SA will fill profit targets on touch whereas Playback will not fill the profit targets on touch. Playback will still hit the stop loss targets on touch (SEE IMAGE 2 & 3) and apply a stricter criteria to the profit targets.

    It seems playback is giving a Worst Case scenario.Which is not necessarily a bad thing especially if you are trading bigger time frames. When trading on 1 tick sensibility strategies, it makes all the difference in the world.

    If I could in some way get Strategy Analyzer to act the same way as Playback, then I could optimize my strategy, but as is I don't see a way to emulate playback criteria on SA.

    This is what I see in summary:
    • SA, fill on touch ON -> Applies same criteria for profit and loss, but not very realistic (should require price to go 1 tick above/below, but to keep profit or loss at original target)
    • SA, fill on touch OFF -> Hits profit target 1 tick above where it should. Hits stoploss at right place
    • PB, fill on touch OFF/ON -> Applies different criteria for profit targets than stoploss targets.


    If I could get Strategy Analyzer not to give 1 tick away then that would be great.

    If I could get both SA and PB to be strict in criteria (have price pass 1 tick above) but to calculate the profit or loss to the original limit prices than that would be ideal.

    Can you please point me in the right direction?

    Thanks,

    Julio
    Attached Files
    Last edited by jortuzar; 05-10-2017, 10:06 AM.

    #2
    Hello jortuzar,

    May I confirm you have reviewed the 'Discrepancies: Real-Time vs Backtest' in the help guide?

    Do you have intra-bar granularity added to the script?

    Are you ensuring that when testing with the Playback connection that no historical data is loading (as this will change the results) and that the first processed bar in backtest is also the same processed bar in real-time (starting a strategy at different times will change the orders it places)?

    Below is a link to a forum post where I give details about intra-bar granularity.
    http://ninjatrader.com/support/forum...297#post491297

    Having the 'Fill limit orders on touch' enabled causes the limit order to fill at the limit price instead of requiring the price to pass through the order for the order to fill.
    Last edited by NinjaTrader_ChelseaB; 05-10-2017, 10:19 AM.
    Chelsea B.NinjaTrader Customer Service

    Comment


      #3
      Thanks NinjaTrader_ChelseaB.

      May I confirm you have reviewed the 'Discrepancies: Real-Time vs Backtest' in the help guide?Yes I have reviewed.

      Do you have intra-bar granularity added to the script? Yes I do.

      Are you ensuring that when testing with the Playback connection that no historical data is loading (as this will change the results) and that the first processed bar in backtest is also the same processed bar in real-time (starting a strategy at different times will change the orders it places)?
      Yes.

      The point/answer I'm looking for is different... It has to do with the fill criteria more than the fill resolution. I understand with backtesting you only have OHLC info vs tick data en PB realtime.

      For simplicity let's just concentrate on IMAGE 3 (Playback realtime with Filled on Touch ON). Could you please explain why the profit targets don't get hit on touch and the stop loss gets hit on touch?

      Thanks,

      Julio

      Comment


        #4
        Hello Julio,

        I am showing that you have sent this exact same question in an email to platform support.

        So that we are not updating you in multiple places, I will continue working with you through email.

        Moving forward, please either post on the forums, or feel free to send in an email to platform support, but please refrain from sending the same inquiry multiple times as this will require more resources from our staff.

        (Sometimes our staff will direct you on the forums to email support, however, we will continue assisting through email and will no longer update the forum thread)
        Chelsea B.NinjaTrader Customer Service

        Comment


          #5
          Ok sorry about that. Whichever works for me.

          Comment


            #6
            Originally posted by NinjaTrader_ChelseaB View Post
            Hello Julio,

            I am showing that you have sent this exact same question in an email to platform support.

            So that we are not updating you in multiple places, I will continue working with you through email.

            Moving forward, please either post on the forums, or feel free to send in an email to platform support, but please refrain from sending the same inquiry multiple times as this will require more resources from our staff.

            (Sometimes our staff will direct you on the forums to email support, however, we will continue assisting through email and will no longer update the forum thread)
            It would be helpful to resolve these issues where other users can see.

            Comment


              #7
              Hello pen15,

              We have confirmed the logic is to fill the Stop Limit order on touch no matter what. So the logic of the Limit order fills is not applied to the Stop Limit orders. We have put through a feature request for the Stop Limit orders to reflect the Limit Orders fill logic. The internal tracking number for this feature request is SFT-2321. Please reference this internal tracking number if you ever have questions regarding this feature request.
              Chelsea B.NinjaTrader Customer Service

              Comment

              Latest Posts

              Collapse

              Topics Statistics Last Post
              Started by rainbow47, Today, 06:10 AM
              0 responses
              7 views
              0 likes
              Last Post rainbow47  
              Started by stenberg98, Today, 04:53 AM
              2 responses
              7 views
              0 likes
              Last Post NinjaTrader_Marco  
              Started by wukefuha, Today, 03:20 AM
              1 response
              7 views
              0 likes
              Last Post NinjaTrader_Marco  
              Started by mbcmbc, Yesterday, 10:12 PM
              1 response
              10 views
              0 likes
              Last Post NinjaTrader_ChrisSch  
              Started by fx.practic, Today, 03:44 AM
              1 response
              15 views
              0 likes
              Last Post NinjaTrader_Marco  
              Working...
              X