• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Backtesting with playback at various speeds

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Backtesting with playback at various speeds

    Hello,
    I have the same issue reported at https://ninjatrader.com/support/foru...d.php?t=106818
    namely, I get different orders depending on the playback speed.
    There is no solution posted there.

    To exemplify the issue, attached are results for 1x, 10x and Max speed, respectively.
    The strategy uses Calculate.OnEachTick
    I am using NT 8.0.14.1 64-bit

    What should I do to avoid this discrepancy? Thank you
    Attached Files

    #2
    Hello utilizator,

    Thank you for your post. Could you possible post simple sample with instructions on how to reproduce this behavior?

    I look forward to your reply.
    Eric B.NinjaTrader Customer Service

    Comment


      #3
      Hi Eric,
      Unfortunately, I cannot provide the code for my custom strategy and indicators. I tried to reproduce the issue by using the NT8 sample strategies, but none produced a single order, I don't know to properly launch them, apparently.

      I'm not sure if the below steps are relevant or not without the actual strategy to run, thanks

      Control Center > New Chart > Data Series: End Date =May25, 2018; Day to Load=1
      Control Center > Connections > Playback >Market Replay; Start=May 25, 2018; End=May26, 2018

      Playback: GoTo May25,2018 9:30AM
      Playback: set 1x speed
      Chart>Strategies>MyCustomStrategy
      This will generate the 1st image I attached

      Playback: GoTo May25,2018 9:30AM
      Playback: set 10x speed
      Chart>Strategies>MyCustomStrategy
      This will generate the 2nd image I attached

      Playback: GoTo May25,2018 4:05PM
      Playback: set MAX speed
      Chart>Strategies>MyCustomStrategy
      This will generate the 3rd image I attached

      Playback: GoTo May25,2018 4:05PM
      Chart>Strategies>MyCustomStrategy
      This will generate the 3rd image as well

      Comment


        #4
        Hello utilizator,

        We will be happy to assist you further, although what you are describing would need to be broken down to a root cause so we could give further input or report the matter further. Without the code, we could not give clear advise on how to avoid what you are encountering.

        The detail you provided is helpful, but we would need to have a strategy to run against and we would also need to know which instrument, bar type, and period to use so we can generate the same trades and behavior you are witnessing. If you cannot reproduce with the SampleMACrossover strategy that comes with NinjaTrader, we can move forward with a reduced and modified version of your strategy or a simple test strategy that can reproduce.

        You may also reach out to me at platformsupport[at]ninjatrader[dot]com if your reduced sample contains some code that you would like to keep off the public forum. Please include the text "Attn Jim" and a link to this thread if that is the case.

        I'm happy to be of any further help with this matter.
        JimNinjaTrader Customer Service

        Comment


          #5
          Hi Jim,
          I managed to somehow reproduce the issue with the Sample multi-timeframe strategy.
          Although there was no difference in 'realtime' orders when Playback at 1x, 10x and MAX speed (as exhibited by my custom strategy),
          the orders are different from the 'historical' ones.
          By 'historical' I mean that:
          * Playback > GoTo 16:00
          * Chart > Strategy>Sample multi-timeframe; CalculateOnEachTick
          * orders generated up to the date on the Playback controller ( in this case 16:00)

          Please see attached images.
          I attached the tick data as well, for June 07, 2018; instrument ES 06-18, thanks
          Attached Files

          Comment


            #6
            Hello utilizator,

            Thanks for the clear and detailed information.

            There are going to be differences in order fills and strategy performance when comparing historical and realtime trades since historically; the data series that submits the order will be used to simulate the order fill instead of the realtime data. However, the difference that you are seeing may not have to do with Historical Fill Processing, but may have to do with the strategy using different Calculate modes. Historically, Calculate.OnBarClose will be forced unless Tick Replay is enabled on that chart. This would mean that when using Calculate.OnEachTick or OnPriceChange, OnEachTick/OnPriceChange is only used with "realtime" streamed data.

            For the thread's reference, I have included links to publicly available documentation for for the items mentioned above.

            Discrepancies: Realtime vs. Backtest - https://ninjatrader.com/support/help...ime_vs_bac.htm

            Developing for Tick Replay - https://ninjatrader.com/support/help...ick_replay.htm

            Understanding Historical Fill Processing - https://ninjatrader.com/support/help...ical_fill_.htm

            Please let me know if I can assist you further.
            Last edited by NinjaTrader_Jim; 06-15-2018, 11:36 AM.
            JimNinjaTrader Customer Service

            Comment


              #7
              Hi Jim,
              Historically, Calculate.OnBarClose will be forced unless Tick Replay is enabled on that chart
              .
              But I did enable Tick Replay, as can be seen from my latest image attached (the Properties one). Is there anything else to be set in order for Historical to match Realtime in backtesting? thanks

              Comment


                #8
                Hello utilizator,

                I see that you are using Market Replay data, but I do not see any Data Series properties showing Tick Replay being enabled. The multiple submissions would also be limited by the EntriesPerDirection property, so we may not be looking at multiple submissions at all.

                Digging a little further, we can see that the SampleMultiTimeFrame strategy is submitting orders with a quantity of 1000, and we see execution markers for smaller quantities of that order. What we are looking at with this strategy are partial fills seen when data is streamed into the strategy (realtime) vs. simply filling the orders with the full quantity as seen with historical processing. I've edited my previous message to be clearer.

                There isn't an option to avoid partial fills in the simulator since the result of partial fills would be closer to what would be expected with real markets.

                I'd be sure to double check your strategy to be certain that the executions total the quantity you are submitting. This would be OK. If there are any other discrepancies involving multiple trades being submitted within the same bar to consider the items in my previous reply, or to confirm that there is not an additional data series that is allowing the strategy to submit multiple orders.

                Please let me know if you have any questions.
                Last edited by NinjaTrader_Jim; 06-15-2018, 11:38 AM.
                JimNinjaTrader Customer Service

                Comment


                  #9
                  Hi Jim,
                  If I correctly understand, only 1-tick bar charts will generate the same orders in both backtesting mode and realtime/playback mode. Is this correct?

                  Comment


                    #10
                    Hello utilizator,

                    Orders generated should be consistent between backtesting/sim trading/live trading so long as they are only based on price data. If the strategy uses strategy performance information we may see some differences in orders generated because of order fill differences.

                    Order fills will show differences when testing against a series other than a single tick series because the bar is used to simulate the order fill, not the tick data. This is where High Order Fill Resolution or adding a 1 tick data series to submit orders to will come into play. This feature or submitting orders to a single tick data series will use another data series to simulate the order fills, thus making them closer to what we would see in real markets.

                    More information for using High Order Fill Resolution can be found below. I've also included an example for Backtesting with intrabar granularity that can be used with Multi Series strategies.

                    Order Fill Resolution - https://ninjatrader.com/support/help...FillResolution

                    Backtesting with intrabar granularity - https://ninjatrader.com/support/help...ipt_strate.htm

                    Finally, my colleague Chelsea has a great resource breaking down discrepancies between historical and realtime data, and what can be done about it.

                    Comparing Historical Realtime and Replay Performance - https://ninjatrader.com/support/foru...d.php?t=102504

                    I look forward to being of any further assistance.
                    JimNinjaTrader Customer Service

                    Comment

                    Latest Posts

                    Collapse

                    Topics Statistics Last Post
                    Started by RedDuke, Today, 11:40 AM
                    0 responses
                    5 views
                    0 likes
                    Last Post RedDuke
                    by RedDuke
                     
                    Started by snbence, Today, 11:19 AM
                    2 responses
                    7 views
                    0 likes
                    Last Post snbence
                    by snbence
                     
                    Started by marwanco, Today, 10:54 AM
                    1 response
                    5 views
                    0 likes
                    Last Post NinjaTrader_ChrisL  
                    Started by daqu40, Today, 11:01 AM
                    0 responses
                    1 view
                    0 likes
                    Last Post daqu40
                    by daqu40
                     
                    Started by n8dogg59, Today, 09:55 AM
                    4 responses
                    7 views
                    0 likes
                    Last Post n8dogg59  
                    Working...
                    X