• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

MarketReplay versus Strategy Analyzer Result Discrepancy

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    MarketReplay versus Strategy Analyzer Result Discrepancy

    Recently, I have been testing multiple strategies in NT8. The strategy is a neural network based strategy. In this example, I am running it for CL 07-17. I have attached the strategy and accompanying indicator it references. I have XLS files from the actual trades but am unable to upload them here.

    My results are mixed. In strategy analyzer, using optimization, I receive these results:


    When I run this strategy for the period in MarketReplay, the fills are completely different (lag behind my almost a bar it seems) and my P/L is mixed.



    Could someone take a look at my code and see if there's a glaring reason why the results are so different?

    Thanks
    Attached Files

    #2
    Here's a list of trades:

    [Strategy Analyzer - Optimized]



    [Market Replay - Based on SA Optimization]

    Comment


      #3
      Hello rdavido,

      Historical data and real-time data (Market Replay) are fundamentally different.

      Please review the following:
      http://ninjatrader.com/support/forum...297#post491297

      May I ensure that intra-bar granularity has been added?

      May I ensure that no historical data is being included with the Playback test?


      To dig into what is causing the different results I would recommend starting with the data. Add prints to ensure that the very first bar (real-time only) in playback matches the first bar of backtest in the strategy analyzer.

      Historical data is recorded for your specific data feed connection. Market Replay is recorded by NinjaTrader from a single source. Each data feed is slightly different with timestamps of ticks as well as qualifying vs non qualifying trades.

      Who are you connected to for historical data?
      Are you recording your own Market Replay data from this source or is this being downloaded from NinjaTrader's Market Replay server?

      Add prints to ensure that the open, high, low, close, volume, and time are the same.
      Chelsea B.NinjaTrader Customer Service

      Comment


        #4
        I'm preparing to add prints. Would you recommend adding the Print to my strategy or the Indicator itself?

        Code:
        		if (IsFirstTickOfBar)
        	    {
        	        double FirstClose = Close[0];
        			Print (FirstClose);
        	    }
        Will this suffice?

        Comment


          #5
          Additionally, I have a secondary data period setup in my indicator

          Code:
          AddDataSeries(Data.BarsPeriodType.Minute, barPeriod);
          My strategy calls back to the indicator and I have a custom variable to declare barPeriod, however, I do not know what to print to ensure I am getting the correct range.

          Furthermore, I have a calculated double from my indicator "l3_0" that I would like to reference within my strategy, how could I do this?

          Comment


            #6
            Hello rdavido,

            I would recommend adding the prints to the strategy (first).

            The goal would be to see what is different.

            The first print in both playback and historical need to the be the same bar (shown by time).

            The playback would not include any historical data so that this builds with the same data that was tested historically.

            On each bar if the times match up, I would recommend printing the values of all variables in conditions that place orders outside of any conditions so that we can understand what was different (either data or order fills).
            Chelsea B.NinjaTrader Customer Service

            Comment

            Latest Posts

            Collapse

            Topics Statistics Last Post
            Started by Johnny, Today, 11:42 AM
            0 responses
            3 views
            0 likes
            Last Post Johnny
            by Johnny
             
            Started by habibalex, Today, 09:03 AM
            2 responses
            10 views
            0 likes
            Last Post sledge
            by sledge
             
            Started by ptpatrader, Today, 11:38 AM
            1 response
            4 views
            0 likes
            Last Post NinjaTrader_ChrisL  
            Started by emuns, Today, 11:20 AM
            1 response
            4 views
            0 likes
            Last Post NinjaTrader_Jim  
            Started by chartish, Today, 10:33 AM
            1 response
            10 views
            0 likes
            Last Post NinjaTrader_PatrickG  
            Working...
            X