• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Same Data, Same Strategy, Different Results?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Same Data, Same Strategy, Different Results?

    I noticed that when moving my strategies over from NT7 to NT8 I got very different results in Strategy Analyzer. So I decided to test the "SampleMaCrossOver" strategy which is the exactly the same in both versions.

    So Using Kinetic EOD data I tested the SampleMACross on RB and also got very different results. Anyone have any thoughts on why i'm seeing such a big difference? I'm sure I am missing something here.

    NT 8




    NT 7

    #2
    Hello aufidius,

    Thank you for your post. I tested the SampleMACrossOver in NinjaTrader 8 and I do get the same results that you have.

    However, I tested the same NinjaTrader 7 parameters and I do not get the same while backtesting. Can you please test using the SampleMACrossOver attached to my reply?

    Please include a screen shot of your results.
    Attached Files
    Eric B.NinjaTrader Customer Service

    Comment


      #3
      Thanks for posting,

      I checked into further since I expect overall similar results from NT7 to NT8 with standard backtests.

      The issue is a difference between the amount of data in NinjaTrader 7 vs NinjaTrader 8.NT7 is only including data from the first expiration date in the database forward which is from 2008. NT8 is returning all data for the entire last contract in the database which is the cause for the different results.

      If you change your start date on NT8 to be 7/21/2008 you should see similar results to NT7.

      I'm checking into this and look into making a change in NT8 to match NT7 here.
      BrettNinjaTrader Product Management

      Comment


        #4
        using your code the results on my machine are the same as before




        Interestingly after my post someone privately told me about this article http://www.trade-robots.com/blog/nin...lts-are-random

        It seems other people have notice inconsistencies with NT7 in the past.

        Comment


          #5
          Based on your explanation it seems to be that NT8's behavior is the ideal way to work. With NT7 users could be assuming they are testing with 10 years of historical data when they are only getting 7 and they are not told. This could lead to people thinking a system is more robust than it actually is...



          Originally posted by NinjaTrader_Brett View Post
          Thanks for posting,

          I checked into further since I expect overall similar results from NT7 to NT8 with standard backtests.

          The issue is a difference between the amount of data in NinjaTrader 7 vs NinjaTrader 8.NT7 is only including data from the first expiration date in the database forward which is from 2008. NT8 is returning all data for the entire last contract in the database which is the cause for the different results.

          If you change your start date on NT8 to be 7/21/2008 you should see similar results to NT7.

          I'm checking into this and look into making a change in NT8 to match NT7 here.

          Comment


            #6
            Originally posted by aufidius View Post
            using your code the results on my machine are the same as before




            Interestingly after my post someone privately told me about this article http://www.trade-robots.com/blog/nin...lts-are-random

            It seems other people have notice inconsistencies with NT7 in the past.
            Both the link that you point to, and your own results are usually an indication that you are not testing the same dataset. For comparison purposes, every single parameter of your tests must be identical. Start dat, end date, time period, possible even the data vendor.

            After all parameters are set, you may want to delete your data (or save it off), and download fresh data (to multiple computers, if your tests are not on the same computer).

            Have I seen this behavior before? Yes. So far, in every instance, after review, it was something that I did that meant that I was not making identical tests. Every time, I corrected the data and parameters issue, I got identical results in backtest.

            Comment

            Latest Posts

            Collapse

            Topics Statistics Last Post
            Started by tecmisc1, Today, 09:57 AM
            0 responses
            2 views
            0 likes
            Last Post tecmisc1  
            Started by mbesha, Today, 09:26 AM
            0 responses
            1 view
            0 likes
            Last Post mbesha
            by mbesha
             
            Started by Brillo, Today, 08:58 AM
            0 responses
            6 views
            0 likes
            Last Post Brillo
            by Brillo
             
            Started by avrege, Today, 01:48 AM
            2 responses
            14 views
            0 likes
            Last Post avrege
            by avrege
             
            Started by markbb10, Today, 07:14 AM
            0 responses
            7 views
            0 likes
            Last Post markbb10  
            Working...
            X