• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

IsValidDataPoint and Strategy Optimize NT8 B11

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    IsValidDataPoint and Strategy Optimize NT8 B11

    If I use IsValidDataPoint in Strategy or Indicator then this causes problems in Strategy Analyzer Optimization.
    If I use BacktestType=Standard there is no problem but if I use BacktestType=Optimization then I get the error
    .IsValidDataPoint cannot be used on a series running with MaximumBarsLookBack.TwoHundredFiftySix.
    This can also cause NT8 to stop (perhaps if the number of combinations to test is too large).
    I presume the difference is that BacktestType=Standard probably runs on a single thread whereas BacktestType=Standard uses multiple threads. So perhaps the quoted error is due to NT8 not yet making IsValidDataPoint thread safe?

    There are a few problems here:
    1) Documentation for MaximumBarsLookBack does not match the current code:
    The page currently says
    Series<double> objects that hold indicator plot values are always set to MaximumBarsLookBack.Infinite
    However a simple test shows that is not true (see below).

    2)The example code on IsValidDataPoint documentation is silly since it uses SMA which sets Value for every bar. In order to test IsValidDataPoint we need a new SMA indicator that only starts plotting after Period inputs.

    Steps to reproduce problem:
    a) Edit SMA indicator save as SMATest.
    SMATest is minimally changed so it only starts setting Value after it gets Period inputs:
    Code:
    sum = priorSum + Input[0] - (CurrentBar >= Period ? Input[Period] : 0);
    if (CurrentBar+1 < Period) return;//New line. Dont set Value until have Period values in sum
    Value[0] = sum / (CurrentBar < Period ? CurrentBar + 1 : Period);
    Also put this
    Code:
    BarsRequiredToPlot = 0;//Value not plotted until have Period values in sum
    in if (State == State.SetDefaults)

    b) Edit strategy SampleMACrossOver, save as TestMACrossOver
    Change SMA references to SMATest.
    Also add a diagnostic Print
    Code:
    else if (State == State.Configure)
    			{
    				smaFast = SMATest(Fast);//changed
    				smaSlow = SMATest(Slow);//changed
    
    				
    BarsRequiredToTrade = 0;
    Print(string.Format("smaSlow.MaximumBarsLookBack = {0}",smaSlow.MaximumBarsLookBack));//reports  TwoHundredFiftySix
    Add this line in OnBarUpdate
    Code:
    if (!smaSlow.IsValidDataPoint(0)) return;//new line
    if (CrossAbove(smaFast, smaSlow, 1))

    When you run TestMACrossOver in Strategy Analyzer BacktestType=Standard you see there is no error, also in the Output window you see
    smaSlow.MaximumBarsLookBack = TwoHundredFiftySix
    Which shows that standard NT8 B11 plots use 256 not infinite as MaximumBarsLookBack documentation says.

    Now comment out the Print line.
    Run TestMACrossOver in Strategy Analyzer BacktestType=Optimization (Optimizer=Default). Set a small number of combinations (here just 2) say Fast range 2 to 3, Slow range say 4.

    On my PC (multi-core) this tends to keep going producing 2 error lines per run
    Strategy 'TestMACrossOver': Error on calling 'OnBarUpdate' method on bar 0: .IsValidDataPoint cannot be used on a series running with MaximumBarsLookBack.TwoHundredFiftySix.
    But if I request a larger number of combinations (say fast range=2 to 10, slow range = 20 to 30) then Strategy Analyzer will freeze after a few repeats, probably displaying something like
    calculating Iteration 1/99
    with nothing happening in progress bar. I then have to press "Abort" button, but Strategy Analyzer has become unusable (have to close window and open new one). If I use a large range and also Optimize Data Series, then NT stops (get Windows message Ninja Trader has stopped working, windows is searching for solution"!). Log records "Unhandled exception: Object not set to an instance of an object".

    Restarting NT8:
    If I try the same thing but with Optimizer=Genetic, then I see I get 8 lines of the error (probably 1 per thread, or one per core) but Strategy Analyzer does not lock up, I dont have to Abort and I can keep repeating this without NT stopping.
    So error handling in the Genetic Optimizer seems to be much better than the Default Optimizer.

    If I switch to using SampleMACrossover: I get no operational problems with either Optimizer (there's just the mathematical problem that the built in SMA plots before it has Period inputs).

    It looks like the current implementation of IsValidDataPoint has not been tested in a multi-threading context such as Optimization.

    #2
    Hello DaveE,

    Thank you for your post.

    We look to improve the documentation overall so this is appreciated.

    Can you attach the strategy 'TestMACrossOver' to your response that you ran for your tests?
    Patrick H.NinjaTrader Customer Service

    Comment


      #3
      TestMACrossOver attached.
      The problem is not just about documentation.
      n the worst case IsValidDataPoint causes NT to stop!
      Attached Files

      Comment


        #4
        Hello DaveE,

        Thank you for your response.

        I am not running into the same behavior on my end. I will continue testing with different combinations for the parameters on my end.
        Patrick H.NinjaTrader Customer Service

        Comment


          #5
          Are you testing on multi-core PC? I was using 64bit NT8 on a 4 core 8 thread CPU.

          Comment


            #6
            I am running NinjaTrader 8 64 bit on a 4 core, 8 thread CPU.
            Patrick H.NinjaTrader Customer Service

            Comment


              #7
              Hello,

              any news about this bug ?

              I am having the same trouble. I have copied the standard SampleMACrossOver and I have only added one sentence for asking about IsValidDataPoint and the Optimizer crashes.
              However the standard backtest runs ok.

              Thanks
              Attached Files

              Comment


                #8
                I am checking that this issue has just been fixed in beta12. Thanks.

                Comment


                  #9
                  I can confirm that B12 fixes the problem so Optimization no longer causes the error ".IsValidDataPoint cannot be used on a series running with MaximumBarsLookBack.TwoHundredFiftySix."


                  There is still however the oddity that I still get report ".MaximumBarsLookBack = TwoHundredFiftySix" from indicators in strategy, regardless of the setting of "Maximum Bars Lookback" combo in Strategy Analyzer.

                  Comment


                    #10
                    Hello DaveE,

                    Thank you for your update.

                    Can you detail the full message for the Log tab of the Control Center?
                    Patrick H.NinjaTrader Customer Service

                    Comment


                      #11
                      The remaining oddity is not an exception. I'm referring to the diagnostic Print in TestMACrossover strategy (attached to a previous post).

                      Comment


                        #12
                        Thank you, Dave.
                        Series<double> objects that hold indicator plot values are always set to MaximumBarsLookBack.Infinite which ensures that charts will always display the entire indicator's calculated values.
                        This means Plot values in an indicator for example. Not an indicator set to a series.
                        Patrick H.NinjaTrader Customer Service

                        Comment

                        Latest Posts

                        Collapse

                        Topics Statistics Last Post
                        Started by Erwin Beckers, 11-04-2018, 02:39 AM
                        4 responses
                        69 views
                        2 likes
                        Last Post brucerobinson  
                        Started by ntbone, Today, 01:25 AM
                        0 responses
                        2 views
                        0 likes
                        Last Post ntbone
                        by ntbone
                         
                        Started by bomanpbm, Yesterday, 09:27 PM
                        0 responses
                        2 views
                        0 likes
                        Last Post bomanpbm  
                        Started by tgn55, Yesterday, 08:47 AM
                        0 responses
                        8 views
                        0 likes
                        Last Post tgn55
                        by tgn55
                         
                        Started by ballboy11, 05-04-2019, 10:26 AM
                        6 responses
                        59 views
                        0 likes
                        Last Post Kermit
                        by Kermit
                         
                        Working...
                        X