Announcement

Collapse

Looking for a User App or Add-On built by the NinjaTrader community?

Visit NinjaTrader EcoSystem and our free User App Share!

Have a question for the NinjaScript developer community? Open a new thread in our NinjaScript File Sharing Discussion Forum!
See more
See less

Partner 728x90

Collapse

sample for IsInstantiatedOnEachOptimizationIteration.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    sample for IsInstantiatedOnEachOptimizationIteration.




    people with nt,



    i have been working with nt on a modest, 4 gb laptop lately, so running optimization processes is not practical in the least.


    instead of running 100 or 150 generations with generation size of 100 or 150 as would be desirable, i have had to run 30 generations with generation size of 100. and when trying to run an optimization for a strategy that includes code for a breakeven stop i really had to scale these parameters to the minimum. i ended up running 5 generations with a size of 100 and this process still took 5 hours - 300 minutes. if i try to run 100 - 150 by 100 - 150 processes, the platform will abort the processes and display incomplete and useless results.


    supposedly, setting IsInstantiatedOnEachOptimizationIteration to false is the one change that should make the biggest difference in the performance of the platform and i'm interested in evaluating whether this is of any help when running optimizations. i had had the idea of modifying my strategies for a very long time so that this parameter could be set to false but have never done this up to now because it is a really difficult undertaking for any non programmer and i haven't been able to get started. i have read all the posts in these fora where the IsInstantiatedOnEachOptimizationIteration setting is discussed and there are no helpful explanations or working samples anywhere.


    so, for this sample strategy i include below, i would like the people with nt support to help me by providing a list of which components must be converted to what and some explanation. i will then search on the internet for more extensive c# resources and will also try to make do with the minimal moving average crossover sample strategy that nt has made available on its help guides.


    this sample strategy does not include any day of the week or time of the day components, but i do have several strategies which use code of this kind, so i want to know whether it is necessary to also reset these components. and it's an identical situation with all the code that goes inside the onorderupdate and onexecutionupdate methods, i also have strategies which include this code.



    Code:
    #region Using declarations
    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.ComponentModel.DataAnnotations;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    using System.Windows;
    using System.Windows.Input;
    using System.Windows.Media;
    using System.Xml.Serialization;
    using NinjaTrader.Cbi;
    using NinjaTrader.Gui;
    using NinjaTrader.Gui.Chart;
    using NinjaTrader.Gui.SuperDom;
    using NinjaTrader.Gui.Tools;
    using NinjaTrader.Data;
    using NinjaTrader.NinjaScript;
    using NinjaTrader.Core.FloatingPoint;
    using NinjaTrader.NinjaScript.Indicators;
    using NinjaTrader.NinjaScript.DrawingTools;
    #endregion
    
    //This namespace holds Strategies in this folder and is required. Do not change it.
    namespace NinjaTrader.NinjaScript.Strategies
    {
    public class tesampletrimod : Strategy
    {
    private SMA SMA1;
    private Momentum Momentum1;
    private ADX ADX1;
    
    protected override void OnStateChange()
    {
    if (State == State.SetDefaults)
    {
    Description = @"te sample tri mod.";
    Name = "tesampletrimod";
    Calculate = Calculate.OnBarClose;
    EntriesPerDirection = 1;
    EntryHandling = EntryHandling.UniqueEntries;
    IsExitOnSessionCloseStrategy = false;
    ExitOnSessionCloseSeconds = 90;
    IsFillLimitOnTouch = false;
    MaximumBarsLookBack = MaximumBarsLookBack.TwoHundredFiftySix;
    OrderFillResolution = OrderFillResolution.Standard;
    Slippage = 25;
    StartBehavior = StartBehavior.WaitUntilFlat;
    TimeInForce = TimeInForce.Gtc;
    TraceOrders = true;
    RealtimeErrorHandling = RealtimeErrorHandling.StopCancelCloseIgnoreRejects ;
    StopTargetHandling = StopTargetHandling.PerEntryExecution;
    BarsRequiredToTrade = 65;
    // Disable this property for performance gains in Strategy Analyzer optimizations
    // See the Help Guide for additional information
    IsInstantiatedOnEachOptimizationIteration = true;
    Posi = 1;
    Smap = 20;
    Momp = 20;
    Adxp = 20;
    Moml1 = -.5;
    Moml2 = .5;
    Adxl = 30;
    }
    else if (State == State.Configure)
    {
    }
    else if (State == State.DataLoaded)
    {
    SMA1 = SMA(Close, Convert.ToInt32(Smap));
    Momentum1 = Momentum(Close, Convert.ToInt32(Momp));
    ADX1 = ADX(Close, Convert.ToInt32(Adxp));
    }
    }
    
    protected override void OnBarUpdate()
    {
    if (BarsInProgress != 0)
    return;
    
    if (CurrentBars[0] < 1)
    return;
    
    // Set 1
    if ((Position.MarketPosition != MarketPosition.Short)
    && (SMA1[0] < SMA1[1])
    && (Momentum1[0] < Moml1)
    && (ADX1[0] > Adxp))
    {
    EnterShort(Convert.ToInt32(Posi), @"sp01");
    }
    
    
    else
    
    
    // Set 4
    if ((Position.MarketPosition == MarketPosition.Long)
    && (SMA1[0] < SMA1[1]))
    {
    ExitLong(Convert.ToInt32(Posi), @"elp01", @"lp01");
    }
    
    
    // Set 3
    if ((Position.MarketPosition != MarketPosition.Long)
    && (SMA1[0] > SMA1[1])
    && (Momentum1[0] > Moml2)
    && (ADX1[0] > Adxp))
    {
    EnterLong(Convert.ToInt32(Posi), @"lp01");
    }
    
    
    else
    
    
    // Set 2
    if ((Position.MarketPosition == MarketPosition.Short)
    && (SMA1[0] > SMA1[1]))
    {
    ExitShort(Convert.ToInt32(Posi), @"esp01", @"sp01");
    }
    
    
    }
    
    #region Properties
    [NinjaScriptProperty]
    [Range(1, int.MaxValue)]
    [Display(Name="Posi", Description="posi.", Order=1, GroupName="Parameters")]
    public int Posi
    { get; set; }
    
    [NinjaScriptProperty]
    [Range(1, int.MaxValue)]
    [Display(Name="Smap", Order=2, GroupName="Parameters")]
    public int Smap
    { get; set; }
    
    [NinjaScriptProperty]
    [Range(1, int.MaxValue)]
    [Display(Name="Momp", Order=3, GroupName="Parameters")]
    public int Momp
    { get; set; }
    
    [NinjaScriptProperty]
    [Range(1, int.MaxValue)]
    [Display(Name="Adxp", Order=4, GroupName="Parameters")]
    public int Adxp
    { get; set; }
    
    [NinjaScriptProperty]
    [Range(-100, double.MaxValue)]
    [Display(Name="Moml1", Order=5, GroupName="Parameters")]
    public double Moml1
    { get; set; }
    
    [NinjaScriptProperty]
    [Range(-100, double.MaxValue)]
    [Display(Name="Moml2", Order=6, GroupName="Parameters")]
    public double Moml2
    { get; set; }
    
    [NinjaScriptProperty]
    [Range(1, double.MaxValue)]
    [Display(Name="Adxl", Order=7, GroupName="Parameters")]
    public double Adxl
    { get; set; }
    #endregion
    
    }
    }


    very well, regards.

    #2
    Hi rtwave, thanks for posting.

    Only objects that are mutable throughout the run time of any one optimization iteration must be reset e.g. custom lists and order objects. I do not see any of these objects in the script you posted. There is an example on the IsInstantiatedOnEachOptimizationIteration demonstrating some objects that need to be reset in State.DataLoaded. See the section "Manually resetting class level variables to take advantage of Strategy Analyzer optimizer performance benefits"
    https://ninjatrader.com/support/help...niteration.htm

    Kind regards,
    -ChrisL
    Chris L.NinjaTrader Customer Service

    Comment


      #3





      people with nt,



      i have been running some experiments with the IsInstantiatedOnEachOptimizationIteration setting and the results have been awful.


      - the platform will actually perform far worse when setting the instantiated parameter to false.


      i have thought it and said it for a long time: the fact that the people with nt do not use the platform to actually trade or develop actual strategies is quite problematic.



      now, the situation has been as follows: i created a very simple strategy, to run on 2 minute bars and only take short positions.


      i then optimized the strategy from 202001 to 202106 over a minimal number of iterations and the process took a little less than 30 minutes with the instantiated parameter set to true:


      Click image for larger version

Name:	20210727 shon instantiated tests t duration 001.JPG
Views:	169
Size:	11.2 KB
ID:	1165365



      the next step was to try and optimize the same strategy, over the same ranges, the same data, under identical conditions but with the instantiated parameter set to false and it has proven impossible. the platform will take much longer to make any progress and eventually it will stop responding entirely:



      Click image for larger version

Name:	20210727 shon instantiated tests f duration crash 001.JPG
Views:	144
Size:	10.0 KB
ID:	1165366


      Click image for larger version

Name:	20210727 shon instantiated tests f duration crash 002.JPG
Views:	141
Size:	9.1 KB
ID:	1165367




      and i have had these issues across all my strategies since i decided to change the instantiated parameter to false. i will try to run this same experiment with the sample strategy i have posted to these fora multiple times to further cement my thesis, but the conclusions are clear to me: the promise of superior performance by changing the instantiated parameter is completely false and it will actually render the nt platform inoperable, therefore i will not consider using it anymore.



      żis nt aware of this craziness? żis this to be considered normal and expected behavior from the nt platform? żis there anything nt can comment on this situation i'm reporting?



      very well, regards.

      Comment


        #4
        Hi rtwave, thanks for your reply.

        The CPU speed, RAM speed, quantity of RAM, and strategy code are the biggest factors for how a strategy is going to perform in an optimization backtest. All of these factors can vary on an individual basis. If you suspect the strategy to be using resources in an inefficient way, reduce the strategy and test how it performs. Another way to test, although sometimes not possible or easy to do, is to test the same data and code on a different computer. E.g. my personal PC has a 10 core i9 and 64GB of RAM clocked at 3600Mhz and optimizations usually don't take very long.

        Kind regards,
        -ChrisL
        Chris L.NinjaTrader Customer Service

        Comment


          #5


          people with nt,



          yesterday i had the chance to run some tests with this strategy i have sent multiple times to the people with nt:


          Code:
          #region Using declarations using System; using System.Collections.Generic; using System.ComponentModel; using System.ComponentModel.DataAnnotations; using System.Linq; using System.Text; using System.Threading.Tasks; using System.Windows; using System.Windows.Input; using System.Windows.Media; using System.Xml.Serialization; using NinjaTrader.Cbi; using NinjaTrader.Gui; using NinjaTrader.Gui.Chart; using NinjaTrader.Gui.SuperDom; using NinjaTrader.Gui.Tools; using NinjaTrader.Data; using NinjaTrader.NinjaScript; using NinjaTrader.Core.FloatingPoint; using NinjaTrader.NinjaScript.Indicators; using NinjaTrader.NinjaScript.DrawingTools; #endregion //This namespace holds Strategies in this folder and is required. Do not change it. namespace NinjaTrader.NinjaScript.Strategies { public class tesampletrimod : Strategy { private SMA SMA1; private Momentum Momentum1; private ADX ADX1; protected override void OnStateChange() { if (State == State.SetDefaults) { Description = @"te sample tri mod."; Name = "tesampletrimod"; Calculate = Calculate.OnBarClose; EntriesPerDirection = 1; EntryHandling = EntryHandling.UniqueEntries; IsExitOnSessionCloseStrategy = false; ExitOnSessionCloseSeconds = 90; IsFillLimitOnTouch = false; MaximumBarsLookBack = MaximumBarsLookBack.TwoHundredFiftySix; OrderFillResolution = OrderFillResolution.Standard; Slippage = 25; StartBehavior = StartBehavior.WaitUntilFlat; TimeInForce = TimeInForce.Gtc; TraceOrders = true; RealtimeErrorHandling = RealtimeErrorHandling.StopCancelCloseIgnoreRejects ; StopTargetHandling = StopTargetHandling.PerEntryExecution; BarsRequiredToTrade = 65; // Disable this property for performance gains in Strategy Analyzer optimizations // See the Help Guide for additional information IsInstantiatedOnEachOptimizationIteration = true; Posi = 1; Smap = 20; Momp = 20; Adxp = 20; Moml1 = -.5; Moml2 = .5; Adxl = 30; } else if (State == State.Configure) { } else if (State == State.DataLoaded) { SMA1 = SMA(Close, Convert.ToInt32(Smap)); Momentum1 = Momentum(Close, Convert.ToInt32(Momp)); ADX1 = ADX(Close, Convert.ToInt32(Adxp)); } } protected override void OnBarUpdate() { if (BarsInProgress != 0) return; if (CurrentBars[0] < 1) return; // Set 1 if ((Position.MarketPosition != MarketPosition.Short) && (SMA1[0] < SMA1[1]) && (Momentum1[0] < Moml1) && (ADX1[0] > Adxp)) { EnterShort(Convert.ToInt32(Posi), @"sp01"); } else // Set 4 if ((Position.MarketPosition == MarketPosition.Long) && (SMA1[0] < SMA1[1])) { ExitLong(Convert.ToInt32(Posi), @"elp01", @"lp01"); } // Set 3 if ((Position.MarketPosition != MarketPosition.Long) && (SMA1[0] > SMA1[1]) && (Momentum1[0] > Moml2) && (ADX1[0] > Adxp)) { EnterLong(Convert.ToInt32(Posi), @"lp01"); } else // Set 2 if ((Position.MarketPosition == MarketPosition.Short) && (SMA1[0] > SMA1[1])) { ExitShort(Convert.ToInt32(Posi), @"esp01", @"sp01"); } } #region Properties [NinjaScriptProperty] [Range(1, int.MaxValue)] [Display(Name="Posi", Description="posi.", Order=1, GroupName="Parameters")] public int Posi { get; set; } [NinjaScriptProperty] [Range(1, int.MaxValue)] [Display(Name="Smap", Order=2, GroupName="Parameters")] public int Smap { get; set; } [NinjaScriptProperty] [Range(1, int.MaxValue)] [Display(Name="Momp", Order=3, GroupName="Parameters")] public int Momp { get; set; } [NinjaScriptProperty] [Range(1, int.MaxValue)] [Display(Name="Adxp", Order=4, GroupName="Parameters")] public int Adxp { get; set; } [NinjaScriptProperty] [Range(-100, double.MaxValue)] [Display(Name="Moml1", Order=5, GroupName="Parameters")] public double Moml1 { get; set; } [NinjaScriptProperty] [Range(-100, double.MaxValue)] [Display(Name="Moml2", Order=6, GroupName="Parameters")] public double Moml2 { get; set; } [NinjaScriptProperty] [Range(1, double.MaxValue)] [Display(Name="Adxl", Order=7, GroupName="Parameters")] public double Adxl { get; set; } #endregion } }

          the results were a mixed bag.

          the version with instantiated true took practically the same,


          Click image for larger version

Name:	20210727 sampletrimod instantiated tests t duration 001.JPG
Views:	259
Size:	10.8 KB
ID:	1165631

          as the version with instantiated false.


          Click image for larger version

Name:	20210727 sampletrimod instantiated tests f duration 001.JPG
Views:	128
Size:	11.4 KB
ID:	1165632


          both processes were fast, but the platform was not able to execute optimizations with larger numbers of generations or generation sizes.


          and the results with instantiated true are quite credible, but the platform delivers dumb results and ugly performance graphs for the instantiated false version, even when all settings and configuration are identical.



          i have also realized that some of my own indicators i have developed take a lot of processing power for the platform to process and work with.



          i have documented and reported numerous issues and crashes when using the nt optimization engine, so i would like to ask of the people with nt to put their fantastic computers to good use and optimize this strategy i have posted - emailed multiple times on 5 minute bars, from january 2020 to date, on the m2k contract, with 4 ticks as slippage and over the following ranges:



          Posi = 1;
          Smap = 3;100;1
          Momp = 3;100;1
          Adxp = 3;100;1
          Moml1 = -0.7;0;0.1
          Moml2 = 0;0.7;0.1
          Adxl = 2;30;1


          and giving the platform a big number of iterations to work with: 150 generations and 150 generation size.



          this results in persistent crashes and failures when i do it, so, perhaps the people with nt can gain valuable insight from this exercise.


          very well, regards.

          Comment


            #6
            Hi rtwave, thanks for your reply.

            Optimizations with large generation and generation size can take a very long time and consume all of your computer's memory. If you get faster results using lower numbers in these parameters, the strategy is working, it just needs time to process the many generations that are being requested.

            Kind regards,
            -ChrisL
            Chris L.NinjaTrader Customer Service

            Comment

            Latest Posts

            Collapse

            Topics Statistics Last Post
            Started by Barry Milan, Yesterday, 10:35 PM
            5 responses
            16 views
            0 likes
            Last Post NinjaTrader_Manfred  
            Started by DanielSanMartin, Yesterday, 02:37 PM
            2 responses
            13 views
            0 likes
            Last Post DanielSanMartin  
            Started by DJ888, 04-16-2024, 06:09 PM
            4 responses
            13 views
            0 likes
            Last Post DJ888
            by DJ888
             
            Started by terofs, Today, 04:18 PM
            0 responses
            12 views
            0 likes
            Last Post terofs
            by terofs
             
            Started by nandhumca, Today, 03:41 PM
            0 responses
            8 views
            0 likes
            Last Post nandhumca  
            Working...
            X