Announcement

Collapse

Looking for a User App or Add-On built by the NinjaTrader community?

Visit NinjaTrader EcoSystem and our free User App Share!

Have a question for the NinjaScript developer community? Open a new thread in our NinjaScript File Sharing Discussion Forum!
See more
See less

Partner 728x90

Collapse

StreamWriter and Optimization

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    StreamWriter and Optimization

    Hello Ninja!

    I want to print some values about my trades, (entryTime, exitTime, entryPrice, exitPrice etc) from my strategy to a textfile. This works for a single backtest. But I want to write multiple files for an optimization, one file for each combination of parameters. This was no problem in NT7, but in NT8 it's no longer working with my code since the optimizer runs in parallell/ asynchronous. One obvious problem is that State == State.DataLoaded is called at the beginning for all combination of parameters, that is the StreamWriter is "initialized" for all runs before it is disposed.

    Is there a way to write multiple textfiles files from the optimizer in NT8 ? (Maybe with multiple StreamWriter objects?) A single textfile with all trades matched with the right parameter values would also be ok.



    The StreamWriter is "initialized" when State = State.DataLoaded

    Code:
    			else if (State == State.DataLoaded)
    			{
                                           tradeNo = 0;    // Reset
    
    		                        if (!File.Exists(fileLocation))
    					{
    						sw = File.AppendText(fileLocation );
    					}
    			}


    and disposed when State = State.Terminated.

    The Writing is done in OnPositionUpdate()

    Code:
    		protected override void OnPositionUpdate(....)
    		{
    			// WriteToFile
    			if (SystemPerformance.AllTrades.Count > 0 
    				&& tradeNo != SystemPerformance.AllTrades.Count )	
    			{
    				Trade lastTrade = SystemPerformance.AllTrades[SystemPerformance.AllTrades.Count - 1];
    				tradeNo += 1;
    
    				try
    				{
    					sw.WriteLine(	          parameter1  + ";"
                                                                            + parameter2  + ";"
                                                                            + lastTrade.Entry.Time + ";"
    									+ lastTrade.Exit.Time + ";"
    									+ lastTrade.Entry.Price + ";"
    									+ lastTrade.Exit.Price);
    				}
    				catch (Exception e)
    				{
    					/// Outputs the error to the log
    					Log("You cannot write and read from the same file at the same time. Please remove StreamReader.", NinjaTrader.Cbi.LogLevel.Error);
    					throw;
    				}
    		}
    / Poseidon_Sthlm
    Last edited by poseidon_sthlm; 08-05-2018, 04:59 AM.

    #2
    Hello Poseidon_Sthlm,

    Your options are to write to different files that have different IDs put into the filename, or add timers to the script and pause threads (requiring extremely thread aware code, causing serious performance issues with optimizations).

    I would recommend generating a different ID for each test by generating a new id and saving this to a variable for each backtest iteration.

    Citizens of the NinjaTrader Community, A common question we hear from clients is 'why are results from backtest different from real-time or from market replay?'. Live orders are filled on an exchange with a trading partner on an agreed upon price based on market dynamics. Backtest orders are not using these market dynamics.
    Chelsea B.NinjaTrader Customer Service

    Comment


      #3
      I would recommend generating a different ID for each test by generating a new id and saving this to a variable for each backtest iteration.
      Hi!

      Thanks for the advice on using and ID to separate the different runs of the optimizer with
      Code:
      System.Guid guid = System.Guid.NewGuid();
      I have a managed to find a solution on how to write textfiles from a strategy during optimization but it doesn't feel very reliable because:
      - when the GUID ID is printed to the output window it shows that the optimizer begins with a truly parallell run for all combinations of parameters and there after makes one or more runs for some combinations of parameters. This feels strange. (But State.DataLoaded is only called once for each combination of parameters.)

      - I'm not sure where to dispose the StreamWriter since State.Termination is called several times, once for each run.

      Is there an "event" that one could use at the beginning / end of an optimization?

      Comment


        #4
        Hello poseidon_sthlm,

        Is the information written to file incorrect?

        (edited)
        Disposing in OnStateChange() when State is State.Terminated if the sw object is not null should be fine.

        There will be times that OnStateChange() for State.Terminated is called but not for an actual backtest instance. OnStateChange() is called several times as part of the setup.

        NinjaTrader_Matthew (who is no longer with NinjaTrader) detailed this out.



        Unfortunately, there are not any events that run before and after an optimization.
        Last edited by NinjaTrader_ChelseaB; 08-09-2018, 11:18 AM.
        Chelsea B.NinjaTrader Customer Service

        Comment


          #5
          The information written to the files is correct (compared to files from separate backtests). But I'm not quite sure what is going on during the optimization. This is a multi timeframe strategy.

          Disposing in OnTermination() if the sw object is not null should be fine
          This is helpful information. Does this mean that all events after the first OnTermination() don't generate any information that is relevant for my texfiles?
          Last edited by poseidon_sthlm; 08-09-2018, 10:57 AM.

          Comment


            #6
            Hello poseidon_sthlm,

            I'm not understanding what information you would be expecting to come from OnStateChange() when the State is State.Terminated.

            State.Terminated runs in OnStateChange when the strategy instance is terminated and gives the script a chance to clean up resources.

            Any information from a backtest you want is likely going to come from OnBarUpdate, OnOrderUpdate, OnExecution, etc. as stuff is happening...

            The State for each backtest iteration is separate from other iterations..

            Just as the State for a script instance running in real-time is separate from all other instances.
            Chelsea B.NinjaTrader Customer Service

            Comment


              #7
              At the end of OnStateChange() I do print the following lines to the output window:

              Code:
              		Print("");
              			Print("********************************************");
              			Print("OnStateChange: " + State);
              			Print("********************************************");
              			Print("sw == null: " + (sw == null));
              			Print("guid: " + guid.ToString() );
              The StreamWriter sw is disposed in OnTermination()

              This results in the following output for the simplest possible optimization for a strategy with one varaibles of two values (See below). State.Historical, State.OmTermination etc are called several times after the sw is disposed. It does look a bit confusing to me. Do I dispose the sw at the right event?

              Code:
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: cfccd79d-0e7e-4883-985e-391b7367d120
              
              ********************************************
              OnStateChange: Configure
              ********************************************
              sw == null: True
              guid: cfccd79d-0e7e-4883-985e-391b7367d120
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 34cdcdea-a719-43cb-9c34-1ddcb8915abc
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 41e5e142-c719-4c12-8761-105ba7ea6cf0
              
              ********************************************
              OnStateChange: Configure
              ********************************************
              sw == null: True
              guid: 34cdcdea-a719-43cb-9c34-1ddcb8915abc
              
              ********************************************
              OnStateChange: Configure
              ********************************************
              sw == null: True
              guid: 41e5e142-c719-4c12-8761-105ba7ea6cf0
              
              ********************************************
              OnStateChange: DataLoaded
              ********************************************
              [B]sw == null: False[/B]
              guid: 34cdcdea-a719-43cb-9c34-1ddcb8915abc
              
              ********************************************
              OnStateChange: DataLoaded
              ********************************************
              sw == null: False
              guid: 41e5e142-c719-4c12-8761-105ba7ea6cf0
              
              ********************************************
              
              ********************************************
              OnStateChange: Historical
              ********************************************
              sw == null: False
              guid: 41e5e142-c719-4c12-8761-105ba7ea6cf0
              OnStateChange: Historical
              ********************************************
              sw == null: False
              guid: 34cdcdea-a719-43cb-9c34-1ddcb8915abc
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              [B]sw == null: True[/B]
              guid: 41e5e142-c719-4c12-8761-105ba7ea6cf0
              
              ********************************************
              [COLOR="Red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: 34cdcdea-a719-43cb-9c34-1ddcb8915abc
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 67c481f2-4cbb-4e55-962f-d153f774dfbc
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: 67c481f2-4cbb-4e55-962f-d153f774dfbc
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: cfccd79d-0e7e-4883-985e-391b7367d120
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: 086d1630-0d17-46b2-924c-de9e68b315db
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 9ee51b48-70ce-4318-b6e0-09c769a1c126
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: c5d7b460-8fd3-4a0b-9163-83755bb67396
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 170067b5-130b-4aa8-a635-e3de8fc2ff0e
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 8d741920-aee1-47fa-9a7a-924ffa446e3a
              
              ********************************************
              
              ********************************************
              OnStateChange: Configure
              ********************************************
              sw == null: True
              OnStateChange: Configure
              ********************************************
              guid: c5d7b460-8fd3-4a0b-9163-83755bb67396
              sw == null: True
              guid: 170067b5-130b-4aa8-a635-e3de8fc2ff0e
              
              ********************************************
              OnStateChange: Configure
              ********************************************
              sw == null: True
              guid: 8d741920-aee1-47fa-9a7a-924ffa446e3a
              
              ********************************************
              OnStateChange: DataLoaded
              ********************************************
              sw == null: True
              guid: c5d7b460-8fd3-4a0b-9163-83755bb67396
              
              ********************************************
              OnStateChange: DataLoaded
              ********************************************
              sw == null: True
              guid: 8d741920-aee1-47fa-9a7a-924ffa446e3a
              
              ********************************************
              OnStateChange: DataLoaded
              ********************************************
              sw == null: True
              guid: 170067b5-130b-4aa8-a635-e3de8fc2ff0e
              
              ********************************************
              
              ********************************************
              OnStateChange: Historical
              ********************************************
              OnStateChange: Historical
              ********************************************
              sw == null: True
              sw == null: True
              guid: 170067b5-130b-4aa8-a635-e3de8fc2ff0e
              guid: c5d7b460-8fd3-4a0b-9163-83755bb67396
              
              ********************************************
              OnStateChange: Historical
              ********************************************
              sw == null: True
              guid: 8d741920-aee1-47fa-9a7a-924ffa446e3a
              
              ********************************************
              OnStateChange: SetDefaults
              ********************************************
              sw == null: True
              guid: 64dd47fa-fe81-41f9-ae77-fae387f8e1e2
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: 64dd47fa-fe81-41f9-ae77-fae387f8e1e2
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: c5d7b460-8fd3-4a0b-9163-83755bb67396
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: 170067b5-130b-4aa8-a635-e3de8fc2ff0e
              
              ********************************************
              [COLOR="red"]OnStateChange: Terminated[/COLOR]
              ********************************************
              sw == null: True
              guid: 8d741920-aee1-47fa-9a7a-924ffa446e3a
              Last edited by poseidon_sthlm; 08-09-2018, 12:21 PM.

              Comment


                #8
                Hello poseidon_sthlm,

                This would be expected as shown in the forum post from NinjaTrader_Matthew. I'm uncertain of when the ID's are being generated in your script but in the output you can see that the ID's are unique for each instance. (Meaning State.Terminated is only called once for each instance)

                I was not able to find a question I could answer from your post. Is this just information you are wanting to share with the community?
                Chelsea B.NinjaTrader Customer Service

                Comment


                  #9
                  Ok, thanks for the help.
                  Last edited by poseidon_sthlm; 08-09-2018, 01:07 PM.

                  Comment


                    #10
                    Hello Ninja!

                    I followed your advice to use a 'globally unique identifier' (GUID) to write some values from the optimizer to a textfile with stream writer. That worked fine for a single time frame startegy. But now I also want to write values from an optimization of a multi-TF strategy (I'v added a secondary 1 Tick TF for intrabar calculations). This throws an error:
                    Code:
                        [TABLE="border: 0, cellpadding: 0, cellspacing: 0"]
                    [TR]
                    [TD="width: 1371"]Strategy 'ABC': Error on calling 'OnStateChange' method: The process cannot access the file 'C:\Users\...\filename.txt' because it is being used by another process.[/TD]
                     		[/TR]
                    [/TABLE]
                    So far I have stored the GUID:s in a Dictionary where the key is the GUID and the value is the file location to make sure that only one textfile is created with stream writer for each run in the optimization.


                    Code:
                          
                    
                    [B]// Variables[/B]
                    
                     // "Global" initialization to avoid reset in State == State.Configure on new run during optimization
                    private Dictionary<string,string> guidFileLocationDictionary     = new Dictionary <string,string>();
                    
                    
                    [B]protected override void OnStateChange()[/B]
                            {
                    
                                else if (State == State.DataLoaded)
                                {
                    
                                       //If file at 'fileLocation' doesn't exist it will create the file.
                                        if (!File.Exists(fileLocation))
                                        {
                                            sw = File.CreateText(fileLocation );
                                            sw.Close();                                                     // Close the file to allow future calls to access the file again.
                    
                                            if ( guidFileLocationDictionary != null)
                                            {
                                                guidFileLocationDictionary.Add(guid.ToString(), fileLocation.ToString() );
                                            }
                    
                    
                    
                    
                    [B]protected override void OnBarUpdate()[/B]
                            {    
                    
                                if ( BarsInProgress == 0 )
                                {
                                    if (IsFirstTickOfBar)
                                    {
                    
                                        if (Bars.IsFirstBarOfSession
                                            && CurrentBar > BarsRequiredToTrade)
                                        {
                                            #region isFirstBarOfSession
                    
                                            // Write only once, at beginning of last session!
                                            if ( Time[0].Date.CompareTo(Bars.ToDate.Date) == 0 )    
                                            {                             
                                                    if (SystemPerformance.AllTrades.Count > 0 )
                                                    {
                                                        // The try-catch block is used for error handling In this case it is used to ensure you only have one stream object operating on the same file at any given moment.
                                                        try
                                                        {
                                                            if ( guidFileLocationDictionary != null
                                                                && guidFileLocationDictionary.Count > 0
                                                                && guidFileLocationDictionary.ContainsKey(guid.ToString() )
                                                                && File.Exists(guidFileLocationDictionary[guid.ToString()]) )
                                                            {            
                                                                sw = File.AppendText(guidFileLocationDictionary[guid.ToString()] );    // Open the path for writing
                    
                                                                foreach( Trade myTrade in SystemPerformance.AllTrades)
                                                                {
                                                                   sw.WriteLine(textToWrite);            // Append a new line to the file
                                                                }
                                        }


                    For a multi TF strategey it seems (when I do print my dictionary to the output window) as if NT creates one instance of the Dictionary for the primary as well as for the secondary TF with the effect that GUID:s for the primary and secondary TF are stored in different objects and that I can no longer check and make sure that only one file is created for each run. How can I solve this issue?

                    / Poseidon_sthlm
                    Last edited by poseidon_sthlm; 12-14-2018, 10:51 AM.

                    Comment


                      #11
                      Hello poseidon_sthlm,

                      The error received indicates that the file is in use when it is being written to. The filename also is not unique, so it appears the strategy is writing to the same file with each optimization iteration. Since the Strategy analyzer processes optimizations in separate threads, your strategy is attempting to write to the same file at the same time. You will need to use unique file names to avoid this error.

                      I have created a modified version of the SampleMACrossover strategy that uses unique file names for each file name to ensure that the same file will not be accessed by separate optimization iterations at the same time. This example is attached. I have also tested this when adding an additional data series for intrabar granularity which will work as well.

                      Please let us know if we can be of further assistance.
                      Attached Files
                      Last edited by NinjaTrader_Jim; 12-14-2018, 03:13 PM.
                      JimNinjaTrader Customer Service

                      Comment


                        #12
                        Hello NinjaTrader_Jim!
                        Thank you for the code sample. It was very helpful and much appreciated.

                        The key here seems to be to set the path for the Stream Writer no later than OnStateChange == State.Configure and to create a filename by concatenation with the GUID, according to your advice. It now works for me.

                        There are still two small issues that one should be aware of if you try to write from an optimization with this method.
                        1) You will get one more textfile than expected. The combination with the default values of your parameters will be written twice. You just have to remember to delete the extra text file.
                        2) When you modify your strategy and compile it, you must make sure that the strategy is refershed in the Startegy Analyzer by selecting the actual strategy once again in the 'strategy drop down box', else you will get the results from the old (cahched?) strategy. When you have to add the parameter values for your optimization once again, then you can be sure that the current strategy is refreshed.

                        Kind regards
                        / poseidon_sthlm
                        Last edited by poseidon_sthlm; 12-17-2018, 08:37 AM.

                        Comment

                        Latest Posts

                        Collapse

                        Topics Statistics Last Post
                        Started by mmenigma, Yesterday, 03:25 PM
                        1 response
                        11 views
                        0 likes
                        Last Post NinjaTrader_Gaby  
                        Started by kujista, Today, 05:44 AM
                        0 responses
                        7 views
                        0 likes
                        Last Post kujista
                        by kujista
                         
                        Started by ZenCortexCLICK, Today, 04:58 AM
                        0 responses
                        9 views
                        0 likes
                        Last Post ZenCortexCLICK  
                        Started by sidlercom80, 10-28-2023, 08:49 AM
                        172 responses
                        2,281 views
                        0 likes
                        Last Post sidlercom80  
                        Started by Irukandji, Yesterday, 02:53 AM
                        2 responses
                        18 views
                        0 likes
                        Last Post Irukandji  
                        Working...
                        X