Problems with Optimizer

[Stephan123]

Hallo,

I'm running an optimization on a strategy and the procedure
is interrupted after only 50% of the parameter variations.
Is this a known bug?
I got aware of it because I save every single result to a file
and therefore I can see if the procedure was complete.

[NinjaTrader_AdamP]

Stephan,

Could you possibly provide me with an example strategy that has this behavior as well as the settings that you use in your optimization so I may replicate this?

Please let me know if I may assist further.

[Stephan123]

Stephan,

Could you possibly provide me with an example strategy that has this behavior as well as the settings that you use in your optimization so I may replicate this?

Please let me know if I may assist further.

Meanwhile I could identify the problem:
Interruption of the optimization procedure only occurs when
executing the userdefined function for writing data to a file.
The problem is caused by two processes operating on the same file.
But: I don't understand why it actually occurs.
Here is the code:

Within OnTermination a function is called.
This function is defined within UserDefinedMethods.
I have reduced the function to the syntax that causes the issue:

System.IO.StreamWriter sw; 
            string kopfzeile_1 = "";
            string kopfzeile_2 = "";
            
            int n_param = this.Parameters.Count; 
            PropertyInfo[] propertyInfos;
            
            propertyInfos = t.GetProperties();

if (File.Exists(path) == false) 
            {
                sw = File.AppendText(path);
                sw.WriteLine(kopfzeile_1);
                sw.WriteLine(kopfzeile_2);
            }
else
            {
                sw = File.AppendText(path); 
            }

if (sw != null)
            {
                sw.Dispose();
                sw = null;
            }

The function is run everytime an particular backtest within the optimiuzation procedure
is finished. Performance results are written to a txt file.
Why does the problem occur and how can it be avoided?
Might it have to do with multithreading?
There is a streamwriter sample indicator available.
It obviously uses try and catch to avoid the above issue. When I would apply this, some of the data probably would just be hindered from being saved to the file. Which is not what I would want.

Thanks in advance, Stephan

[NinjaTrader_AdamP]

stephan,

Optimization is run multi-thread so its possible that it will access a file at the same time if you are using the same file for all output of every strategy. You could possibly get around this by naming your output files based on the input parameters of the optimization run.

For example, suppose we have a strategy that enters longs only when 2 SMA's cross the appropriate way. Thus, we would have 2 inputs, a slow period and a fast period.

So if we run :

Slow period = 6 to 10 in steps of 1
Fast period = 1 to 5 in steps of 1

Then our first file would be named "SMAoutputF1S6", second "SMAoutputF2S6", ect.

Another solution is to disable multi-thread during optimization. This is unsupported, however in your initialize method if you set MultiThreadSupport = false; then it will disable multithreading in optimization runs.

The only other work around would require managing file access in a multi-threaded environment, which can be complicated depending on your level of familiarity with it and is also unsupported.

Here's a link to "checking if a file is in use" : http://stackoverflow.com/questions/8...file-is-in-use

However, its possible when you check if a file is in use and find its not, two processes could still attempt to access it at the same time.

Please let me know if you require additional assistance.

[Stephan123]

Thank you-
disabling MultiThreading solved the issue.

[NinjaTrader_AdamP]

Stephan,

Please keep in mind this will result in a performance reduction.

Please don't hesitate to contact us should you require additional assistance.

[Stephan123]

Stephan,

Please keep in mind this will result in a performance reduction.

Please don't hesitate to contact us should you require additional assistance.

I'm aware of it. To my surprise the optimization wasn't significantly slower after disabling
Multithreading.

May be it's of interest for NT to provide some sample code how to
avoid this kind of problems using MultiThreading.

[NinjaTrader_AdamP]

Stephan,

I believe this has some information on using locks and such.

http://stackoverflow.com/questions/640122/is-there-a-global-named-reader-writer-lock

Ill mention your suggestion to our product manager.

Please let me know if I may assist further.

[kucharek]

Hi AdamP,

I have the same problem.
What you mean by "files" in your 11-30-2011, 03:32 PM thread?:

Then our first file would be named "SMAoutputF1S6", second "SMAoutputF2S6", ect.

Should I prepare second SMA indicator and named it "SMA2"? And then use SMA and SMA2 in my strategy code?

Please explain.

[NinjaTrader_RyanM1]

Hello kucharek,

What problem are you having exactly? Adam's suggestion was for providing unique filenames per distinct optimization run.

[kucharek]

Hi Ryan,

I have problem with Walk Forward fr.

Simple strategy (less parameters, optimization options) works ok - it gives results BUT Ninja Trader freezes when I want to save these results. I have to restart NT then.

More complicated strategy (more parameters, optimization options) freezes during work forward test calculation. NT restart is also required.

[NinjaTrader_RyanM1]

Thanks kucharek.
Can you please use Help > Mail to support feature so we can see any messages returned. Include a link to this thread and the date time when you most recently saw this. Thanks for sending in these files.

[kucharek]

Ok, I will do that.