Optimizer Results Sometimes Corrupted

[jmondanaro]

I was Experimenting with the Genetic Optimizer when I noticed I had an incorrect input parameter on my run. So I fixed the parameter and re-ran the optimization. However, one of the results was corrupted. I've attached a screen shot that show the top 10 results from the optimization but the details for the individual entry #2 are completely different. The other 9 entries are correct. I haven't seen this very often and I cannot figure out how to recreate it consistantly.

Jason

[jmondanaro]

Sort of reproducable

I have managed to get this to reproduce a few times in the following scenario.

After receiving your results and waiting for the screen to become responsive again (really this also needs to get worked on) switch the analyzer to look at the Analysis view with the graph set to Cumulative Net Profit. Now click on a different result set. Chances are it will be incomplete/corrupted. Switching to another data set and back wont fix it. You need to switch the view off of the analysis view, say back to summary and switch results a few times before going back and your results maybe reloaded in their correct form.

Also, any reason why switching to view a result is so incredibly slow?

-Jason

[NinjaTrader_PaulH]

Hi Jason,

Thanks for your posts.

We appreciate your participation in the beta test program and your feedback will help us improve the product. We just released beta 10 and while I am unsure if any of the updates will change your observations there were 6 separate issues resolved in the optimizer in this release, it would be better if you could re run your tests here after updating to the latest release.

Thanks again and we look forward to your update.

[NinjaTrader_Matthew]

We have made some changes in this area in the most recent release 8.0.0.10 which was just announced a few minutes ago.

In particular, these two items could potentially be related to the issue you are experiencing:

Defect
9368
Strategy Analyzer
Optimization results sometimes differed between iterations of the same test if the strategy hosted additional data series

Change
9491
NinjaScript, Strategy Analyzer
The IsInstantiatedOnEachOptimizationIteration property is now set to true by default. Note: Strategy Optimizations could run slower. Please see Help Guide entry IsInstantiatedOnEachOptimizationIteration to take advantage of performance benefits.

If you have some time, can you please re-run your scenario in B10 and report back?

You can find info on this release here: http://ninjatrader.com/support/forum...ad.php?t=84041

[jmondanaro]

I'm sorry but the Analyzer seems to go into space and never execute. I saw this happen infrequently in the 8.0.0.9 version, but it's 100% in v10. I hit run, the CPU goes up to about 30% and just sits there and the strategy doesn't seem to execute more than the OnStateChange State=Configure and that only once so it never finishes a single scenario to move onto another. Aborting doesn't ever return either. I need to kill NinjaBroker and restart it completely.

I wonder if this is because I also tend to turn on Optimize Data Series using Ticks. Using the built in SMA Strategy it ran fine without the data series optimization but running it again with it on and it seems to have gone into space (memory doesn't really change but CPU stuck at 30% utilization) if it finishes and comes back I will also send you my strategy to see if my code or the number of parameters I am using are overloading things.

[NinjaTrader_Matthew]

I'm sorry but the Analyzer seems to go into space and never execute. I saw this happen infrequently in the 8.0.0.9 version, but it's 100% in v10. I hit run, the CPU goes up to about 30% and just sits there and the strategy doesn't seem to execute more than the OnStateChange State=Configure and that only once so it never finishes a single scenario to move onto another. Aborting doesn't ever return either. I need to kill NinjaBroker and restart it completely.

I wonder if this is because I also tend to turn on Optimize Data Series using Ticks. Using the built in SMA Strategy it ran fine without the data series optimization but running it again with it on and it seems to have gone into space (memory doesn't really change but CPU stuck at 30% utilization) if it finishes and comes back I will also send you my strategy to see if my code or the number of parameters I am using are overloading things.

Try setting the following property to false during your Strategies State.SetDefaults

	IsInstantiatedOnEachOptimizationIteration = false;

Then close the strategy analyzer and open it again, running your strategy again.

Does it return more quickly with this property turned off?

[jmondanaro]

Hello.

A Few things. Adding the parameter you indicated doesn't seem to make a difference. Also, now in version 8.0.0.10 strategies that call a drawing function seem to fail. I get a NullReference Exception during OnBarUpdate when I try drawing a line. This is because there is no ChartControl, at least that is what VStudio's debugger finds inside @line.cs. This seems like a unintended problem with the new release?

Second, It stall takes 5 minutes or so for the first iteration or initial load of the strategy to run. Then once that has finished, I see that 4 new instances are started (probably because I have a 4 core machine?) If I try to Abort during that first 5 minutes the system goes Zombie. The UI is responsive unlike 8.0.0.9, but the Run button never comes back and the CPU never stops, I let the system run for 2 hours like this. Here is an Output from my strategy. Since your are running multiple threads I added a GUID Id to each instance that is re-initialized when Configure is called.

So you can see at 10:02:00 instance cfa3a starts and the system churns for about 5 minutes. No data is loaded and that root instance never does a thing. Finally you see the 4 new instances created at 10:07:49 and they all seem to work (except for the new OnBar blow up do to the drawing issue I noted) The 4 "real" instances seem to process very fast and I confimred that by commenting out the Draw lines and everything flew after the first 5-6 minutes of churn.

00000000-0000-0000-0000-000000000000][SetDefaults]
[00000000-0000-0000-0000-000000000000][Configure]
[cfa3a746-2388-4302-a0b3-d12a62900d81][2016-03-21 10:02:00]
[cfa3a746-2388-4302-a0b3-d12a62900d81][0/False/21/1.5/8/4/15:25:00/09:35:00/2/2 (Period/AggressiveTrendRequired/KeltnerPeriods/KeltnerWidth/paramProfitTarget/paramStopLoss/TradeDayEnd/TradeDayStart/TradeRisk/TrendBreakBarCount)]
[00000000-0000-0000-0000-000000000000][SetDefaults]
[00000000-0000-0000-0000-000000000000][SetDefaults]
[00000000-0000-0000-0000-000000000000][SetDefaults]
[00000000-0000-0000-0000-000000000000][SetDefaults]
[00000000-0000-0000-0000-000000000000][Configure]
[00000000-0000-0000-0000-000000000000][Configure]
[00000000-0000-0000-0000-000000000000][Configure]
[00000000-0000-0000-0000-000000000000][Configure]
[66acfd8e-fc79-4075-a6e8-feaa38ba4499][2016-03-21 10:07:49]
[d7aacf30-6ca2-4742-b40b-f8fb18a6e9c6][2016-03-21 10:07:49]
[13395566-3659-4b4f-b5be-c96bf8d7897f][2016-03-21 10:07:49]
[8102df1d-fd26-4679-9c95-8171de847426][2016-03-21 10:07:49]
[66acfd8e-fc79-4075-a6e8-feaa38ba4499][2000/False/115/6.5/54/15/15:25:00/09:35:00/400/7 (Period/AggressiveTrendRequired/KeltnerPeriods/KeltnerWidth/paramProfitTarget/paramStopLoss/TradeDayEnd/TradeDayStart/TradeRisk/TrendBreakBarCount)]
[d7aacf30-6ca2-4742-b40b-f8fb18a6e9c6][2000/True/136/7.5/14/8/15:25:00/09:35:00/400/8 (Period/AggressiveTrendRequired/KeltnerPeriods/KeltnerWidth/paramProfitTarget/paramStopLoss/TradeDayEnd/TradeDayStart/TradeRisk/TrendBreakBarCount)]
[13395566-3659-4b4f-b5be-c96bf8d7897f][2000/True/114/2.5/48/5/15:25:00/09:35:00/400/9 (Period/AggressiveTrendRequired/KeltnerPeriods/KeltnerWidth/paramProfitTarget/paramStopLoss/TradeDayEnd/TradeDayStart/TradeRisk/TrendBreakBarCount)]
[8102df1d-fd26-4679-9c95-8171de847426][2000/False/177/7.5/60/14/15:25:00/09:35:00/400/9 (Period/AggressiveTrendRequired/KeltnerPeriods/KeltnerWidth/paramProfitTarget/paramStopLoss/TradeDayEnd/TradeDayStart/TradeRisk/TrendBreakBarCount)]
[13395566-3659-4b4f-b5be-c96bf8d7897f][DataLoaded]
[8102df1d-fd26-4679-9c95-8171de847426][DataLoaded]
[d7aacf30-6ca2-4742-b40b-f8fb18a6e9c6][DataLoaded]
[66acfd8e-fc79-4075-a6e8-feaa38ba4499][DataLoaded]
[13395566-3659-4b4f-b5be-c96bf8d7897f][Historical]
[8102df1d-fd26-4679-9c95-8171de847426][Historical]
[d7aacf30-6ca2-4742-b40b-f8fb18a6e9c6][Historical]
[66acfd8e-fc79-4075-a6e8-feaa38ba4499][Historical]
Strategy 'TestStrategy': Error on calling 'OnBarUpdate' method on bar 1148: Object reference not set to an instance of an object.
[d7aacf30-6ca2-4742-b40b-f8fb18a6e9c6][Terminated]
Strategy 'TestStrategy': Error on calling 'OnBarUpdate' method on bar 244: Object reference not set to an instance of an object.
[66acfd8e-fc79-4075-a6e8-feaa38ba4499][Terminated]
Strategy 'TestStrategy': Error on calling 'OnBarUpdate' method on bar 1358: Object reference not set to an instance of an object.
[8102df1d-fd26-4679-9c95-8171de847426][Terminated]
[cfa3a746-2388-4302-a0b3-d12a62900d81][Terminated]
Strategy 'TestStrategy': Error on calling 'OnBarUpdate' method on bar 123: Object reference not set to an instance of an object.
[13395566-3659-4b4f-b5be-c96bf8d7897f][Terminated]

[NinjaTrader_MichaelM]

Hello jmondanaro,

So that I may further investigate this behavior in order to submit a bug report or provide you with a resolution, please feel free to email your strategy to [email protected] with the subject line: "ATTN: Michael M http://ninjatrader.com/support/forum...d.php?p=453466" at your convenience.

Thank you in advance!