Net Profit in 3D chart is wrong... very wrong

[tgn55]

I have noticed several times, after running an optimization, the Summary View, Trades view etc... all show a Net Profit for my strategy of say $16,000.
Yet when I look at a 3D surface plot (a wonderful feature)... it shows a max net profit of only about $5,000... a huge difference! Another instance... net profit nearly $12,000 ... 3D chart shows about $2,200!

Something seems seriously wrong with the calculation of the value to plot on the Z-azis! ... Either that or the values shown on the Z-axis are out of whack.

My strategy has 5 parameters... the issue seems only to manifest when I select Parameter 2 to create the 3D view. All looks fine in the various 2D plots...

Hope someone can help ... or is this a bug?
Cheers,

[NinjaTrader_Jesse]

Hello tgn55,

Thank you for the post.

I wanted to check, are you able to see this happen also with the sampleMACrossOver strategy? If you are can you provide the steps to see the difference you had described? If you are unable to see this happen with the sample because it has two parameters, it would be helpful if you can provide a sample of the parameters you have used so we can test in a test strategy.

Could you also take a screenshot showing the results compared with the graph?

I look forward to being of further assistance.

[tgn55]

Hey Jesse... I've given this issue a lot of thought - and I think I may have a better handle on it now. I'll try to explain...

The 3D chart obviously shows a 2D slice through (in my case) a 5D space (5 parameters).
The question then is, given any pair of selected parameters (X and Y axis on the surface plot), what value is used for the surface plot value... there will be many to pick from depending on the optimisation parameter range of the 3 unselected variables.

I had assumed that NT would select the highest of all the candidates (say I am optimising on net profit) ... but ... what if it doesn't do that?
Maybe it uses the average? Or something else. I have no insight... only the developers (or you, perhaps ;-)) can enlighten me... but I suspect that is why I am seeing values way less than the max across ALL parameters at any point.

I don't think looking at strategies with only two parameters is going to shed any light on this... the issue simply doesn't and can't exist in that scenario.

I have attached a screenshot demonstrating this... a relatively mild case, where the plotted max is about 70% of the actual max... as noted in my initial post, I've seen cases where it is only about 20%

If you want to test yourself, you will need a strategy with probably 4 or more parameters... probably none of the sample strategies have that.
I don't really want to share my strategy...

Hope this helps...

[NinjaTrader_Jesse]

Hello tgn55,

Thank you for the reply and image.

I don't think looking at strategies with only two parameters is going to shed any light on this... the issue simply doesn't and can't exist in that scenario.

Thank you for confirming that.

If you want to test yourself, you will need a strategy with probably 4 or more parameters... probably none of the sample strategies have that.
I don't really want to share my strategy...

If you would like compose a small sample with the number of inputs which is needed to see the problem, we can explore that further together. Our support won't ask for your specific strategy in nearly all cases so you can keep your logic secret, I would mainly just need to see something that simulates the problem in whatever logic that equates to. I tried adding more parameters to the sampleMACrossOver, however without adding more trading logic which uses the parameters that does not help in reproducing the problem.

I am also unsure on what the calculations are that are used internally with this tool, but with a sample of the problem that we can explore I can provide that to development to gain more insight.

I look forward to being of further assistance.

[tgn55]

Thanks Jesse...

I did some more playing... it looks to me like it is plotting the average of the collection of values from the unselected variable. Confirmed this by using my same strategy, just reduced the number of optimization runs, exported grid to Excel, ran a pivot over it, showing average values of the free variable. They match pretty closely to the values shown in the 3D plot.

A small issue remains... and not too important. It is really hard to read values of the 3D plot, as the Z-axis labels don't line up very well with the axis tickmarks! (And are weird values to boot... but that's a whole different agenda;-)

I've attached another screenshot, and the associated data from Excel, as images since I can't upload .xlsx docs...

Anyway, my main issue is resolved... when I saw huge apparent discrepancies, I did wonder if there was some serious error... but my faith has been restored!
Just need to understand how the 3D chart works...

Hope this helps someone else. Maybe a note could be added to the documentation ??

PS The 3D chart is just fantastic for evaluating strategy parameters... I really love it. Now I can use it with more confidence!

[NinjaTrader_Jesse]

Hello tgn55,

Thank you for providing the update and details.

A small issue remains... and not too important. It is really hard to read values of the 3D plot, as the Z-axis labels don't line up very well with the axis tickmarks! (And are weird values to boot... but that's a whole different agenda;-)

I will put in a feature request here to see if the text and other items can be cleaned up to be more readable.

Hope this helps someone else. Maybe a note could be added to the documentation ??

I will also put in a request to see if the general guidelines behind how this calculates can be documented in more detail, I believe this would benefit everyone.

I look forward to being of further assistance.