Can You Chart A Walk Forward Optimization?

[Tomhgriff1]

Hi,

As I understand it, the walk-forward optimization results show the total of all the optimized out of sample results in the top line. And then the subsequent lines below just show which iteration performed best according to your metrics.

My question is, how can I see an equity curve or list of trades for the total at the top. Without this it doesn't really tell me anything useful. It just says the system made or lost x. I have no idea how smooth the equity curve was etc. No way of looking at the trades to check execution accuracy in the backtest. Even if theres a way to pull it into excel to made a chart myself would be useful but I only seem to see one line at the top with an end total.

Thanks
Tom

[NinjaTrader_Jim]

Hello Tom,

Thanks for your post.

Parameters are determined from optimizing the In Sample Optimization Period and then those parameters are then used to backtest the Out Of Sample Test Period. As such, each result listed is a different backtest, using different parameters. Since they are separate backtests, the additional displays are not available for the merged results.

I have submitted a request on your behalf to be able to see other displays for the merged backtest results in the entire walk forward optimization. I will update this thread when I have received a ticket ID.

SFT-632

Feature requests are implemented based on client feedback, feasibility, discretion from Product Management, and the ongoing tasks of the Development team. As such, we cannot offer an ETA or promise of fulfillment.

We look forward to being of further assistance.

[adaptive7]

Hello NT Forum and Support!

1. SFT-549
2. SFT-632
3. SFT-3152

I basically would love to have three features:
1. Cumulatie Net Profit Graph over all the trades (not only the single optimization runs)
2. Monte Carlo analysis over all the trades
3. Export all the trades at once

I see that all the functionality (Cumulative Net Graph over Trades, Monte Carlo Analysis, Export) is basically already there, it's just not working on all the trades. I am not too happy about the fact that this has been a request for years now.

Please add my vote to the topic. Maybe you can provide an update on those requests?

Thanks, best regards

[NinjaTrader_Jim]

Welcome to the forums adaptive7!

SFT-549 tracks the ability to export all trades with a single operation. I have added a vote on your behalf. Trades can still be exported from the Trades Grid of a Trade Performance window by right clicking in the grid and selecting Export. There are currently 10 votes.

SFT-632 tracks interest for the ability to use all displays in the Strategy Analyzer's Combined results. This would allow for Cumulative Net Profit graphs as per your first request. I have added your vote. There are currently 14 votes.

SFT-3275 tracks interest in being able to create Monte Carlo simulations in the Trade Performance window. I have added your vote here. There are currently 2 votes.

Please let us know if there is anything else we can do to help.

[bkonia]

I'm interested in these features as well. Where can we vote for them?

I also have another related feature request:

You can double-click on a prior walk forward optimization in the Strategy Analyzer Log, to reload its information in the main Strategy Analyzer window. However, as shown in the attached screenshot, when you expand the disclosure triangle to view the results of the individual backtests, it only displays the summary statistics for each backtest in the top pane. The middle pane, which is supposed to contain the detailed information for each backtest, such as trades, etc... contains all zero values.

Is there a setting that would enable the log to retain the detailed information for each walk forward optimization backtest? I know about the IncludeTradeHistoryInBacktest setting and I have that set to true. It does retain the trade history for the most recent backtest, but when you load a walk forward optimization from the log, the trade history is lost and all the reports show zero values. If no such setting exists, I'd like to submit a feature request for an option to log the full trade history and all reports from each backtest in a walk forward optimization.

[NinjaTrader_Jim]

Hello bkonia,

Feature requests are tracked internally. To add a vote, simply let our support staff know you would like to have a request entered. I have added votes for the feature requests noted in post #4 on your behalf.

I am not sure if the behavior you are describing would be an expected limitation that we could track with a feature/enhancement request or if this is a case that would need correction.

Could you provide some steps I can take to reproduce this with the SampleMACrossover strategy? If not possible with the SampleMACrossover strategy, please provide a custom strategy and steps I can take to reproduce.

For example:

1. Run Strategy X with Y settings using a Walk Forward Optimization
2. Close and ReOpen the Strategy Analyzer and Run strategy X with Z settings
3. Open Strategy Analyzer Log and select previous optimization made with Step 1 and view the Summary grid to see all 0's

Some direction like the above would be what I would need to reproduce on my end. Please also ensure you are on the latest version of NinjaTrader 8, Release 22.1.

If you would not like to share your custom strategy, you can email it to me at platformsupport [at] ninjatrader [dot] com. Please include the text "Attn Jim 2378992" and a link to this thread. When sharing your strategy, we ask that you remove unrelated code so we are left with a simple test case that can reproduce the issue without complex strategy logic included.

We look forward to assisting.

[bkonia]

Thank you Jim, I was able to figure it out on my own.

[jadawo]

Hi Jim, please add my vote to the proposals in post #4. These would be huge improvements. Any update on them by the way? (Since this thread is a few months old).

[NinjaTrader_Jim]

Thanks for your feedback jadawo, I have added your votes.

There is no update here. New features will be noted in the Help Guide's Release Notes page once they are implemented. SFT ID's are different that the action ID's noted in the Release Notes page, so you may reference the descriptions of added features to confirm if a feature has been implemented.

We receive many requests and do not offer ETA's or promise of fulfillment with Feature Requests. Interest is tracked and if demand is there and implementation is feasible, the feature would be considered.

Release Notes - https://ninjatrader.com/support/help...ease_notes.htm

We look forward to assisting.

[peterslane1]

Please add my vote for all of the feature requests
1. SFT-549
2. SFT-632
3. SFT-3152

Thanks Peter

[NinjaTrader_Jim]

Thanks for the feedback peterslane1, your votes have been added.

[simple_goodoboy]

Hello Tom,

Thanks for your post.

Parameters are determined from optimizing the In Sample Optimization Period and then those parameters are then used to backtest the Out Of Sample Test Period. As such, each result listed is a different backtest, using different parameters. Since they are separate backtests, the additional displays are not available for the merged results.

I have submitted a request on your behalf to be able to see other displays for the merged backtest results in the entire walk forward optimization. I will update this thread when I have received a ticket ID.

SFT-632

Feature requests are implemented based on client feedback, feasibility, discretion from Product Management, and the ongoing tasks of the Development team. As such, we cannot offer an ETA or promise of fulfillment.

We look forward to being of further assistance.

Please add my vote for SFT-632.

Thanks and when will this be fixed.

[NinjaTrader_Jim]

Thanks for your feed back simple_goodoboy, your vote has been added.

This is a feature/enhancement request, interest is tracked before changes are considered and we cannot offer an ETA.

[walter739]

Please add my vote for all of the feature requests
1. SFT-549
2. SFT-632
3. SFT-3152

Thanks Walter

[NinjaTrader_ChelseaB]

Hello Walter,

I've added your votes to SFT-549, SFT-632, and SFT-3152.

Thank you for your voice.