Optimization: Profit Factor

[akvevo]

Hi,

I am keen to hear opnions on this issue I am seeing.
I am running a simple optimization on 2 years's worth of Tick Bars.
(CL: 2000 tick bars for instance).

If I "Optimize On" Net Profit, I see an output full of results.
What I mean by by "full results" if list of the "best # results" each with several hundred trades, So far so good.

However, If I chooe to Optimize on like "Profit Factor", each optimzation run as 1 or 2 trades. Its as if it literally stops (or doesnt take any more trades) when it reaches a maximum profit factor, which is obviosuly meaningless as an optimzatoin technique.
But perhaps I am missing something.

Can someone shed any light on this dilema ?

Ak

[NinjaTrader_Jim]

Hello akvevo,

Thanks for opening the thread.

Optimize On lets you select the performance metric that the strategy is to be optimized against.

Optimizing on Profit Factor will configure the strategy to use parameters that have the greatest return per trade. This can restrict the number of trades that the strategy can place.

Information on optimization backtests, along with a tutorial video that provides further explanation can be found in the help guide here: https://ninjatrader.com/support/help...a_strategy.htm

If you have any additional questions, please don't hesitate to ask.

[akvevo]

Thanks Jim,

"This can restrict the number of trades that the strategy can place" concerns me.
The help you page you provided doesnt mention the logic thats used.

The optimization should take every trades bound by the optimizale parameters.
If I select "Max%Profitable" as my "Optimize On", then I dont expect the number of trades to be restricted to 1 trade simply because the first winning trades yields an Max%Profitable of 100%.

Do you agree ? Thats the problem I am seeing.

[NinjaTrader_Jim]

Hi akvevo,

I could have been clearer with my claim.

I was referring to the number of trades taken by the strategy may be decreased if the optimization backtest discovers some settings that take more profit through less trades.



Getting back to your issue with backtesting, could you provide me answers to the following questions so I can reproduce this behavior on my end?

Do you see results when running the same test on the SampleMaCrossOver strategy in NinjaTrader with the same settings as your strategy?

Can you reproduce this behavior with a Data Series type other than tick bars?

Is your strategy a multi instrument or multi time frame strategy?

Do you receive an error on screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?

I look forward to being of further help.

[akvevo]

Thanks Jim,

I ran tests using SampleMaCrossOver on CL 08-17 for:

- daily bars since 2005
- tick bars (2000) for the last 2 years.

I seems like the issue is visible when backtesting using tick bars, not my strategy.
If you run the SampleMaCrossOver on CL 08-17(2000 tick bars) and:

- optimize on "max net profit": i see full results with seveal hundred trades.
- optimize on "profit factor": each run gives zero trades
- optimize on "max avg profit": each run gives zero trades

I see no errors.
Ive attached a couple of screenshots.

Can you replicate?

[NinjaTrader_Jim]

Hello akvevo,

Thanks for providing me that information.

I am able to see the same results, as well. I think the issue is that there is not enough tick historical tick data available to drive the backtest.

You can check how much historical tick data you have from the Historical Data window. This window can be accessed from the Control Center by navigating to Tools > Historical Data.

When I did not see any trades in the backtest, I switched to the chart under the Strategy Analyzer's Display Selector. I did not see any bars as the backtest appeared to abort and after checking the historical data I had downloaded for the CL, I noticed there was not any available that could drive the test.

If your provider does not offer enough historical tick data, you might want to consider downloading the tick data from a third party and importing the data manually for your backtests.

Importing data and further documentation on Historical Data can be referenced here: https://ninjatrader.com/support/help...ta_manager.htm

I've attached a video testing with a smaller time frame which showed results: https://www.screencast.com/t/n8OF92siV4kI

Please let me know if I may be of further assistance.

[akvevo]

Thanks Jim,

My provider is Continuum.

I dont think the "issue is that there is not enough tick historical tick data available to drive the backtest".

I have enough tick data, I can backtest on 2 years back on tick data, and can Optimize on Max Net Profit - so the data is there. (I've also double checked the data in "Historical Data" and I see tick data going 2 years on CL. - I am attaching a screenshot showing tick data back to 2015).

The fact that I can optimize and get full results if I optimize on "Max Net Profit" shows that the data is there. Its when I switched to other metrics like "Profit Factor", there are no trades.

So, i suspect something else is going on

[NinjaTrader_Jim]

Hello akvevo,

Thanks for verifying that information.

I may wish to arrange a call with you to look into this further. Before we cross that bridge, could you confirm that you are on the latest version of NinjaTrader 8? The current release is 8.0.7.1. You can download the installer here: http://ninjatrader.com/PlatformDirect

Could you also confirm if performing the test described in my video will show results? Please test with a new instrument as well so we can confirm that we are downloading a consistent chunk of historical data that can be used for the back test.

https://www.screencast.com/t/iSnPB8r9dln

If you are still able to encounter the issue on the latest version following the same steps I have in the video, please write in to platformsupport[at]ninjatrader[dot]com with the thread URL and the text "Attention Jim" so we can arrange a time for a remote support session.

[akvevo]

Thanks Jim,

You are indeed correct.
Performing the test you described does provide results.
I was already using the latest version.

I am running additional tests to try and identify why some Optimization run on Profit Factor return 0-3 trades, while some Optimizing on Net Profit can return several hundred trades.

[NinjaTrader_Jim]

Hello akvevo,

If you can come across a scenario to reproduce this on a new instrument using the SampleMACrossover strategy so I can do the same on my end, I could provide further explanation on the results.

I would also suggest to test on your chunk of historical data with multiple backtests starting from a point where you know you get adequate results and then adding more days to load. Once you have identified a day that is providing questionable results, I would recommend testing just this day, and then testing around this day to see if there is an issue with a piece of historical data that is recorded.

Please let me know if I can assist you further.