MAE and MFE clearly wrong

[tgn55]

I can see reports about MAE/MFE being wrong going back to 2007 ;-( And regularly reported since then...

I have simple strategy, a MA crossover, only ever trades 1 contract... and the errors are clearly NOT related to Bid/Ask spread or any other such artifact. They are simply, and very evidently, completely rubbish in many instances.

Clear example: Multiple trades, some winners, some losers,and MAE/MFE are both reported as the SAME VALUE !!! Seriously, that is so wrong it is not funny. And no appeals to Bid/Ask etc, or indeed any other factor, alter the fact that the CODE IS CLEARLY WRONG !!!

Please see the marked up Trade Performance Report that highlights the obvious idiotic errors. In fact, there is a pattern... in this case, every other trade's MAE/MFE is bogus.

Prior to running this report I performed a Repair Database. After the Exit on Session Close trade... things just go down the toilet...

When can we get past "show it in Sample MA Crossover". Seriously... the reporting engine does need to work on more complex strategies... albeit my strategy is in essence fundamentally the same... go long on crossabove, go short on crossbelow. Yes I have some additional logic to filter trades.. but the basic trade execution is THE SAME pattern as the much-quoted Sample MA thing.

Regards,
T.

[NinjaTrader_Kate]

Hello tgn55,

Thank you for your reply.

We do ask if you can reproduce it using the Sample MA Crossover, because that's a known quantity with which we can often replicate. It also helps rule out something in the strategy being tested being the cause of an issue. You didn't mention whether you were able to reproduce the same behavior of the MAE and MFE being the same using that strategy or not.

I would be happy to look further into your particular situation. Please send me your database, log, and trace files so that I may look into what occurred.

You can do this by going to the Control Center-> Help-> Email Support

Expand 'Other files' and enable 'Database file'.

Please reference the following ticket number in the body of the email: 2305087 ATTN Kate W.

Thanks in advance; I look forward to assisting you further.

[tgn55]

Thanks Kate.

After I posted this, I had an afterthought... while the identical MAE.MFE values are obviously wrong, I do suspect it is the result of something weird in my trade/execution data.
I am still frustrated by the anomalous backtest results NT often throws up... this one is annoying but minor.

My MAJOR beef is actual profit/loss results are sometimes COMPLETE nonsense... but the "fix" (as advised by NT)... run the backtest in a new Analyzer tab/window. Which, with the SAME data, the SAME strategy, the SAME parameter settings... gives a COMPLETELY different set of trades. All that changed was re-running in a new window. What the !@#$????

No explanation... just do it. OK... and which set of trades/results is the user supposed to have confidence in ??

Anyway, I stray off topic... apologies. Will pursue this significant gripe in a new thread... it just seems related that sometimes, NT seems to get quite confused about trade history/performance... so... they may have related root causes - perhaps.

Cheers,
T.

PS Will email trace files when I get a few spare minutes... but, I strongly suspect it is at least partly data related...

[Jim-Boulder]

Kate - (or anyone else who may know) - what is the status of these reporting errors in MAE / MFE? I just ran a YTD trade Performance summary and I see some trades where the MAE or MFE are +3500 or more. I spot checked multiples of them to insure that these were not held overnight. They were not. in some cases the trade only last 1 3 minute bar in duration, so a $3500 excursion is not possible (I verified on the chart). While these are sim trades, I am relying on the trade performance data and specifically the MAE and MFE to evaluate these strategies, and I am questioning whether these 2 metrics are even reliable.

Can you update the current status of this issue?

While I have the podium, assuming they are not reliable or at least we don't know if they are, has anyone developed an "internal" trade tracker inside of a strategy that tracks the MAE and MFE equivalents (and perhaps write them out to a file)? I'm hoping someone has written a simple work around until the official one gets fixed (I mean a user, or a support person as a temp fix).

[NinjaTrader_Kate]

Hello Jim-Boulder,

Thank you for your inquiry.

We are currently working with tgn55 to isolate his particular issue. In your case, my initial idea would be that the trade pairing is off due to the position of the account not being flat at the beginning of the day the Trade Performance was run from. If you choose a day on which you know you had no open positions to start from, do you see inaccurate MAE/MFE calculations?

As far as internally tracking within a strategy, it looks like this indicator that's publicly available on our user app share may contain something like what you're looking for:

https://ninjatraderecosystem.com/user-app-share-download/cci_forecaster_de-v7-2/

Please let us know if we may be of further assistance to you.

The NinjaTrader Ecosystem website is for educational and informational purposes only and should not be considered a solicitation to buy or sell a futures contract or make any other type of investment decision. The add-ons listed on this website are not to be considered a recommendation and it is the reader's responsibility to evaluate any product, service, or company. NinjaTrader Ecosystem LLC is not responsible for the accuracy or content of any product, service or company linked to on this website.

[tgn55]

Been occupied with other matters for a while, but, just throwing in this...

The "solution" of starting the report when flat really does not cut the mustard.
I have strategies running non-stop on multiple instruments... there is NEVER a time when all instruments/positions are flat! Asking the user to work around this is, well... pretty ordinary at best.

Am I missing something? I appreciate that there may be some weird circumstances that make matching a little less trivial than it first appears, but I can't find a reason why, given a trade,, having an entry and an exit, should not ALWAYS begin with an entry, as mentioned above.

Surely, changing the reporting code to STRICTLY enforce this would resolve all the issues around MAE/MFE, weird PnL results etc

T.

[jtrading82]

Any update on this? I traded 1 ES 03-21 contract today and the MAE is reported incorrectly.

Entry time: 1/20/2021 11:02:56 AM
Exit time: 1/20/2021 11:08:15 AM

MAE should be 0.75pts, shows as 0.5pts

Please check on your end before asking me to send screenshots, etc. as I first reported this in original post's thread back in Apr 2018. It also seems many other users have been reporting this for years.

[NinjaTrader_Kate]

Hello jtrading82,

Thank you for your note.

To clarify, was this a single trade? Meaning the entry quantity was 1 and the exit quantity was 1 - no scaling out occurred? What time zone are you located in so that I may best try to reproduce?

What were the entry and exit prices for the trade?

Thanks in advance; I look forward to assisting you further.

[jtrading82]

Yes, just 1 qty, 1 round-trip trade, it is as basic as it gets. There should be no excuses.

Entry time: 1/20/2021 11:02:56 AM PST BUY 1 contract @ 3843.0
Exit time: 1/20/2021 11:08:15 AM PST SELL 1 contract @ 3843.25

[NinjaTrader_Kate]

Hello jtrading82,

Thank you for your reply.

In order to assess whether this is a example of a known limitation or something that we need to run past our development team, we'll need to review your log, trace, and database files.

You can send these by going to the Control Center-> Help-> Mail to Platform Support.

Ensure to check 'Database file' and ' Log and trace files'.

Please reference the following ticket number in the body of the email: 2305087 ATTN Kate W.

Thanks in advance; I look forward to assisting you further.

[jtrading82]

Thanks Kate and sent. Can you also list known limitations on MAE/MFE so we can be aware of these limitations?

[NinjaTrader_Kate]

Hello jtrading82,

Thank you for your reply.

I didn't receive all the necessary files I've requested, so I've followed up with you via email on that.

As far as known issues with MAE/MFE calculations go, here's the major ones:

• Scaling in and scaling out
• Partial fills
• Reporting from provider
• Not enough executions being loaded from DB, (only loads 3 days back) resulting in wrong 'execution' position tracking in real-time
• Missing executions, e.g. 2 days ago 2 long ES, you don't connect 1 day ago with NinjaTrader and flatten using the mobile client, then login today with NinjaTrader. From an execution perspective Ninjatrader still thinks you are long 2 because it never gets the flattening execution.
• When internal position tracking is off is that the entry execution would register as an exit and therefore not track min/max price. If this occurs, NinjaTrader will set Entry.MinPrice and Entry.MaxPrice then equal to the Exit price which results in MAE/MFE of Entry.Price - Exit.Price.

It is worth noting that there is no overall concept of 'Trade' in NT8. Trades are built from pairing executions. The entry execution is tracking low/high price seen while in position.

Thanks in advance; I look forward to assisting you further.

[butt_toast]

Has this been fixed?

I emailed support on March 9 of this year and even included a youtube video showing an example using micros.

The number of contracts was being multiplied against both MAE and MFE.

The answer I was given was "since we don't have the full range of motion in the Trade Performance window, we don't know the highest high and the lowest low to properly calculate MFE/MAE under all scenarios."

I had to drop it because I felt bad that I was putting the NT representative in a tough spot and I know what it's like to deal with customer questions that you're unable to address when the company is at fault due to calculation errors (specifically regarding trading software).

However knowing MAE is one of the single most important stats for any trader.

I've been paying more than $40/mo for third party software for this state ALONE and if NT would correct this simple error it would be very nice especially considering the cost of a lifetime license.

I'd rather not post the link in here as it may show personally identifiable information on it but if someone from NT looks at emails with this subject line: Re: MAE Question {3003218} you should find the video.

[NinjaTrader_Kate]

Hello butt_toast,

Thank you for your reply.

I'm not aware of any recent changes to MAE/MFE calculation in the last several releases. Is there a specific feature request number or other tracking number you wish to inquire about? Generally speaking, there are limitations within the platform in regards to calculating these values that have already been touched on in this thread.

Please let us know if we may be of further assistance to you.

[butt_toast]

Hello butt_toast,

Thank you for your reply.

I'm not aware of any recent changes to MAE/MFE calculation in the last several releases. Is there a specific feature request number or other tracking number you wish to inquire about? Generally speaking, there are limitations within the platform in regards to calculating these values that have already been touched on in this thread.

Please let us know if we may be of further assistance to you.

Thanks for the reply.

I wouldn't frame this as a "feature request" as the software is already attempting to have this feature (MAE/MFE). This is a bug fix.

To reiterate what I wrote in here today "The number of contracts was being multiplied against both MAE and MFE". For example in the video I sent via email the number of contracts was being multiplied (I suspect - you can see why in the video) by the MAE value so an MAE of 2 was being displayed as 4.

Regarding the tracking number, there is a number that support added to the subject line I pasted in the comment you replied to.

Let me know if you can't find it and I'll email the link to the unlisted youtube video again.

For the record, I do not expect this bug to be fixed right now. It would be really nice if the devs would look into it.

I already had to purchase 3rd party profiling addons, 3rd party footprint, 3rd party relative volume, and I also pay for third party analytics because of the issue with MAE, so my expectations are extremely low for this to be resolved.