Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

MAE and MFE clearly wrong

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    I can see reports about MAE/MFE being wrong going back to 2007 ;-( And regularly reported since then...

    I have simple strategy, a MA crossover, only ever trades 1 contract... and the errors are clearly NOT related to Bid/Ask spread or any other such artifact. They are simply, and very evidently, completely rubbish in many instances.

    Clear example: Multiple trades, some winners, some losers,and MAE/MFE are both reported as the SAME VALUE !!! Seriously, that is so wrong it is not funny. And no appeals to Bid/Ask etc, or indeed any other factor, alter the fact that the CODE IS CLEARLY WRONG !!!

    Please see the marked up Trade Performance Report that highlights the obvious idiotic errors. In fact, there is a pattern... in this case, every other trade's MAE/MFE is bogus.

    Prior to running this report I performed a Repair Database. After the Exit on Session Close trade... things just go down the toilet...

    When can we get past "show it in Sample MA Crossover". Seriously... the reporting engine does need to work on more complex strategies... albeit my strategy is in essence fundamentally the same... go long on crossabove, go short on crossbelow. Yes I have some additional logic to filter trades.. but the basic trade execution is THE SAME pattern as the much-quoted Sample MA thing.

    Regards,
    T.
    Attached Files
    Last edited by tgn55; 04-17-2020, 09:52 PM.

    Comment


      #17
      Hello tgn55,

      Thank you for your reply.

      We do ask if you can reproduce it using the Sample MA Crossover, because that's a known quantity with which we can often replicate. It also helps rule out something in the strategy being tested being the cause of an issue. You didn't mention whether you were able to reproduce the same behavior of the MAE and MFE being the same using that strategy or not.

      I would be happy to look further into your particular situation. Please send me your database, log, and trace files so that I may look into what occurred.

      You can do this by going to the Control Center-> Help-> Email Support

      Expand 'Other files' and enable 'Database file'.

      Please reference the following ticket number in the body of the email: 2305087 ATTN Kate W.

      Thanks in advance; I look forward to assisting you further.

      Kate W.NinjaTrader Customer Service

      Comment


        #18
        Thanks Kate.

        After I posted this, I had an afterthought... while the identical MAE.MFE values are obviously wrong, I do suspect it is the result of something weird in my trade/execution data.
        I am still frustrated by the anomalous backtest results NT often throws up... this one is annoying but minor.

        My MAJOR beef is actual profit/loss results are sometimes COMPLETE nonsense... but the "fix" (as advised by NT)... run the backtest in a new Analyzer tab/window. Which, with the SAME data, the SAME strategy, the SAME parameter settings... gives a COMPLETELY different set of trades. All that changed was re-running in a new window. What the [email protected]#$????

        No explanation... just do it. OK... and which set of trades/results is the user supposed to have confidence in ??

        Anyway, I stray off topic... apologies. Will pursue this significant gripe in a new thread... it just seems related that sometimes, NT seems to get quite confused about trade history/performance... so... they may have related root causes - perhaps.

        Cheers,
        T.

        PS Will email trace files when I get a few spare minutes... but, I strongly suspect it is at least partly data related...

        Comment


          #19
          Kate - (or anyone else who may know) - what is the status of these reporting errors in MAE / MFE? I just ran a YTD trade Performance summary and I see some trades where the MAE or MFE are +3500 or more. I spot checked multiples of them to insure that these were not held overnight. They were not. in some cases the trade only last 1 3 minute bar in duration, so a $3500 excursion is not possible (I verified on the chart). While these are sim trades, I am relying on the trade performance data and specifically the MAE and MFE to evaluate these strategies, and I am questioning whether these 2 metrics are even reliable.

          Can you update the current status of this issue?

          While I have the podium, assuming they are not reliable or at least we don't know if they are, has anyone developed an "internal" trade tracker inside of a strategy that tracks the MAE and MFE equivalents (and perhaps write them out to a file)? I'm hoping someone has written a simple work around until the official one gets fixed (I mean a user, or a support person as a temp fix).
          Jim-Boulder
          NinjaTrader Ecosystem Vendor - Elephant Tracks

          Comment


            #20
            Hello Jim-Boulder,

            Thank you for your inquiry.

            We are currently working with tgn55 to isolate his particular issue. In your case, my initial idea would be that the trade pairing is off due to the position of the account not being flat at the beginning of the day the Trade Performance was run from. If you choose a day on which you know you had no open positions to start from, do you see inaccurate MAE/MFE calculations?

            As far as internally tracking within a strategy, it looks like this indicator that's publicly available on our user app share may contain something like what you're looking for:

            https://ninjatraderecosystem.com/use...aster_de-v7-2/

            Please let us know if we may be of further assistance to you.

            The NinjaTrader Ecosystem website is for educational and informational purposes only and should not be considered a solicitation to buy or sell a futures contract or make any other type of investment decision. The add-ons listed on this website are not to be considered a recommendation and it is the reader's responsibility to evaluate any product, service, or company. NinjaTrader Ecosystem LLC is not responsible for the accuracy or content of any product, service or company linked to on this website.
            Kate W.NinjaTrader Customer Service

            Comment


              #21
              Been occupied with other matters for a while, but, just throwing in this...

              The "solution" of starting the report when flat really does not cut the mustard.
              I have strategies running non-stop on multiple instruments... there is NEVER a time when all instruments/positions are flat! Asking the user to work around this is, well... pretty ordinary at best.

              Am I missing something? I appreciate that there may be some weird circumstances that make matching a little less trivial than it first appears, but I can't find a reason why, given a trade,, having an entry and an exit, should not ALWAYS begin with an entry, as mentioned above.

              Surely, changing the reporting code to STRICTLY enforce this would resolve all the issues around MAE/MFE, weird PnL results etc

              T.

              Comment


                #22
                Any update on this? I traded 1 ES 03-21 contract today and the MAE is reported incorrectly.

                Entry time: 1/20/2021 11:02:56 AM
                Exit time: 1/20/2021 11:08:15 AM

                MAE should be 0.75pts, shows as 0.5pts

                Please check on your end before asking me to send screenshots, etc. as I first reported this in original post's thread back in Apr 2018. It also seems many other users have been reporting this for years.

                Comment


                  #23
                  Hello jtrading82,

                  Thank you for your note.

                  To clarify, was this a single trade? Meaning the entry quantity was 1 and the exit quantity was 1 - no scaling out occurred? What time zone are you located in so that I may best try to reproduce?

                  What were the entry and exit prices for the trade?

                  Thanks in advance; I look forward to assisting you further.
                  Kate W.NinjaTrader Customer Service

                  Comment


                    #24
                    Yes, just 1 qty, 1 round-trip trade, it is as basic as it gets. There should be no excuses.

                    Entry time: 1/20/2021 11:02:56 AM PST BUY 1 contract @ 3843.0
                    Exit time: 1/20/2021 11:08:15 AM PST SELL 1 contract @ 3843.25

                    Comment


                      #25
                      Hello jtrading82,

                      Thank you for your reply.

                      In order to assess whether this is a example of a known limitation or something that we need to run past our development team, we'll need to review your log, trace, and database files.

                      You can send these by going to the Control Center-> Help-> Mail to Platform Support.

                      Ensure to check 'Database file' and ' Log and trace files'.

                      Please reference the following ticket number in the body of the email: 2305087 ATTN Kate W.

                      Thanks in advance; I look forward to assisting you further.
                      Kate W.NinjaTrader Customer Service

                      Comment


                        #26
                        Thanks Kate and sent. Can you also list known limitations on MAE/MFE so we can be aware of these limitations?

                        Comment


                          #27
                          Hello jtrading82,

                          Thank you for your reply.

                          I didn't receive all the necessary files I've requested, so I've followed up with you via email on that.

                          As far as known issues with MAE/MFE calculations go, here's the major ones:
                          • Scaling in and scaling out
                          • Partial fills
                          • Reporting from provider
                          • Not enough executions being loaded from DB, (only loads 3 days back) resulting in wrong 'execution' position tracking in real-time
                          • Missing executions, e.g. 2 days ago 2 long ES, you don't connect 1 day ago with NinjaTrader and flatten using the mobile client, then login today with NinjaTrader. From an execution perspective Ninjatrader still thinks you are long 2 because it never gets the flattening execution.
                          • When internal position tracking is off is that the entry execution would register as an exit and therefore not track min/max price. If this occurs, NinjaTrader will set Entry.MinPrice and Entry.MaxPrice then equal to the Exit price which results in MAE/MFE of Entry.Price - Exit.Price.
                          It is worth noting that there is no overall concept of 'Trade' in NT8. Trades are built from pairing executions. The entry execution is tracking low/high price seen while in position.

                          Thanks in advance; I look forward to assisting you further.
                          Kate W.NinjaTrader Customer Service

                          Comment

                          Latest Posts

                          Collapse

                          Topics Statistics Last Post
                          Started by AlexsOptions, Today, 05:42 PM
                          2 responses
                          8 views
                          0 likes
                          Last Post AlexsOptions  
                          Started by philli4093, Today, 05:20 PM
                          2 responses
                          5 views
                          0 likes
                          Last Post philli4093  
                          Started by memonolog, Today, 04:58 PM
                          0 responses
                          1 view
                          0 likes
                          Last Post memonolog  
                          Started by robvul, Today, 04:20 PM
                          1 response
                          10 views
                          0 likes
                          Last Post NinjaTrader_ShawnB  
                          Started by sandman, Today, 01:54 PM
                          3 responses
                          24 views
                          0 likes
                          Last Post NinjaTrader_Jim  
                          Working...
                          X