• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

Replay Problems

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Replay Problems

    I am having problems with the Market Replay. I get hugely different results in backtesting, replay and running a strategy live with IB.

    Using your SampleMACrossOver as an example, I get the following results:
    Time: 2/7/12 - 4/7/12
    Parameter 34/68
    EURUSD
    1 Minute
    Backtest: 13380 € Profit
    Replay: 23.019€ Realized Loss (420€ unrealized Loss)

    I know that there are differences between backtesting and replay. But turning a 13.380 profit into a 23.019 loss seems more than a bit odd.

    Can you reproduce these results on your side? It seems that my Market Replay is behaving much stranger now, than it used to do. I already tried to reset the DB and reload all the historical data. It does not make a difference.
    Do you have any explanation?
    By the way: If I run a strategy live with IB, it is much closer to the backtesting than to the replay results. What is wrong?

    #2
    Hello moon_121,
    To assist you further may I know are you running the strategy in Market Replay with Calculate on bar close set to true or false.

    I look forward to assisting you further.
    JoydeepNinjaTrader Customer Service

    Comment


      #3
      "On bar close" is set to "true". It gets even worse if I try it with false.
      Last edited by moon_121; 07-05-2012, 04:21 AM.

      Comment


        #4
        Hello moon_121,
        Unfortunately backtest, replay and live results can vary.
        http://www.ninjatrader.com/support/h...ime_vs_bac.htm

        Backtest is calculated at the end of the bar and thus if you are running the strategy with Calculation on bar close set to false then the results can be even farther.

        Also please note, there can be data discrepancies between IB and Market Replay since IB uses snapshot data to construct the bars.
        JoydeepNinjaTrader Customer Service

        Comment


          #5
          I knew I would get this generic reply. It is the same answer NT gives everybody who has problems with backtesting, replay or setting up a strategy with Renko.
          Perhaps you could explain a slight difference between backtesting and replay with this answer. But the complete opposite? With SMA-cross? With 1 Minute charts? It can't get any dumber than that. We are not talking Ticks or Renkos here.
          At least, the backtest should be approximately the same like the replay.
          And by the way: I am using Kinetick as market data provider. And as I said: Everything is set to "on bar close: true".
          Here is my challenge: Why don't you show me an example, where your backtesting resembles a market replay. Can it be done? Or is one of them complete rubbish?

          Comment


            #6
            Hello moon_121,
            Backtesting by default does not use bid/ask data to execute the orders. This is not the case while you are live. This can make a considerable difference. You can further enhance your code and add the bid/ask series and submit the orders in the bid/ask series while backtesting your strategy.

            We are aware of the demands of trades regarding this discrepancies and it is already in the our feedback/enhancement list and we hope to enhance the backtest feature further in our next major release.
            JoydeepNinjaTrader Customer Service

            Comment


              #7
              I am glad to hear that you are working on a better backtesting.
              But:
              I can't believe that the bid/ask difference in a market replay makes all the difference. My SMAcross-Example executed only 6 trades during 3 days. With bid/ask diffrences it should perhaps have a 1000€ difference to the backtest. But I am getting 26000 € difference. I think that is a bit to much to explain by a bid/ask spread.
              Could you at least tell me, if you are getting the same replay and backtesting results on your machines? I suspect that something is wrong with my db.

              Comment


                #8
                Hello moon_121,
                Can you send a screenshot of the exact backtest settings so that I can test it at my end.

                I suppose you are testing from 2nd July - 4th July 2012 (and not from 7th February - 7th April 2012).

                I look forward to assisting you further.
                JoydeepNinjaTrader Customer Service

                Comment


                  #9


                  I tried to replicate the results with an other opimization run in backtest and got some different results.
                  In the screenshot, you see the Strategy Analyzer Window on the left hand side. On the right side are the results I got earlier and saved in Excel. (Disregard the PointAndFigure results. This would only make everything more complicated.)
                  Attached Files
                  Last edited by moon_121; 07-05-2012, 07:17 AM.

                  Comment


                    #10
                    Hello moon_121,
                    Unfortunately the screenshot did not upload. Can you please reload it again.

                    Edit: Never mind got it. Thanks.
                    JoydeepNinjaTrader Customer Service

                    Comment


                      #11
                      Hello moon_121,
                      It may be the case that new data are being loaded when you are backtesting/optimizing the strategy again. Forex session starts in the evening and ends in the following day. So todays data will be also included if you set the end date to 4th July '12.
                      JoydeepNinjaTrader Customer Service

                      Comment

                      Latest Posts

                      Collapse

                      Topics Statistics Last Post
                      Started by Sh00tHerMcGavin, Today, 07:39 PM
                      0 responses
                      3 views
                      0 likes
                      Last Post Sh00tHerMcGavin  
                      Started by Mberg, Today, 05:13 PM
                      0 responses
                      1 view
                      0 likes
                      Last Post Mberg
                      by Mberg
                       
                      Started by Barry Milan, Today, 01:33 PM
                      2 responses
                      9 views
                      0 likes
                      Last Post marcus2300  
                      Started by NW Trader, Today, 03:42 PM
                      2 responses
                      8 views
                      0 likes
                      Last Post NW Trader  
                      Started by ralett, Today, 01:42 PM
                      1 response
                      5 views
                      0 likes
                      Last Post NinjaTrader_ShawnB  
                      Working...
                      X