• If this is your first visit, you will have to register before you can post. To view messages, please scroll below and select the forum that you would like to visits. Questions? Be sure to check out the Forum FAQ.

Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

IsFirstTickOfBar vs OnBarClose for backtest & live

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • NinjaTrader_ChelseaB
    replied
    Hello buzzripper,

    Add a 1 tick series for intra-bar granularity. Submit the order when the tick series is updating before the primary series bar closes. Place the order to the 1 tick series bars in progress so the order fills with accurate price data.

    Please see the Backtesting NinjaScript Strategies with an intrabar granularity reference sample that demonstrates intra-bar granularity.

    Leave a comment:


  • buzzripper
    replied
    Hi Chelsea,

    Originally posted by NinjaTrader_ChelseaB View Post
    In the meantime, if it is necessary to have both, it is still possible to add intra-bar granularity to a script in the code itself for order fill accuracy and use TickReplay to update indicators with Calculate set to OnPriceChange or OnEachTick historically.
    Can you elaborate on this? Correct me if I'm wrong, but my understanding is this:
    • If you want accurate fills (i.e. EnterLong() be submitted immediately, not wait until the end of the bar), use Order fill resolution = High
    • If you want to evaluate and decide on a buy during a bar, not after its closed, use TickReplay
    • Order fill resolution = High and TickReplay do not work together (due to an issue with the current NT8 software)
    So not understanding your statement about "if it is necessary to have both"... how do i get my EnterLong() method to execute before the end of the current bar? I don't have any control over that do I?

    Thanks

    Leave a comment:


  • NinjaTrader_ChelseaB
    replied
    Hello cutzpr,

    To confirm, you have tested adding a 1 tick series to the script, and OnBarUpdate did not trigger for each tick?

    You are wanting to place an order before the bar closes, is this correct?

    Leave a comment:


  • cutzpr
    replied
    The only way I found to get around this was to add a duplicate data series exactly the same as the primary.
    Create a bool enter/exit variable that the Primary data series would modify
    Do no processing on the secondary data series except submit your orders. Since it is immediately processed by the strategy after it is done processing the primary data series.

    This is the closest thing I can do to mimic a callback that the current bar is closed.
    Last edited by cutzpr; 11-30-2018, 09:08 PM.

    Leave a comment:


  • NinjaTrader_ChelseaB
    replied
    Hello cutzpr,

    Because of how the orders are being placed, this is expected behavior.

    You would need to place the order before the bar closes with intra-bar granularity if you would an order to be placed (and possibly filled) on the bar where the condition was true.
    https://ninjatrader.com/support/foru...377#post773377

    Leave a comment:


  • cutzpr
    replied
    To piggy back off of fxRichard's frustration. He is right. The Open Price of the next candle when as order is placed is not always the same price as the Candle that just Closed. Actually, it almost never is. This leads to flawed results because the orders are placed at a different price than the Close of that Bar. I wrote a quick method shown below that you can test. You can see from the results I have uploaded that sometimes there is a huge variation in the ticks. If I am running this method on the
    if(IsFirstTickOfBar
    , I only expect to see a slight difference in one or two ticks.


    My test settings.


    Instrutment: EURUSD
    Bar Type: Heiken Ashi
    Base Period: Minute
    Base Period Value: 5
    Tick Replay: True



    PHP Code:
    if(IsFirstTickOfBarzWriteBarOpenCloseLog(); 
    PHP Code:
    public void zWriteBarOpenCloseLog()    
            {

                    
    string barTime string.Format("{0:G}"Time[0]);

                    
    string BarOpenCloseLogName =  NinjaTrader.Core.Globals.UserDataDir "\\BarOpenClose.csv";
                    
    string BarOpenCloseLogHeader =

                        
    "Time"                "," +
                        
    "Barof BarOpenClose        "    "," +
                        
    "BarsInProgress"            "," +
                        
    "Open"                "," +
                        
    "Close"                "," +
                        
    "Previous Bar Close"    "," +
                        
    "Open[0] = Close[1]" "," +
                        
    "Difference in Ticks"         "," +
                        
    "Bar % Complete";


                        
    string diff Math.Round(((Math.Abs(Open[0]-Close[1]))/TickSize),2).ToString();
                        Print(
    diff);
                        
    string eq = (Open[0] == Close[1]) ? "TRUE" "FALSE";

                    
    string BarOpenCloseLogString =
                        
    barTime         "," +
                        
    CurrentBar     "," +
                        
    BarsInProgress         "," +
                        
    Open[0]            + "," +
                        
    Close[0]        + "," +
                        
    Close[1]        + "," +
                        
    eq                "," +
                        
    diff            "," +
                        
    String.Format("Value: {0:P2}."Bars.PercentComplete); // Does not work at all during backtest, only valid during realtime


                    
    try
                    {
                        if (!
    File.Exists(BarOpenCloseLogName))
                        {
                            
    using (StreamWriter sw6 = new StreamWriter(BarOpenCloseLogNametrue))
                               {
                            
    sw6.WriteLine(BarOpenCloseLogHeader); // If file doesnt exist, create it and add the Header
                            
    sw6.WriteLine(BarOpenCloseLogString); // Append a new line to the file
                            
    sw6.Close(); // Close the file to allow future calls to access the file again.
                            
    }
                        }
                        else
                        {
                            if (!
    zIsFileLocked(BarOpenCloseLogName)) //If file is not locked for editing
                            
    {

                                
    using (StreamWriter sw6 = new StreamWriter(BarOpenCloseLogNametrue))
                                   {    
                                
    sw6.WriteLine(BarOpenCloseLogString); // Append a new line to the file
                                
    sw6.Close(); // Close the file to allow future calls to access the file again.
                                
    }
                            }
                            else
                            {

                            }

                        }
                        }
                        catch (
    Exception e)
                        {
                            
    // Outputs the error to the log
                            
    Log("BarOpenCloseLog - You cannot write and read from the same file at the same time."NinjaTrader.Cbi.LogLevel.Error);
                            Print(
    e.ToString());
                        }

                } 




    Attached Files
    Last edited by cutzpr; 11-25-2018, 05:42 PM.

    Leave a comment:


  • NinjaTrader_ChelseaB
    replied
    Hello fxRichard,

    To clarify, you are using NinjaTrader 8, you are placing an order after a bar has closed, the Order fill resolution is set to Standard, and the order is showing as being filled on the bar that triggered the condition?

    With the order fill resolution set to standard this should not be the case.
    Please let me know if you would like a short sample that demonstrates this.

    You are using an index of [1] for the most recently closed bar, is this correct? (Using an index of [0] would be for the newly opened and currently building bar.)

    Leave a comment:


  • fxRichard
    replied
    Originally posted by tradertakeo View Post
    Hope this is what you are looking for.

    Code:
    	if (IsFirstTickOfBar)
    	{	
    		if(State == State.Historical)
    		{
    			openValue = Open[0];
    			closeValue = Close[0];					
    		}
    		else
    		if(State == State.Realtime)
    		{
    			openValue = Open[1];
    			closeValue = Close[1];		
    		}					
    	}

    Thanks for your response TraderTakeo, I am aware of this but what needs to be solved in NT is the capability to get a callback when a bar closes when using OnEachTick.

    When checking for IsFirstTickOfBar wich OnEachTick you get a "similar" but not same result as doing your calculation with OnBarClose. The biggest effect is when an actual order is placed when performing backtesting or running on historical data as any orders placed on the current bar will not be "filled" until the open of the next bar (without intrabar data).

    If NT simply had a callback to tell you when a bar is closing then the index value for all data series is still on the closing bar and any orders placed will be more in line with backtesting and real time (for simple order execution).

    This is not so much the issue of the index value but the order handling and processing in NT and when it happens.

    Leave a comment:


  • NinjaTrader_ChelseaB
    replied
    Hello fxRichard,

    Backtesting (historical) and real-time are fundamentally different.
    Below is a link to the help guide that details this.
    http://ninjatrader.com/support/helpG...ime_vs_bac.htm

    If you want intra-bar actions like triggering orders with Calculate.OnEachTick, you must have TickReplay enabled.

    May I confirm you have TickReplay enabled?

    Leave a comment:


  • tradertakeo
    replied
    Hope this is what you are looking for.

    Code:
    	if (IsFirstTickOfBar)
    	{	
    		if(State == State.Historical)
    		{
    			openValue = Open[0];
    			closeValue = Close[0];					
    		}
    		else
    		if(State == State.Realtime)
    		{
    			openValue = Open[1];
    			closeValue = Close[1];		
    		}					
    	}

    Leave a comment:


  • fxRichard
    replied
    Thanks Chelsea.

    When using Calculate.OnEachTick and using IsFirstTickOfBar the last bar that closed is now index 1 in the Price data series.

    This presents a problem when placing orders after a calculation when a bar closes. Let's look at the two scenarios:

    Scenario 1 - Calculate.OnBarClose

    I want to evaluate the current closing bar, if it closes up I want to immediately take a market entry long, if it closes down I want to immediately go short.

    Code:
    if(Close[0] > Open[0])
        EnterLong();
    else if(Close[0] < Open[0])
        EnterShort();
    If I am running live the order will be immediately placed. If I am backtesting the order will be placed and "simulated filled" immediately on the opening of the next bar. Not a problem.

    Scenario 2 - Calculate.OnEachTick
    In this example we want to do the exact same thing except we don't know when a bar closes unless we use IsFirstTickOfBar. So let's assume the same scenario in which we now have to reference the closing candle at data series index 1 instead of 0 since we are on the first tick of the new bar:

    Code:
    if(Close[1] > Open[1])
        EnterLong();
    else if(Close[1] < Open[1])
        EnterShort();
    If I am running live the order will be immediately placed similarly to using OnBarClose. If I am backtesting the order will be placed and "simulated filled" at the open of the NEXT bar. This is expected and known due to the fact that there is no intrabar data unless you setup one of the scenarios above that you mentioned.


    I would expect there to be some way to handle order placement/fills in backtesting with OnEachTick similar to OnBarClose. To fix this problem we need to be able to do our calculation at the close of a bar while the closing bar is still at data series index 0 and not 1. Once it moves to 1 using "IsFirstTickOfBar" we then have this issue of the order being placed at the open of the following bar.

    Hope this helps clarify.

    Leave a comment:


  • NinjaTrader_ChelseaB
    replied
    Hello fxRichard,

    I'm not able to understand.

    You can tell a bar has closed with Calculate.OnEachTick with IsFirstTickOfBar.
    http://ninjatrader.com/support/helpG...ttickofbar.htm

    Can you clarify what you are not able to do?

    Leave a comment:


  • fxRichard
    replied
    Thanks for your response Chelsea. I am familiar with backtesting vs real-time and don't understand why we don't have the capability to determine a bar close when doing Calculate.OnEachTick.

    I understand we don't have intrabar data but I would expect NT to have 2 callbacks something like: OnBarUpdate and OnBarClose etc.

    Not saying this is the solution but there is definitely a better way to handle real-time vs backtesting results and how NT processes data.

    NT has the capability to use Calculate.OnBarClose and reference the current bar at index 0 and evaluate that bar to place a trade at the OPEN of the next bar and get simulated fills at that open price.

    Just by moving to Calculate.OnEachTick we have no real way to know when the current bar closes. So NT has the IsFirstTickOfBar which tells us the last bar just closed but now we are referencing the last bar at index 1 and any calculations we take to place a trade won't happen until the open of the next bar. I understand the reasons why, there is no intrabar data etc, you don't know if the developer technically did that calculation intrabar and would have been filled mid bar etc.

    There is no elegant way in NT to process the current bar upon close with Calculate.OnEachTick to do something like I mentioned above.

    If NT can tell us when a bar closes and call onbarclose I would expect it would give us some way to know the same event when doing calculate.oneachtick.

    Leave a comment:


  • NinjaTrader_ChelseaB
    replied
    Hello fxRichard,

    Please review the help guide document on the differences on real-time vs backtest (historical).
    http://ninjatrader.com/support/helpG...ime_vs_bac.htm

    When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

    Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick'.

    Below is a link to the help guide on Calculate.
    https://ninjatrader.com/support/help.../calculate.htm


    Adding intra-bar granularity can help with this.

    Intra-bar granularity adds a second data series such as a 1 tick series using AddDataSeries() so that the strategy or indicator has the individual ticks in the historical data in between the High and Low of the primary series. This allows for more accurate trades by supplying the correct price at the correct time for the order to fill with. The orders placed will need to use the BarsInProgress (BarsArray) index of that finer resolution series.
    -Adding a 1 tick series with AddDataSeries() does not enable IsFirstTickOfBar to work correctly in historical data. (IsFirstTickOfBar will always be true)

    Below is a link to an official reference sample that demonstrates how to add intra-bar granularity.
    https://ninjatrader.com/support/helpGuides/nt8/en-us/backtesting_ninjascript_strate.htm

    Also, here is a link to the differences on real-time vs backtest (historical).
    http://ninjatrader.com/support/helpG...ime_vs_bac.htm

    As well as a link to the help guide on the AddDataSeries() method.
    http://ninjatrader.com/support/helpG...dataseries.htm

    A link to the help guide on BarsInProgress.
    http://ninjatrader.com/support/helpG...inprogress.htm

    And a link to the help guide on Multi-Time Frame & Instruments. Please see the section 'How Bar Data is Referenced', and 'Accessing the Price Data in a Multi-Bars NinjaScript'.
    http://ninjatrader.com/support/helpG...nstruments.htm

    The Fill limit orders on touch option will also change whether a limit order requires a tick through to fill or if it fills on touch, which can change the price at which a limit order fills and changes the end performance.
    https://ninjatrader.com/support/help...a_strategy.htm
    https://ninjatrader.com/support/help...mitontouch.htm

    Also, below I'm including a link to the help guide on Historical Order Backfill Logic.
    https://ninjatrader.com/support/help...fill_logic.htm



    In NinjaTrader 8, there have been two new enhancements so that programmers may not have to manually add this secondary series and code the script for high accuracy fills (Order Fill Resolution) or for intra-bar actions (TickReplay) depending on the needs of the script.

    The first is TickReplay.
    TickReplay is used for the intra-bar actions. This adds a 1 tick series to the script to be used for triggering OnBarUpdate when Calculate is set to .OnEachTick or .OnPriceChange. This also allows OnMarketData to trigger historically for last market events only (the bid and ask will not trigger).
    -TickReplay will allow for indicators to update for each tick or price change and for scripts to trigger actions intra-bar.
    -TickReplay allows for IsFirstTickOfBar to work correctly in historical data and will be true for the first tick of the bar only
    -TickReplay does not provide fill prices for accurate order fills.

    Below are links to the help guide on TickReplay.
    TickReplay - http://ninjatrader.com/support/helpG...ick_replay.htm
    Developing for Tick Replay - http://ninjatrader.com/support/helpG...ick_replay.htm
    OnMarketData - http://ninjatrader.com/support/helpG...marketdata.htm
    IsFirstTickOfBar - https://ninjatrader.com/support/help...ttickofbar.htm


    The second is Order Fill Resolution.
    This has two settings, Standard and High. When set to standard, the behavior of order fills will be accurate to the close of the bar without any intra-bar information. When set to High, this allow you to select a secondary data series to be used to provide intra-bar pricing information solely for the purpose of order fill accuracy. The highest you could set this to is 1 tick.
    -Order fill resolution does not provide intra-bar granularity to OnBarUpdate for actions to be triggered intra-bar.
    -
    Order fill resolution does not enable IsFirstTickOfBar to work correctly in historical data.

    Below is a link to the help guide on Order Fill Resolution. (Above this you can also read about the Historical Fill Algorithm)
    http://ninjatrader.com/support/helpG...FillResolution


    Currently, at this time, there is a known issue (tracked with ID# NTEIGHT-10782) where the High Fill Order Resolution and TickReplay cannot be used together. These work great when used on their own, however, when used together a complication arises that prevents the script from processing. Our developers are working hard to have this corrected in an upcoming build of NinjaTrader 8.

    In the meantime, if it is necessary to have both, it is still possible to add intra-bar granularity to a script in the code itself with AddDataSeries() for order fill accuracy and use TickReplay to update indicators with Calculate set to OnPriceChange or OnEachTick historically and/or have IsFirstTickOfBar work correctly in historical data.


    It is not advised or necessary to change the Calculate mode during run-time.
    Last edited by NinjaTrader_ChelseaB; 07-31-2019, 06:40 AM.

    Leave a comment:


  • fxRichard
    started a topic IsFirstTickOfBar vs OnBarClose for backtest & live

    IsFirstTickOfBar vs OnBarClose for backtest & live

    I have a strategy that runs with calculation mode OnEachTick however in a backtest environment data is only processed OnBarClose.

    What I'm after is how to get an "OnBarClose" in a live OnEachTick call to OnBarUpdate.

    The reason for this is that I have to have separate logic to determine at what point we are looking at price (or any) data series.

    For example if I run Calculate.OnEachTick and I want to look for a simple thing such as the last candle closed UP I would have to do something like this:

    Code:
      if(IsFirstTickOfBar && Close[1] > Open[1])
         Print("Up Candle");
    This is fine and works the same in a back test BUT imagine we now want to take a trade :

    Code:
      if(IsFirstTickOfBar && Close[1] > Open[1])
        EnterLong(...);
    In a live environment this will immediately take the trade on the opening of the current candle. In a back test the trade won't get placed until the close of the current candle we are on. To fix this we need to move our logic to using Calculate.OnBarClose for a backtest which is also fine but then we have to do some silly stuff such as:

    Code:
      ...in State.Configure
      IsBackTest = Account.Name == Account.BackTestAccountName;
      
      if(IsBackTest)
         Calculate = Calculate.OnBarClose();
    
      ...in OnBarUpdate
      if(IsBackTest && Close[0] > Open[0])
         EnterLong(...);
      else if(IsFirstTickOfBar && !IsBackTest && Close[1] > Open[1])
         EnterLong(...);
    See where I'm going with this? I would like to keep processing utilizing the same data series index without having to do a bunch of extra logic throughout the code. Meaning can we still do Calculate.OnEachTick and know when the bar is actually closing to reference the current candle as index 0 vs index 1 that it would be for OnBarClose or during a backtest simply to keep our backtest entries in line with our live entries on the same bar?
    Last edited by fxRichard; 01-16-2017, 01:17 PM.

Latest Posts

Collapse

Topics Statistics Last Post
Started by martyn73, Today, 03:09 PM
0 responses
2 views
0 likes
Last Post martyn73  
Started by martyn73, Today, 03:05 PM
0 responses
1 view
0 likes
Last Post martyn73  
Started by mic414, Today, 02:17 PM
0 responses
1 view
0 likes
Last Post mic414
by mic414
 
Started by U0000999, Yesterday, 06:18 PM
1 response
19 views
0 likes
Last Post NinjaTrader_ChrisL  
Started by Jurase, Today, 10:44 AM
4 responses
22 views
0 likes
Last Post Jurase
by Jurase
 
Working...
X