Thread to discuss SampleIntrabarBacktest Reference Strategy

[aventeren]

I have created this thread to discuss the SampleIntrabarBacktest sample reference strategy that @NinjaTrader_Josh posted here.

For ease, I have attached his sample strategy.

The description post as of 2.24.14 is:

Default Strategy: Backtesting NinjaScript Strategies with an intrabar granularity
Reference sample for NinjaTrader 6.5.XXXX.X or greater.

As of NinjaTrader 6.5, you can submit orders to different Bars objects. This allows you the flexibility of submitting orders to different timeframes. Like in live trading, taking entry conditions from a 5min chart means executing your order as soon as possible instead of waiting until the next 5min bar starts building. You can achieve this by submitting your orders to a more granular secondary bar series to achieve an "intrabar" fill.

The attached reference sample demonstrates the following concepts:
Finding entry conditions on the primary bar object
Submitting orders to the secondary bar object for an intrabar fill

Important methods and properties include:

Add()
BarsInProgress
EnterLong()

Other methods and properties of interest include:

BarsArray
EnterLongLimit()

Import instructions for NinjaTrader 6.5.XXXX.X or greater:

Download the file contained in this thread to your PC desktop
From the Control Center window, select the menu File > Utilities > Import NinjaScript
Select the downloaded file

[aventeren]

Change: 1 Minute to 1 Range

I am first interested in converting the secondary bar series from a 1 Minute series to a 1 Range series. To do this, you simply change the Add PeriodType call in Initialize() like this:

 protected override void Initialize()
        {
			/* Add a secondary bar series. 
			Very Important: This secondary bar series needs to be smaller than the primary bar series.
			
			Note: The primary bar series is whatever you choose for the strategy at startup. In this example I will
			reference the primary as a 5min bars series. */
			Add(PeriodType.Range, 1);
			
			// Add two EMA indicators to be plotted on the primary bar series
			Add(EMA(Fast));
			Add(EMA(Slow));
			
			/* Adjust the color of the EMA plots.
			For more information on this please see this tip: http://www.ninjatrader-support.com/vb/showthread.php?t=3228 */
			EMA(Fast).Plots[0].Pen.Color = Color.Blue;
			EMA(Slow).Plots[0].Pen.Color = Color.Green;
			
            CalculateOnBarClose = true;
        }

So we now have a 1 Range secondary series that can be referenced via BarsInProgress == 1 or BarsArray[1]. We can also access the secondary series as Josh did by calling the BarsArray index number in the order commands, as Josh did with:

EnterLong(1, 1, "Long: 1min");
EnterShort(1, 1, "Short: 1min");

Where the syntax is looking for (int barsInProgressIndex, int quantity, string signalName). Therefore, the first 1 calls the BarsInProgress 1 series, which was his 1 Minute series and is now our 1 Range series. Therefore, let's change these to:

EnterLong(1, 1, "Long: 1range");
EnterShort(1, 1, "Short: 1range");

So we now have replaced the 1 Minute secondary series with a 1 Range series and we have slightly modified the order calls to reference the 1range signal name.

My first question is how Josh would change his commented example given that we are now using the 1 Range secondary series:

/* When working with multiple bar series objects it is important to understand the sequential order in which the
			OnBarUpdate() method is triggered. The bars will always run with the primary first followed by the secondary and
			so on.
			
			Important: Primary bars will always execute before the secondary bar series.
			If a bar is timestamped as 12:00PM on the 5min bar series, the call order between the equally timestamped 12:00PM
			bar on the 1min bar series is like this:
				12:00PM 5min
				12:00PM 1min
				12:01PM 1min
				12:02PM 1min
				12:03PM 1min
				12:04PM 1min
				12:05PM 5min
				12:05PM 1min 
			
			When the OnBarUpdate() is called from the primary bar series (5min series in this example), do the following */

So let's say that we are wanting to test the sample EMA cross system like COBC = false when we actually start trading. Therefore, we would be looking for entries to be made within the primary bar series' bars.

Because NT7 stamps each bar with the final (Close) time, the bar that opens at 12:00PM and will close at 12:05PM will be stamped at the 12:05PM bar. Let's assume that the individual tick that caused the EMAs to cross for the first time occurred somewhere within the 12:05PM bar, which means that the cross occurred somewhere between 12:00:01PM and 12:05:00PM. How would the example call order text change (above)? For this example, let's assume that the OHLC for the 12:05PM bar was 94.00, 100.00, 90.00 and 97.00, respectively. How would then 1 Range series be processed?

My second question: Is the 1 Range data series only processed when it is needed? In other words, given the above example where an order would have been placed at some point within the 12:05PM bar, which we might then imply for our example that the cross did not happen within the 12:00PM bar, which would mean that we would not need the code to process the 1 Range series for the 12:00PM bar. How does NT7 handle these secondary data series? Does it first have to load the secondary data series for every bar before running the back test (much slower back test, as the entire secondary series needs to be loaded) or is the secondary series only called and accessed when it is needed (faster back test, as only portions of the secondary data series are accessed)?

That's all for now--I'm sure the answers to these questions will spur me towards the next series of clarifications.

Thanks,

Aventeren

[NinjaTrader_Bertrand]

Aventeren, I guess I don't understand your sequence question - when mixing different bars building approaches you would not even have a guarantee that the the added series would be smaller to provider the intrabar backtesting simulation, so I don't reallty see the point here, unless you for example used a larger Range chart for example as primary as well.

Both series would need to be loaded and held in memory to be processed by the backtest, per the session template and market used NinjaTrader will first time create a cache to work from which speeds up the process significantly on subsequent requests then.

[aventeren]

Aventeren, I guess I don't understand your sequence question - when mixing different bars building approaches you would not even have a guarantee that the the added series would be smaller to provider the intrabar backtesting simulation, so I don't reallty see the point here, unless you for example used a larger Range chart for example as primary as well.

Both series would need to be loaded and held in memory to be processed by the backtest, per the session template and market used NinjaTrader will first time create a cache to work from which speeds up the process significantly on subsequent requests then.

Thanks, Bertrand.

I've read on the inter webs that one can use a 1 Range secondary series to analyze for backtesting purposes the tick by tick progression within a time based primary bar. Using my OHLC example values (below), how can we know how price moved from the Open to the Close for back testing entries and exits?

Thanks for your clarification on the back end loading/memory/cache question I had.

[NinjaTrader_JC]

Hello aventeren,

For a Tick by Tick progression you may want to use a "1 Tick" granularity but we will not know how many Ticks are going to be in a bar so there is no way of knowing how many or what the time stamps of the bars are going to be. Same goes with Range bars since they are Time Independent.

With a "1 Tick" granularity NinjaTrader guarantees them to be in the correct order but you may have any number of ticks for each bar.

[aventeren]

Hello aventeren,

For a Tick by Tick progression you may want to use a "1 Tick" granularity but we will not know how many Ticks are going to be in a bar so there is no way of knowing how many or what the time stamps of the bars are going to be. Same goes with Range bars since they are Time Independent.

With a "1 Tick" granularity NinjaTrader guarantees them to be in the correct order but you may have any number of ticks for each bar.

Valid point. So at a minimum, to have an indexed primary and secondary series, we would need to use matched series? For instance, 60 minute and 1 minute, 200 tick and 1 tick, 100 range and 1 range, etc? Would these example pairs give us the intrabar back testing granularity that we are seeking?

[NinjaTrader_Bertrand]

Correct, with those groupings you would know the added bar is smaller and could provide a finer granularity relative to the primary.

[aventeren]

Secondary Series; Loading, Memory, Cache

Back to the secondary series being loaded into memory so that NT has a cache to work from, would it somehow be possible to not have the secondary series loaded into memory until it was needed (I'm thinking some sort of logic would call the secondary series)?

For instance, instead of loading the entire secondary series within Initialize(), would it be possible to just call a portion of the secondary series from the database? For instance, between 12:00:01PM and 12:05:00PM on a given day (from our example below), we could trigger a bool state change (from False to True) that would then trigger the secondary series call between the primary series open and close times (or prices for range chart)?

This would be akin to drawing a syringe of lake water instead of sucking up the whole lake (i.e., targeted database calls versus loading entire secondary series).

[NinjaTrader_JC]

Hello aventeren,

That would not be possible inside of NinjaTrader 7 as you would need to add the Series inside of Initialize() for NinjaTrader 7 to be able to process this.

[Ravish]

Quick question: Why does the 2nd (and I am assuming all subsequent) series have to be "smaller than the primary series".

I am trying to write a strategy that looks at entry signals on the M1 chart based on 3 indicators, with trend confirmation from M5 and H1.

So this means I have to run the backtest on the H1 chart, and Add() the M5 and M1 timeframes as additional bar sets, which means the visual chart in the backtester is an H1 chart, so I can't see the entry conditions provided by my 3 M1 indicators being fulfilled.

[NinjaTrader_CalH]

Hello Ravish,

Thank you for your note.

You do not have the subsequent Bar Series being added to be smaller.

What is implied here is the OnBarUpdate() order in which it gets called with the added Bar Series.

If you have a M1, M3 and H1 bar series with H1 being the primary.
All three of these will call the OBU when the H1 is closed.
H1 will process OBU first since it is the primary, then the rest of the Bar Series in the order they were added in the Initialize().

Let me know if I can be of further assistance.

[Ravish]

Thanks Cal

So what you're saying is I can run the strategy on the M1 as a primary bar, and then just check the BarsInProgress property in OBU to make sure I'm executing in the right context?

[NinjaTrader_CalH]

Ravish,

Correct, you would just need to be sure of which BarsInProgress you want to use for filtering out in OBU