Backtesting with Multiple Time Frames - Strategy Analyzer question

[gredenko]

When adding data series to my strategy, I was previously adding 15 minute, 30 minute, 1 hour, etc in my code, and when I backtest my strategy, I would enter 5 minutes in the strategy analyzer. This would result in the 0 index data series being 5 minutes, and the other data being indexes 1, 2, and so on.

My question is, if I add the 5 minute data in my code, and all my code references the values of the indeces while never using index 0 (so I use index 1 for 5min, 2 for 15 min, etc), should my trading results be equal regardless of what value I enter in the strategy analyzer? If I enter 1 minute in the analyzer or 5 minutes, will my results be the same since the code only looks for data series of index 1 and greater?

I've found that they are not equal, and so I want to find out if it's a bug in my code where I'm still referencing that value or not. The main reason I would like this to work is so I can be sure my code is working the same for whichever chart I choose to view.

[NinjaTrader_Jim]

Hello gredenko,

Thanks for your question.

If you program in your own additional data series and skip all operations of the primary data series, it would be expected that historical trades performance (backtesting) results would match.

You could perform a similar check to skip iterations on the primary data series:

if(BarsInProgress == 0) return;

Order submissions would need to be sent to the additional data series, and any Price Series would need to have the correct BarsInProgress reference.

As a rough example: EnterLongLimit(1, true, 1, Lows[1][0], "Entry");

The openly available Multi Time Frame & Instruments documentation can provide further direction for troubleshooting your multi series script.

The Multi Time Frame & Instruments - https://ninjatrader.com/support/help...nstruments.htm

Please let us know if we may be of further assistance.

[gredenko]

Thanks for mentioning the bit about order submissions. If using market orders, will it make a difference what time frame I'm on when submitting the order?
I have recreated a new strategy, with very little of my original code in it, but I am still getting some different results across different values in the strat analyzer. I am only using market orders and stop loss orders, entering long when the 5 minute MACD is greater than 10 (random simple logic). When I backtest in the daily chart vs. the 5 minute chart, I get several extra trades on the 5 minute vs. daily.

[NinjaTrader_Jim]

Hello gredenko,

The data series that the order is submitted to will affect the result.

If you have not taken a look yet, we have an example strategy on backtesting with intrabar granularity.

I would recommend to submit orders to a single tick data series and then use your other data series' to drive your strategy logic. This way, you will have fills that give a closer representation of real markets, and you can maintain consistency of programming in your own data series for strategy logic.

Here is a link to our backtesting with intra-bar granularity example: https://ninjatrader.com/support/foru...ead.php?t=6652

[gredenko]

Thanks for sharing that example. I noticed it says "Very Important: This secondary bar series needs to be smaller than the primary bar series." Why is that?

I know this crossover is being checked on the primary (5 minute) series, but if you were checking only on secondary or other data series, would the comment still be relevant?

I've attached my simple strat here, and would appreciate it if you could take a look and help me find out what might be causing my different results.

Basically, I add several data series, and then pull the desired one using getIndex(time_frame) which I made sure to use throughout my code. I don't see any remaining references to the primary data series, so I'm not sure what may be missing.

[NinjaTrader_Jim]

Hello gredenko,

The strategy will need to submit orders to a data series smaller than it is iterating on. It would not be logical to say, submit an order to a 15 minute data series when our primary data series iterates every minute. We would then see mismatched results as the primary series logic could exit our positions before the entry actually occurs.

For your script, I would still advise to skip out of BarsInProgress == 0 instances of OnBarUpdate() and then to use a single tick data series for order submission. For example:

if(BarsInProgress == 0) return;

// additional logic

EnterLong([I]BarsInProgress for your 1 tick data series[/I], Quantity, Signal);

Your strategy would then:

• Use a single tick data series for order fills to more closely represent real market fills
• Only submit orders when your programmed data series allow it

Please let me know if you are not seeing matching backtest/historical trade performance when changing the primary data series after taking these steps.

[gredenko]

I'm not sure if I'm not understanding something with OnBarUpdate, but I cannot get my program to run properly still.

I've added a 1 tick data series, and my strategy is able to skip the primary bar, enter my trade logic and log a message with Log(), after my stoploss/target are set and my position should have been opened.

To open my position, I'm using EnterLong(getIndex(0),DefaultQuantity,""); where getIndex(0) returns the 1-tick data series. I've tried on multiple time frames in strategy analyzer, but the Log() message gets printed for several different acceptable trades, and no trades get taken.

What might I be missing still?

Code:

if (valid(true)) {
target_long = getTarget(true);
long_stop = Lows[getIndex(5)][0] - loss_size; //bottom of last 5 minute candle
setProfitTarget(target_long); //sets target +100 for now
setStopLoss(long_stop);
takeLong(); //(this just calls EnterLong(getIndex(0),DefaultQuantity,""))
log(Times[getIndex(5)][0]+" Profit target set to "+target_long+". Loss set to "+long_stop+"."); //this gets printed, but no trades get taken
}

[NinjaTrader_Jim]

Hello gredenko,

I've created a video demonstration showing what steps would be necessary to create a strategy that uses intrabar granularity and submits orders on a manually added data series. The result is a strategy that operates off of its own added data series as opposed to using the primary data series given by the user.

Demo - https://www.screencast.com/t/XimwKfKGLLy3

			else if (State == State.Configure)
			{
				AddDataSeries(Data.BarsPeriodType.Tick, 1); 	// BIP 1
				AddDataSeries(Data.BarsPeriodType.Minute, 30); 	// BIP 2
			}
		}

		protected override void OnBarUpdate()
		{
			if(CurrentBars[0] < 2 || CurrentBars[1] < 2 || CurrentBars[2] < 2)
				return;
			
			if(BarsInProgress == 0 || BarsInProgress == 1) // Skip primary series and single tick series
				return;
			
			if(Close[0] > Open[0] && Position.MarketPosition == MarketPosition.Flat)
				EnterLong(1, 1, "Entry");
			if(Position.MarketPosition == MarketPosition.Long)
				ExitLong();
		}

Please take the same steps as I have before working the implementation into your strategy. If you have any additional questions, please don't hesitate to write back.

[gredenko]

Thanks for taking the time to make the video, but I am still having some issues.

I've removed almost all of my code and just used your OnBarUpdate and added the 1 tick + 30 minute dataseries, but my program simply doesn't run.

It simply doesn't reach the section of code to take the trades, so nothing happens. Are my settings interfering in some way?

Another quick question: Why did you use Close[0] and Open[0] in the example? Why doesn't this cause variance in the backtesting results (unless it does)?

Calculate = Calculate.OnBarClose;
EntriesPerDirection = 1;
EntryHandling = EntryHandling.AllEntries;
IsExitOnSessionCloseStrategy = false;
ExitOnSessionCloseSeconds = 30;
IsFillLimitOnTouch = false;
MaximumBarsLookBack = MaximumBarsLookBack.TwoHundredFiftySix;
OrderFillResolution = OrderFillResolution.Standard;
Slippage = 1;
StartBehavior = StartBehavior.WaitUntilFlat;
TimeInForce = TimeInForce.Gtc;
TraceOrders = false;
RealtimeErrorHandling = RealtimeErrorHandling.StopCancelClose;
StopTargetHandling = StopTargetHandling.PerEntryExecution;
BarsRequiredToTrade = 20;


Here is my OnBarUpdate:

protected override void OnBarUpdate() {
if (CurrentBars[0] < 2 || CurrentBars[1] < 2 || CurrentBars[2] < 2)
return;

if (BarsInProgress == 0 || BarsInProgress == 1) // Skip primary series and single tick series
return;
if(Close[0] > Open[0] && Position.MarketPosition == MarketPosition.Flat)
EnterLong(1, 1, "Entry");
if (Position.MarketPosition == MarketPosition.Long)
ExitLong();
}

[NinjaTrader_Jim]

Hello gredenko,

The strategy options should be fine unless you see that the strategy has not reached a flat state when enabling from a chart or from the Control Center. (See Wait Until Flat Start Behavior for more information.) I would recommend testing in the Strategy Analyzer as I have with the code provided before modifying your script.

The code I gave you along with the demonstration should work if it is copied into a blank strategy verbatim. Were you sure to include the AddDataSeries() portion of the script in OnStateChange() under State.Configure?

Checking Close being greater than the Open is an arbitrary condition. As I mention in the video, Close represents the Close of the iterating bar while Closes[][] references the closing price of a specified bar series.

You mentioned that the code simply does not reach the section of code to take trades. When you add prints to the code to see how far the strategy executes, where do you see your last print? When you print out the variables of any of the conditions used, are they all becoming true and allowing the strategy to submit orders?

I have included some resources on Debugging NinjaScripts and Start Behaviors (Wait Until Flat will not start submitting orders until the strategy is in a flat "virtual position" based off of the simulated trades made with historical data.)

Debugging NinjaScript code - https://ninjatrader.com/support/foru...ead.php?t=3418

Syncing Account Positions (Wait Until Flat) - https://ninjatrader.com/support/help...#WaitUntilFlat

If you would like additional help debugging your strategy I could have a member of our business development team provide a list of NinjaScript Consultants who would be happy to debug your code with you.

[gredenko]

I did add some prints earlier and they were not being reached after this following chunk, and so it was just always returning for some reason. I tested over a period of Jan 1 2015 to Nov 06 2015.

if (CurrentBars[0] < 2 || CurrentBars[1] < 2 || CurrentBars[2] < 2)
return;

if (BarsInProgress == 0 || BarsInProgress == 1) // Skip primary series and single tick series
return;

I did add the DataSeries also, as below:
else if (State == State.Configure) {
AddDataSeries(BarsPeriodType.Tick, 1);
AddDataSeries(BarsPeriodType.Minute, 30);

I can create a new strategy and copy paste your code, which I am sure works, but if it's a setting or something else that may have been giving me issues earlier, I'd like to know it and fix it in my own code for future reference. So if you do have any ideas please let me know.

[NinjaTrader_Jim]

Hello gredenko,

Thanks for your reply.

I have uploaded the complete demonstration strategy with additional prints that can show you the progression of the strategy logic. You may test this strategy in the Strategy Analyzer, Market Replay, or the simulated data feed with the NinjaScript output window open to observe the logic as it executes.

I encourage you to observe how I have written the prints so they tell me what is happening in the strategy for each line of code. This approach makes print statements an invaluable tool for debugging.

Please make sure to set BarsRequiredToTrade to 0 to prevent the primary data series from restricting trades on the secondary series and changing the progression of trades.

[gredenko]

Thank you for all your help with this strange issue. I ended up replacing bits of my code with yours until I found the issue: my overloaded OnPositionUpdate/OnOrderUpdate functions that were empty. I guess when I was executing trades, it was entering those and doing nothing with the order whenever there was one. After taking those functions out, I'm able to get results.

However, even with your code, I'm getting different results when trying 1 minute, 15, 35 minute intervals. I'm not sure if this means there's an issue with the settings when creating the strategy, because I don't know what else to think of. The code is copy pasted from your file into mine, and then I get wildly different results in the strategy optimizer.


Afterward, I tried creating a new strategy with some of my typical settings (exitonclose = false, slippage = 1), and copy pasted your code onto the new strategy, and again it does not return any results when it is run. I am getting more confused with each attempt. I'm not sure what's causing the code to run or not run anymore. I'd appreciate any tips. The new strategy features your entire file with the strategy name changed and then it simply won't run.

[NinjaTrader_Jim]

Hello gredenko,

I am not sure what modifications you have made to the example. I may suggest using a diff checker tool to see if there are any over looked differences.

As another tip, you could leave a copy of the example on hand, and then make modifications to the code line by line and testing after each addition to ensure that the modifications are made appropriately.

We can also have a member of the Business Development team reach out with information on NinjaScript Consultants who would be happy to debug your code with you.

Please let me know if I can be of further help.

[gredenko]

Hello gredenko,

I am not sure what modifications you have made to the example. I may suggest using a diff checker tool to see if there are any over looked differences.

As another tip, you could leave a copy of the example on hand, and then make modifications to the code line by line and testing after each addition to ensure that the modifications are made appropriately.

We can also have a member of the Business Development team reach out with information on NinjaScript Consultants who would be happy to debug your code with you.

Please let me know if I can be of further help.

I may have fixed the issue by connecting with a new demo account and using a brand new strategy. I will update this if needed. Thank you again for the help!