How to pass Primary DataSeries to Custom Method

[koganam]

Thanks, koganam, I'll have to think about this. The challenge is that I am trying to first find the actual price that caused the cross to occur, and then I am trying to use that price to calculate what the ATR would have been at that price. Given my goal of wanting to create a sound backtest, I'm beginning to believe that the only way I'll be able to do this is by creating synthetic OHLC DataSeries variables that are then passed to a custom ATR Method that takes the synthetic OHLC DataSeries variables and calculates the ATR. As of right now, my triggerDataSeries only tracks the Close values--and not the OHL--which makes calculating the ATR impossible. So I need to fix this...

I've been trying to avoid re-writing each indicator I backtest, but clearly in some cases I'll have to.

The frustrating thing is that I'll have two different strategies to test the same idea: one for backtesting and one for forward testing--because with forward testing I'll be able to use the stock indicators.

The deeper I get into this backtesting wormhole, I'm surprised that there is not wider consensus on how to code an intrabar strategy backtest. Maybe I just don't understand how to properly use the lower time frame secondary series (i.e., 1 min) to create sound backtest results.

How are you backtesting your strategies? How would you backtest the simple MACross strategy? (Side note, clearly I'm not interested in trading the MACross, I'm just trying to use it to understand how to create a sound backtest).

Thanks for your help!

Aventeren

I backtest intrabar by using a secondary 1-range barSeries. However, I hardly ever actually test intrabar. I find such fine-grained testing unnecessary, as a retail trader. If I were designing HFT, it would be a different kettle of fish. For retail trading purposes, as far as I am concerned, a bar speaks only when it closes. I have seen too many times when conditions whip back and forth, creating numerous losses for Strategies that do not wait for a bar to close.

In fact, because of this, despite "the customer being always right", I generally refuse to create Strategies that do not trade on barClose. They take extremely long to optimize, and then when I actually run statistics on them, I discover that they are often just tightly curve-fit on a lookahead bias, and unviable.

Every Strategy that I have supplied to brokerage auto-trading partners enforces COBC=true. They can change it to false: I simply return it to true.

[koganam]

Just to confirm, if I my triggerDataSeries contains historical values equal to the corresponding bar indexed Close values (i.e., I Set triggerDataSeries[0] to Close at the end of OnBarUpdate) and a triggerDataSeries[0] value equal to a temporary "test" value, when I pass the triggerDataSeries to the NT EMA indicator, the value I receive back should be equal to what the EMA would be with the "test" value (i.e., triggerDataSeries[0] was Set to "test" value) and the historical Close values, correct?

IOW, the EMA doesn't care what DataSeries you give it, because it will calculate an EMA based on those values--and therefore the EMA can be passed a DataSeries type and does not require an IDataSeries type?

I'm seeing that there is a distinct difference between a DataSeries type (which just holds one bar indexed value) and an IDataSeries type (which appears to hold the bar indexed OHLC values). And furthermore, that many of the stock NT indicators require IDataSeries type inputs, which DataSeries are decidedly not?

Can someone please conclusively state that an IDataSeries type is NOT a DataSeries type? It just seems to me that they share the words "DataSeries"--but that their only common feature is that they are bar indexed, and that a DataSeries type will not work as passed parameter to an indicator that requires an IDataSeries type?

Boy we're in the weeds. I hope this isn't getting confusing. Am I overthinking this?

Thanks,

Aventeren

Yes, you are overthinking it, unfortunately. IDataSeries is an Interface to a DataSeries. Every indicator in NT can take a DataSeries as a parameter.

[aventeren]

I backtest intrabar by using a secondary 1-range barSeries. However, I hardly ever actually test intrabar. I find such fine-grained testing unnecessary, as a retail trader. If I were designing HFT, it would be a different kettle of fish. For retail trading purposes, as far as I am concerned, a bar speaks only when it closes. I have seen too many times when conditions whip back and forth, creating numerous losses for Strategies that do not wait for a bar to close.

In fact, because of this, despite "the customer being always right", I generally refuse to create Strategies that do not trade on barClose. They take extremely long to optimize, and then when I actually run statistics on them, I discover that they are often just tightly curve-fit on a lookahead bias, and unviable.

Every Strategy that I have supplied to brokerage auto-trading partners enforces COBC=close. They can change it to false: I simply return it to true.

I really appreciate your perspective, koganam. Perhaps I am overthinking this.

I was told by NT Support (Support Thread Here") that to use a 1 Range secondary series that you then had to use a matched higher time frame ("HTF") primary series--so in this case a (1 + X) Range primary series. Other examples would be a 800 tick - 1 tick, 60 min - 1 min, 15 Range - 1 Range, etc.

So when you use your 1 Range secondary, are you also using a Range primary?

Have you found a way to use a 1 Range secondary series to backtest a non-Range primary series? If so, would you mind dropping a few hints?

Again, I really appreciate your perspective, koganam.

Thanks,

Aventeren

[koganam]

I really appreciate your perspective, koganam. Perhaps I am overthinking this.

I was told by NT Support (Support Thread Here") that to use a 1 Range secondary series that you then had to use a matched higher time frame ("HTF") primary series--so in this case a (1 + X) Range primary series. Other examples would be a 800 tick - 1 tick, 60 min - 1 min, 15 Range - 1 Range, etc.

So when you use your 1 Range secondary, are you also using a Range primary?

Have you found a way to use a 1 Range secondary series to backtest a non-Range primary series? If so, would you mind dropping a few hints?

Again, I really appreciate your perspective, koganam.

Thanks,

Aventeren

I do not know why you were told that, and it may well be true, but not in my experience. I use a 1-range secondary series rather than a 1-tick one for the simple reason that the former is a closer simulation to reality, where the market must usually trade through a price before it fills the order. Moreover, it is far less processor-intensive to use a 1-range series instead of a 1-tick series, where multiple ticks will come in at the same price on a really liquid equity.

My signals are taken from the higher time frame series and executed on the 1-range series. Granted that a lot of my automated (and manual) trading strategies depend on Range Bars, I have also optimized on time-bars.

[aventeren]

I do not know why you were told that, and it may well be true, but not in my experience. I use a 1-range secondary series rather than a 1-tick one for the simple reason that the former is a closer simulation to reality, where the market must usually trade through a price before it fills the order. Moreover, it is far less processor-intensive to use a 1-range series instead of a 1-tick series, where multiple ticks will come in at the same price on a really liquid equity.

My signals are taken from the higher time frame series and executed on the 1-range series. Granted that a lot of my automated (and manual) trading strategies depend on Range Bars, I have also optimized on time-bars.

His point was that Range and Tick bars are not synced to time charts, so running a 1 Range chart on a time chart is impossible because the secondary series would not know where to pick start and stop in the context of the primary bar--unless of course you programmed in some ToTime() functions to ensure that the secondary series was only operating within the boundaries of the primary bar.

I sure would be interested in seeing a primary time chart (minute, day, week, etc) coupled with a 1 Range secondary bar for back testing, but heretofore I have been unable to see one in the wild. That's what I was trying to do with my FindTriggerPrice() Method, but alas I have hit a wall.

To the extent you would be willing to share an example, I'd love to see it.

Thanks,

Aventeren

[koganam]

His point was that Range and Tick bars are not synced to time charts, so running a 1 Range chart on a time chart is impossible because the secondary series would not know where to pick start and stop in the context of the primary bar--unless of course you programmed in some ToTime() functions to ensure that the secondary series was only operating within the boundaries of the primary bar.

I sure would be interested in seeing a primary time chart (minute, day, week, etc) coupled with a 1 Range secondary bar for back testing, but heretofore I have been unable to see one in the wild. That's what I was trying to do with my FindTriggerPrice() Method, but alas I have hit a wall.

To the extent you would be willing to share an example, I'd love to see it.

Thanks,

Aventeren

Then there is a fundamental misunderstanding here of the juxtaposition of time and price. You are overthinking things. Synching is automatic, because it depends only on time. You check the larger time frame for a trigger, and execute the order on the 1-range bars.

IOW whatever happens after you receive a trigger happens after you receive a trigger. You are simply executing on a smaller bar series, because the smaller bar series closes its bars much more frequently than the larger time frame bars, and hence provides a barUpdate more frequently (i.e., effectively, every bar update on the 1-range barSeries is an intrabar update on some bar of the larger time frame series). The issue is not on which bar it is related, the only issue is: "when is the trigger; when is the order". The order will always be subsequent to the trigger. You just need to use the proper BarsInProgress index to differentiate which barUpdate event you are handling.

[aventeren]

Then there is a fundamental misunderstanding here of the juxtaposition of time and price. You are overthinking things. Synching is automatic, because it depends only on time. You check the larger time frame for a trigger, and execute the order on the 1-range bars.

IOW whatever happens after you receive a trigger happens after you receive a trigger. You are simply executing on a smaller bar series, because the smaller bar series closes its bars much more frequently than the larger time frame bars, and hence provides a barUpdate more frequently (i.e., effectively, every bar update on the 1-range barSeries is an intrabar update on some bar of the larger time frame series). The issue is not on which bar it is related, the only issue is: "when is the trigger; when is the order". The order will always be subsequent to the trigger. You just need to use the proper BarsInProgress index to differentiate which barUpdate event you are handling.

That makes a ton of sense. Thanks.

I'll work on that and post up what I ended up doing. My hope is that others will come across this thread and not have to crawl the backtesting learning curve like I am currently doing. To that end, I thank you for your help.

There is a saying in Spanish that says "Poco a poco se va legos", which translates roughly to "Little by little, one goes far." Clearly that's where I'm at.

Thanks a ton.

All best,

Aventeren