Historical fill processing - Range bar type missing

[misova]

(NT8 Beta3)
Currently there are more types of Historical fill processing on High resolution:

• Minute
• Second
• Tick

I would say, that most interesting bar type for efficient simulation of right fills is Range bar, because it price-based and provides almost ideal precision if used with some smaller value.

But this most useful type for simulation of order-fills is missing.

Using Tick-bars or Second bars is far more less useful, because these bars are not price-based and many same bars can be processed without change of the price, or when very volatile market -> then Second bar can be also too big for detection of right fill.

I would suggest to add there type Range bar type, because that is where the most value is for most correct simulation of order fills.

[NinjaTrader_AlexC]

Hi misova,

Range bars currently exist in both NinjaTrader 7 and 8, though they are built based on tick data. To clarify, are you saying that Range should be a type of historical data in and of itself, without being based off of tick data? If so, can you provide a more detailed explanation, possibly even some examples, of the practical benefit of this?

[Calonious]

I think you're confusing this with the actual DataSeries Range Bar type. This Historical fill gives users the ability to add a secondary DataSeries in which your OnBarUpdate will get called in a more granular fill of the primary DataSeries that you set for the test.

This was not available in NT7 unless you programmed it in.

Now if you're saying that there should be an option for a Range granularity in the Historical Fill Process, that kind of defeats the purpose of this property, as the Tick data will work for that intrabar granularity of the Range Bar.

You can still run a Range Bar type and then set your Fill Process to use tick data meaning have your logic run on a shorter secondary DataSeries but display your results on a the primary DataSeries .

[misova]

Ok so to clarify this:
(I think all cases using Unmanaged mode).

In NT7 - we had an option to send orders into specific dataseries. I used mostly Range-1 bars, that were added to the strategy programtically. This worked well.

In NT8 - I found, that there is a new option in the GUI - Order fill resolution - I naturally expect, that this GUI settings replaces functionality - that was previously implemented in NT7 programatically.
So I set Order fill resolution to value High - which enables additional selection of Type. I set this value to "1 Tick".
Now I expect, that by this setting - strategy has already predefined dataseries, where the orders will be sent.
I would like to use Range-1 bars, bar there is not such option - that is why I asked for it. I thought, I cannot use Range-1 bars for filling orders, because in GUI is set another value 1 Tick.

Now - I think I know the source of confusion - I think these new settings are mostly used in Managed Mode for simple users, but when using Unmanaged Mode - I can still send orders to my specific dataseries, as I did it in NT7. Am I right ?

If yes, then how
it will work, when I set in the GUI 1-Tick data, but then programatically set Range-1 bars? Which one will be used? Or they GUI settings will be overriden programmatically?

[misova]

Thankx Calonious,

The reason, why I asked for 1Range bars instead of 1Tick bars, is that
there are a lot more tick-bars, then range-bars.
For example: backtest can process 10 millions tick-bars - when using tick-data.
But when using 1Range bars - there can be only 1 million of bars.
Both have the same precision, but in when using tick-bars, processing will be 10x longer.

It is often sufficient to use bigger range bars like 2Range bars / 4 Range bars,
that still provide overally great precision along with fast and efficient backtesting.

Backtesting with pure 1tick-data is often big waste of processing without any value
for order-fill resolution.

---
Now I am slowly getting the idea - that the old functionality in NT7 - where in Unmanaged mode we programatically set the dataseries, where orders were send - is probably untouched and remains the same in NT8.

I just do not understand how the new GUI settings interfere / replace functionality, which we can do programatically.

[koganam]

Thankx Calonious,

The reason, why I asked for 1Range bars instead of 1Tick bars, is that
there are a lot more tick-bars, then range-bars.
For example: backtest can process 10 millions tick-bars - when using tick-data.
But when using 1Range bars - there can be only 1 million of bars.
Both have the same precision, but in when using tick-bars, processing will be 10x longer.

It is often sufficient to use bigger range bars like 2Range bars / 4 Range bars,
that still provide overally great precision along with fast and efficient backtesting.

Backtesting with pure 1tick-data is often big waste of processing without any value
for order-fill resolution.

---
Now I am getting the idea - that the old functionality in NT7 - where in Unmanaged mode we programatically set the dataseries, where orders were send - is probably untouched and remains the same in NT8.

I just do not understand how the new settings interfere / replace the functionality, which we can do programatically.

I agree completely with your reasoning, not the least reason being that I also used 1-range bars for intrabar fills in multitimeframe tests. Why? A 1-range bar means that the price actually moved, implying a more reasonable chance that the price was traded through, rather than touched: multiple ticks can come in at the same price.

[NinjaTrader_Matthew]

You can still use the NT7 approach if you'd like, you can submit orders to different series if you add them manually. That has not changed. You can use this approach for both Managed and Unmanaged orders. If you do program a strategy with a AddDataSeries(), you will not be able to run a backtest with High Order resolution and a message similar to below will be displayed:

High Order Fill Resolution is only available for single-series strategies. For multi-series strategies, please program directly into your strategy the more granular resolution you would like to simulate order fills with. Backtesting continued with 'Standard (faster)' Order Fill Resolution

The High Order Fill resolution can also be used for managed or unmanaged orders. The advantage here is you do not need to explicitly program your strategy to add the higher resolution, nor do you need to program you strategy to submit orders to another bar series. This is ideal for non-programmers and programmers alike who do not want to waste valuable development time overcoming backtesting limitations.

To be clear, High Fill Order resolution has nothing to do with OnBarUpdate. If you add a high 1-tick series, it will not call OnBarUpdate any more frequently and should not delay the processing of the information in that method. All this option is going to do is provide tick data for internal execution engine to fill you on a the best price inside the bar your strategy is testing on. The performance between a range and a tick bar would be negligible in this new mode. In fact, if you set a Range bar instead a Tick bar, the timing of your execution could be delayed since there is no gurantee that a range bar will form at the correct time the order should have executed. This is why we opted for tick, minute, second updates - since this is a timing issue, not a price issue. The added data series must be more granular than the primary series for accuracy, and there is just no way to gurantee that for something like a Range bar or other types which do not rely on time.

If you do prefer range bars, you can always use the NT7 approach with AddDataSeries()

[misova]

HI Mathew,
thank for your great explanation how things internally work in NT8.

For me, this is completely clear now.

I would just add, that this would useful to add into documentation.
There will be more power-users / programmers, that will also have questions
how the GUI vs. programmatic functionality for setting order-fill resolutions work together and how these mechanisms coexist together without influencing each other.

[NinjaTrader_Matthew]

Absolutely agree - there will be a learning curve for sure. Documentation is an ongoing process throughout beta. We hope to have more resources put together once we go to the final production release to better equip the general user base as people start to migrate. For now, your feedback is very important in driving this process and this thread has given me some great ideas on what we should focus on in this area.