Fill Logic

[GrumpyTrader]

In NT7 you could define your own fill logic. Does that still exist in NT8?

[NinjaTrader_DaveI]

Hello,

It is not possible to create custom fill logic in NinjaTrader 8 the way you could with NinjaTrader 7. With the added complexity of order fill resolution in NinjaTrader 8, including things like Tick Replay, multiple fill resolutions, and the like, we decided not to open this up for custom development at this time.

Can you elaborate a bit on what your goal would be with custom fill logic? There may be another way to accomplish what you are looking for, or it may illuminate us as to a potential enhancement to make in the future.

[GrumpyTrader]

Hi Dave,
I made it so that I could execute same bar close orders. This has been hashed out numerous times in the forum. I understand your guys stance on not allowing these order for backtesting. It is just such a pain to work around this limitation using a second dataset. All the hacks you have to do to indicators to get partial bar calculations. I really wish you guys would address this issue in a better way. My suggestion would be to add a daily dataset that has a partial bar option. That way you can still use all the existing indicators with no modification.

[NinjaTrader_DaveI]

Thank you. I'm still not 100% clear on the goal, but I may be missing something obvious. The High Fill Resolution was designed to eliminate the need to add a secondary data series to achieve intra-bar fills. You can now fill orders down to a 1-tick granularity in the Strategy Analyzer, regardless of the primary bar interval you are using to test. Tick Replay was designed to allow for intra-bar, tick-by-tick indicator calculations on historical data. Is there a specific scenario in which these are not meeting your goal?

[GrumpyTrader]

Hi Dave,
Unless I'm missing something, (which is entirely possible) the Fill Resolution doesn't solve the issue I am talking about. Here is a simple example:

Dataset:
1 Day Bars

Entry rules:
a) Close[0] < SMA(5)
b) Close[0] > SMA(200)
c) High[2] < High[3] && High[1] < High[2] && High[0] < High[1]
d) Low[2] <Low[3] && Low[1] < Low[2] && Low[0] < Low[1]

Exit rules:
a) Close[0] > SMA(5)

buy/sell on the close of the bar (in reality some specified time before the close).

Fill resolution doesn't help you here as you need to calculate the 5/200 day sma using a partial bar calculation where the last observation isn't the prior day close but the most recent price traded.
For a SMA that is quite simple: I add a data series (say 1 minute) and calculate the sma by going:
(SMA(1DaySeries, 4)[0] * 4 + Close[1][0]) /5
But clearly as you use more complicated indicators this becomes very difficult and time consuming to figure out the partial derivative. Additionally, adding the additional dataseries also makes running back tests slow. You are also limited by how much 1 minute data you have.

In other platforms I would backtest just using daily bars on bar close orders to find statistical edges and then code it for execution with executions happening [x] seconds before the close. I also experimented with market on close orders but found I had more phantom orders that way as I had to calculate the entry 20 minutes before the close to submit the order. This first approach resulted in very little slippage over time.

In my opinion the most elegant solution would be to have a partial bar option when creating a data stream (similar to your fill resolution) that would have as the last bar the current (not yet closed) bar price updated at the fill resolution designation. For example, a daily bar with the last bar being the last traded price. This would allow the user to use all the indicators without modification to get the partial value at that point in time. It would also be in keeping to your philosophy of not allowing orders on the close of a bar.

[NinjaTrader_DaveI]

If you have the tick data in addition to the daily data available, then you should be able to use Tick Replay in conjunction with the High order-fill resolution in the Strategy Analyzer (using a Daily interval) to achieve exactly that. Are you finding that you are not getting intra-bar indicator calculations when you have Tick Replay enabled in your backtest (assuming that you have downloaded tick data for the instrument as well)?

[GrumpyTrader]

Ok I found this thread which explains it well.

http://ninjatrader.com/support/forum...ill+resolution

The issues with this approach when backtesting are:
a) cpu intensive
b) time intensive
c) limited by how much tick data you have

[NinjaTrader_DaveI]

I see where you're coming from now, thank you. We are always keeping our eyes open for potential performance enhancements to implement before the full production release, including issues such as this one.

[GrumpyTrader]

Hi Dave,
Here is how I was able to hack a fix. I would appreciate it if you guys could suggest a more elegant solution.

            //  DAILY BARS
            if (BarsInProgress == 0)
            {
                if (CurrentBars[0] > 0)
                {
                    price[1] = Close[0];
                }
            }
            //  5 MINUTE BARS
            else if (BarsInProgress == 1)
            {
                if (CurrentBars[0] >= 0)
                {
                    if (Times[0][0] != Times[1][0])
                    {
                        price[0] = Close[0];
                    }
                }
            }

This will create a price series of daily bar prices that are displaced by one. The current element of the array is the 5 minute close. This allows me to make function calls in the normal manner when intrabar daily bar (EG. SMA(price, 5)[0] ). When I am at the close of a daily bar though I need to calculate with one offset (EG. SMA(price, 5)[1] ). No need for partial derivatives this way. The Times[0][0] != Times[1][0] bit ensures that I use the daily bar settlement price instead of the 5 minute closing price.

It's not perfect as I'm not sure if this works for all bar types and time frames but it seems to work for my special case.

Ideally it would be nice to have the partial bar built into NT8. Logically it seems like it should be a part of the High Fill Resolution.

Do you agree that providing a bar series like this simplifies intrabar / eod calculations?

[GrumpyTrader]

Ok Dave, I ran into a snag with my hack. It seems as though when I try to calculate the intrabar daily calculation, it will calculate the first time then just return the cached value until the bar is incremented. Is there a way to force recalculation? I've attached the code.

[NinjaTrader_DaveI]

Edit: I believe I'm starting to understand the goal now. I'll post back soon with an update.

[GrumpyTrader]

Dave,
I don't think we are on the same page. The High Fill Resolution doesn't do what I am asking.

I am trying to backtest a strategy that uses daily bars and enters on the close of the signal bar ( not the next bar open, see sample strategy mentioned below).

Tick replay isn't practical because a) I don't have 10 years of tick data to make a backtest on daily bars meaningful and b) it takes for ever to back test strategies using tick replay, let alone run an optimization. Also, NT8 will crash around half the time when backtesting using tick replay.

The only solution to my problem that I know of to solve this is to add a second data series of a lower time frame to calculated your rules right before the close to execute your end of day trades.

Is there anything not clear about the above?

[NinjaTrader_Brett]

GrumpyTrader,

We did not make any changes in NT8 that would help satisfy your direct use case unfortunately. To do what your looking to would require AddDataSeries() with a lower time frame such as 1 minute bars. You would need to evaluate your Daily bar condition simulated with the minute bars on the last X bar of the 1 minute series like you found.

If your looking for more background why it is how it is with NinjaTrader a bar technically isn't closed until the tick of the 'next' bar comes in. That's technically how we know to close the bar before it since we're event driven.

Therefor if we changed the logic in backtests to execute on the close of the bar this wouldn't be truly reflective of what you would see live. As you would run such a strategy live that daily bar wouldn't technically close until the tick of the next bar comes in. We don't control when that next bar will come in and in the case of RTH daily bar providers this is the first tick of the next day.Meaning the close of that ddaily bar is 'delayed' compared to what your actually looking for. This is where an underlying minutes series comes into play and you would reference the daily bars values from your 1 minute strategy and take action on the last 1,2,3 bars of the 1 minute series.