NT 's support is the best I've dealt with in 30 years. Back test accuracy question.

[demarcog]

Hi,

I appreciate all the help Ninja support provides . I've worked on Wall St. as an IT professional for for 30 years both as a provider and consumer of many software products. I can say unequivocally Ninja Support is by far the absolute best product I've ever worked with and the Support in particular is the best I've ever experienced. IBM, Microsoft, Apple, Oracle, HP Microsoft, SAP, etc. I've , dealt with ton a daily basis for decades and they are not even in the same ballpark as Ninja Trader. If you don't believe me try opening up a ticket at 5:00pm on a Friday night or anytime at all with anyone of them and see what happens. Inexperienced people may not be aware of the level of support we are getting and sometimes I've heard a few vague grumblings in a room where I trade so wanted to set the record straight.

I realize this is a well discussed issue and apologia for the redundancy but for me as a system developer t is the single most important feature of the product and with COVID Hhow I make my living or not, so apologies for beating a dead horse.

My only issue and for me it's a major problem is my systems utilize non time based charts and I realize historical back testing using only OHLC data makes it impossible to have an accurate back test report. In addition I'm assuming the discrepancy goes well beyond slippage with is minimal for the most liquid instruments and or can be factored in via the slippage paramete.

Every tick can be downloaded and is downloaded prior Playback. I'm getting such massively different numbers running a back test on a unirenko , or renko , or point nd figure bar for example of a system that back tests with $1.00 a day for argument sake to loosing $5.00 a day. and I specify Calculate on each tick and Order Fill resolution of High. It's impossible to Playback in normal speed for each change in a strategy to actually find out how it performs. I also specify the most recent bar[0] for all my indicators and take not strategy positions open/close intrabar, always On Bar Close.

If the tick data is downloaded is there anyway to have the Back Test application program use available he tick data when running a back test?


When you specify Order Fill Resolution High, Type Tick and Value 1 , that seems to imply the tick date will be used to insure an accurate back test for non standard bar types? Can that be requested as an enhancement if this is not what is happening. The fact that you can download the tick data, that Ninja during normal hours builds the bars based on that real time tick data and generated orders aligned with the current bid//ask tick data seems to indicate that the functionality is already there and done in Playback mode.

With Unirenko or renko bars with OHLC you do not know the actual open .

With Unirenko if you specify a 1 tick Trend and a 1 tick offset, then the open and of the next bar should be the exact real opening price of the next bar which eliminates that inaccuracy and the reversal setting/price would also be the actual price so with a Unirenko setting of Trend 1 Offset 1 and reversal of X shouldn't that generate a accurate back test even without having the tick data available?

If not is there anyway of incorporating the downloaded tick data in the back test ?

If not can you suggest anything to have the back test with Unirenko or renko come more closely to the actual Playback/Sim results?

I would be more then happy if 75% of the profits disappeared but to have a system report with a 50% success rate drop to a 10% success rate make developing a successful trading system infinitely more difficult.

Thank you all so very much for your continued great support.

glen

[NinjaTrader_Kate]

Hello demarcog,

Thank you for your post.

Setting your Order Fill Resolution to high using a 1 tick series will only improve the accuracy of the prices that orders are filled at. The primary series does not use this additional 1 tick series to build the bars for the primary series, the primary series will still only be able to use the OHLC of the bars.

Since Unirenko are not a built in bar type I can't speak to that, unfortunately. However, we do have an existing feature request to allow the use of Market Replay data in the Strategy Analyzer if you'd like me to add a vote to that.

Renko based bars will be inaccurate in backtesting because even when using a single tick series, the historical time stamps of several ticks may be the same and the historical data doesn't necessarily know which actually came in first. This is problematic because a single tick in difference in a bar can create a completely different chart.

We would suggest if you're using Renko to use Market Replay data in the Playback Connection for testing as this will be the most accurate to real time.

Please let us know if we may be of further assistance to you.

[demarcog]

Hi Kate,



Thank you for the response it is appreciated.

Yes please do add my vote to the use of the Market Replay data to the Strategy Analyzer.

If I could advocate a bit further regardless of a users desire to have this feature, wouldn't providing the most accurate back test/optimization report possible be a desired design objective? Years ago tick data was only available typically through a different vendor (Tick Data") and not managed nor integrated with platforms such as eSignal and Tradestation. With this data already integrated and available (if downloaded) with the Ninja platform seems to suggest that for strategies that utilize Calculate on Bar Close that the tick series will only be required for bars where trades occurred. Is this something that could be provided as an Add On rather then requiring integrating the code changes with the Ninja Product itself?

It's interesting and fascinated by the statement "that a 1 tick series will only improve the accuracy of the prices that orders are filled. at". Because that is the root of all the problems with inaccurate back test problem with all my back tests.

I'm sorry for for not being smart enough to understand why this wouldn't resolve my problem as aside from a bar where an order is executed I have all my strategies set to use "On Bar Close" for indicators .I'm even perfectly fine with opening and closing positions at the opening price of the next bar. It seems the 1 tick series is specifically designed for this situation and don't understand why with the tick series the back test is so inaccurate?


If even tried to readjust and using a unirenko setting of 1 tick trend and 1 tick offset which my understanding is that the that the open of the next bar is at the exact opening price 1 tick above of below the close of the last bar even without the 1 tick series why wouldn't this results in an accurate price execution price in the back test and especially with the 1 tick series being used for fills why are the fills so far off?

All the discrepancies are with fill prices and still do not understand how with the settings, using On Bar Close for everything and Order Fill Resolution 1 tick that I'm seeing these discrepancies. If I picked the worst fill price, highest or lowest tick against the position for every trade this would still not account for the huge differences.

Is it possible the discrepancies go beyond the obvious and I am seeing another issue that perhaps we can investigate?

Thank you,

Glen

[NinjaTrader_Kate]

Hello demarcog,

Thank you for your reply.

I have added a vote for you to a current feature request to allow the use of Market Replay data in the Strategy Analyzer. This request is being tracked under the number SFT-3979.

As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted in the Release Notes page of the Help Guide.

Release Notes — https://ninjatrader.com/support/help...ease_notes.htm

Since there are many diffferent versions of Unirenko bars and there is not one built-in to the platform, I would not be able to offer support there. Renko type bars do not work properly with historical data as it does not allow them to redraw the bar and large discrepancies between a backtest on historical data is expected.

We would look at it like this: If you are using a bar type that cannot reliably be used with historical data because it rewrites the open, you can use Market Replay with the Playback Connection to test this. The Strategy Analyzer is meant to allow you to quickly see how the strategy may have fared over a long time period, and is there so you can refine your strategy and then test using Market Replay to further refine prior to going live. If you are using Unirenko bars with the Strategy Analyzer, regardless of settings this would still be expected to return results better than you would see in real time. Even with tick data, the problem is that there can be several ticks with the same time stamp. Since NinjaTrader has no way of knowing historically which returns first, it is very likely that a tick could be assigned to a bar historically that it was not in real time, causing variations in the chart.

Please let us know if we may be of further assistance to you.

[demarcog]

HI Kate,

Thank you very much for the very informative response it is very much appreciated. I appreciate you adding my vote to the change process. For something like have as accurate a back test as possible using tick data for the strategy analyzer one would think that unlike some other requested enhancement where the number of people requesting is a factor, this goes to the accuracy of the product itself.

Apologies but I do not understand what you mean by redraw the bar and I hope you don't mind my follow up questions. I am the dummy here please don't interpret my questions as disagreeing in any way with what you stated. It's just really important to me to understand fully what we are discussing.

I used to do point and figure by hand and time is completely ignored, once a price tick hitting your your threshold for box size and reversal hits a certain price the box is drawn and can never change. With the tick stream you know exactly when a stop/limit market order would be executed or the time/price pair for when an order executed. I worked with a friend coding this in javascript for the eSignal platform and our back test reports that processed tick data were accurate. I would share gladly the code it's quite a bit and to large to attach here.

Similar case for Unirenko time isn't factored. NT builds the chart type now real time and does not have to redraw a past historical bar.
Renko agreed is a bit of a hybrid on some platform where a time interval is specified.

I understand he issue n the multiple time stamps with the same or different prices. You do have the sequence number of the tick records coming in..

Real time NT and the playback programs have this functionality. now. As long as you have all the ticks you can draw the bars and record accurate prices.

It would seem that many features of NT8, for example, the strategy analyzer, the strategy optimizer, the strategy builder, the AI strategy analyzer code builder were much more difficult problems to solve then an accurate back test for non time interval types.

Thank very much,

glen

[NinjaTrader_Jim]

Hello Glen,

This is Jim responding on behalf of Kate who is out of the office at this time.

Renko bars and Line Break bars feature functionality to call a RemoveLastBar method to remove the last bar, so the BarsType can call AddBar to place a new open. This is done with the reversal behavior of the BarsType. Since this involves removing the last bar with active data and redrawing it, this behavior can only be simulated with realtime data.

Please let us know if there is anything else we can do to help.