Who wants an accurate back test report for every bar type,...vote for SFT-3979.

[demarcog]

********** Cast your Vote for system Enhancement SFT-3979 **********

If any NT customers are interested in an enhancement to a future release that would enable accurate back testing for non time based interval bar types, please cast your vote for enhancement SFT-3979, by replying to this post or opening another.

This modification will incorporate tick data into the Strategy Analyzer to enable a much more accurate back test report for all other chart types: Range, Renko, Unirenko, Point and Figure, Kagi, Line Break and Heiken Ashi .

Recently I have been become increasingly interested in non time based char types. The idea being that a chart based entirely on price changes, eliminating the element of time offer an viable alternative that along with traditionally charts complement one another and will frequently outperform the results of, a single chart type..

The bane of my existence moving forward with the research is that for non time based charts almost every strategy back tested, grossly inflates the profitability to appear as if the "holy grail" of strategies has been discovered. Then in real time trading the exact opposite is the case.

Currently the only way to more accurately test a strategy on non time charts is to run it in a real time SIM account or downloading the tick data and connecting to and running it in Playback Connection mode. Both methods are not viable as a means of testing a strategy and across any significant period of time.

However this enhancement will integrate the instruments tick data with the Strategy Analyzer's current processing of only OHLC bars for calculating entry and exit will be very close to actual trading results.

If this is an important issue to you please weigh in and vote. This enhancement is being seriously considered and the more interest the better the likelihood of it being implemented.

The group I trade with daily whom I cannot mention by name in this forum charges less money a day then my Starbucks espresso. It's a live real time live market chat/chart room looking at live charts and pointing out the actual setup as they occur. The when a trade is taken you see the entry and exit exact entry and exit. In several months I've not seen a loosing day in Gary's afternoon Futures trading room. He makes easily $2k a day, every day in his sample one lot account used for tracking the setups he teaches every day. Thomas n the morning focuses on equities along with futures and uses UniRenko but also is the master of reading traditional time charts.

Because of the inaccuracy issue addressed by this enhancement, I cannot "prove" with back test report that that these setups are profitable and back test manually which was necessary before I started taking the trades but is too time consuming to experiment with altering input variables for system enhancement.

For anyone struggling as have I these setups are the most profitable I've seen in many years and primarily based on UniRenko style charts with standard indicators. Traditional time charts are used with UniRenko for trade confirmation as the UniRenko settings tend to be short term.

So if you are interested in being able to validate the most profitable trading system across any type of chart which should be the objective and ideally already built into the product *************** Cast your Vote for SFT-3979************************

Thank you,

glen


[email protected]

[NinjaTrader_Jim]

Hello demacog,

Thanks for your post.

I thought it may be worthwhile to share some general perspective on backtesting.

Backtesting uses historical data which is comprised of snapshots of how the market moved. This does not contain tick by tick information, bid/ask or complete L1 or L2 recording of how the market moved.

Less data means less processing and faster backtests. The Strategy Analyzer is useful for analysis over extended periods of time to get an idea on how the strategy would perform, with the understanding that the data is a snapshot and cannot tell the full tale of how the market moved.

Since the data is just a snapshot, we only have OHLC values of the data series we submit orders to to estimate where the order has filled.

Additionally, we cannot process Calculate.OnPriceChange or Calculate.OnEachTick with historical data because there is no intrabar movement. (Tick Replay can allow intrabar Calculate.OnEachTick and Calculate.OnPriceChange logic, but this does not change how the orders are filled, which will still be filled according to the data series we submit orders to.)

A final discrepancy to note is that some bars like Renko bars cannot be built the same with historical processing because the bars can reverse and repaint the open of the bar. This can be observed with realtime but cannot be observed in a backtest since the bars are already built.

There will always be discrepancies between historical and realtime/live because they are fundamentally different. Historical processing has a trade off for speed, but if you want more accuracy, this would come with added data and added processing time.

SFT-3979 tracks interest in offering the same functionality that can be done with the Playback connection using Market Replay data but in the Strategy Analyzer. As you may be aware, Market Replay playback is a complete snapshot of L1 and L2 data and the amount of time needed to complete a test this granular is far longer than what is needed with historical data in a backtest.

More information on these discrepancies and how historical fill processing works can be found below.

https://ninjatrader.com/support/help...ime_vs_bac.htm

https://ninjatrader.com/support/help...ical_fill_.htm

We will be happy to track additional votes, and I hope this detail is informative for those wanting to understand why backtests and realtime/live trading results are different.

We look forward to assisting.

[afcmaff]

Add my vote for SFT-3979

[NinjaTrader_Jim]

Thanks for sharing interest afcmaff,

Your vote has been added.

[Mike0405]

it would be nice to have this option, so i vote yes for SFT-3979

[NinjaTrader_Jim]

Hello Mike0405,

Your vote has been added.

[kkelso]

+10 from me and my 9 other colleagues in our Skype trading circle

[hugoxalapa]

• I vote yes for SFT-3979

[NinjaTrader_Jim]

Hello kkelso and hugoxalapa,

I have added votes on your behalf.

kkelso, I do not have any means to verify the number of persons in your trading circle. Please have your colleagues post here, or send an email to platformsupport [at] ninjatrader [dot] com with the text "Upvote for SFT-3979" and include a link to this thread.

As concepts behind backtest accuracy are often overlooked or misunderstood, I am providing another link to my explanation post here: https://ninjatrader.com/support/foru...45#post1139645

[kkelso]

Thank you Jim. I will post this link to my group and forward to others I know who are also trying to develop auto and semi-auto strategies with the same frustrations of having no way to accurately rate or rank their programming acumen other than running them with live feed market data daily in a sim account. BTW, Hugo is one of our group.

[EB Worx]

+1 backtesting accuracy is more important than backtesting speed
Erwin

[NinjaTrader_Jim]

Hello Erwin,

I have added your vote.

I should be clear and point out that if you are processing a strategy that operates with Calculate set to OnBarClose, and you use High Order Fill Resolution with a single tick data series (or bake in a single tick data series and submit orders to that data series) your results will be very similar to what is seen in the Playback Connection with Market Replay data.

Testing a strategy with Market Replay data, and then reviewing a Trade Performance report or Realtime Strategy Performance report would give you the most accurate information since Market Replay is mimicking realtime data and playing it back exactly how it was recorded.

I mention this also because if we think about how we can improve backtesting to be more accurate and thorough, it means adding more data into the mix which means more processing time. This essentially means make backtesting more like Market Replay.

[mohawkTrader]

Please add my vote as well.

In my experience, the baking in a single tick series can be used to get exact (or very very close) trailing stops or exits if they are not based on the non-standard bars. However, even in this case, and using tick bars for execution, the entry fills appear incorrect. This is because, as mentioned, the non-standard bars (like Renko) have look ahead information. So the future information is leaking into the backtest and inflating results.

It appears that in the historical run, the bars are processed by the time stamps (so even when using the tick resolution) if the signal is coming from the non-standard bar then there is information coming from the future. Would there be any way to create a secondary time stamp for these non-standard bars which shows when they truly are "closed" (for on bar closed), and then that secondary time series used in the backtest. This way the proper tick bar would be selected post processing of the non-standard bar, and an accurate fill simulated. The framework then would not have to be re-written, just perhaps something added to those bar types to provide the secondary timestamp?

For example, let's say that currently the bar shows a closing time of 10. But really the bar is calculated based on information up to 13. We have tick series for 10, 11, 12, 13 or even finer resolution. If the secondary series could show a stamp of 13, then we'd know not to execute the entry against the 10 tick but against the 13. Therefore, we aren't seeing into the future (seeing 13 at time 10). Hope that makes sense.

Just throwing out an idea, but would like to see non-inflated results straight from the analyzer however it's implemented.

[NinjaTrader_Jim]

Hello mohawkTrader,

I have added your vote.

It would not be possible to do as you describe, and we should not backtest Renko bars (and other bars that remove the last bar and repaint the open, like LineBreak bars) because those bar formations cannot be accurately recreated with historical data. With historical data, we just have the bars that have closed, after they have reversed and repainted their open.

The easy answer would be to use the Market Replay with that Playback Connection, as this mimics realtime data, but for using the Strategy Analyzer, you may want to try using a custom Renko bar like UniRenko, which does not remove and repaint the last bar, and to then submit orders to the single tick data series to have accurate fill prices.

Unirenko (User App Share publicly available link) - https://ninjatraderecosystem.com/use...nko-bartype-8/

The NinjaTrader Ecosystem website is for educational and informational purposes only and should not be considered a solicitation to buy or sell a futures contract or make any other type of investment decision. The add-ons listed on this website are not to be considered a recommendation and it is the reader's responsibility to evaluate any product, service, or company. NinjaTrader Ecosystem LLC is not responsible for the accuracy or content of any product, service or company linked to on this website.

[AndreaBhs]

Hello Support,
please add my vote to this request SFT-3979

Also for me accuracy is more important than speed while backtesting an intraday strategy

thank you, best regards

Andrea BHS