Backtesting with Renko bars

[CMillz]

Serious question, are there any NinjaTrader users that make money?

The backtests are worthless. The walk forward method is so clunky I can't do anything with it.

The program literally crashes on me 3 to 4 times a day and each time I lose about an hour of work.

Other than vendors selling indicators, can traders actually make money using this pile of crap program?

[CMillz]

Oh what do you know, before anyone could even have a chance to respond the program crashed again.... bravo

[NinjaTrader_PatrickG]

My name is Patrick and I am the Platform Support Team Lead.

I see from your post history that it seems like you're using custom strategies or at least a strategy which may use a custom add-on. Please clarify - do these issues occur if you use the built-in 'Sample MA Crossover' to run an identical backtest?

[CMillz]

Patrick,

I'm a little frustrated this morning because of losing basically an hour worth of work because NT crashed on me twice.

Any backtest I run in Ninja has different live results. Particularly with Renko bar types. I've added slippage (which still doesn't make sense to me how you guys calculate that) and tried to code a worse price fill in, and still the tests show positive results but the actual results are a loser.

I'm at the point now where I don't think any backtest that is run in NinjaTrader can be trusted. My question then is, are there any methods which produce something somewhat similar to a live result. And can Renko-based aggregations provide any type of testing that is somewhat similar to live results?

[NinjaTrader_PatrickG]

Renko bars re-draw in real-time. There are no means to simulated this accurately.

This means that Renko bars cannot be accurately backtested due to their fundamental nature. This is not a NinjaTrader limitation and I would expect any platform would have difficulty matching live behavior while backtesting with a Renko bar type.

[nepenthean]

I understand your frustration insomuch as I just posted about 8.0.9.0's instability with NO indicators other than a few MA's. I trade live using NT7, much slower but has not crashed since i can remember.

[JerryA]

I agree. NT8 crashes all the time, loses workspaces and charts. The "preset" to default settings on indicators doesn't work well. I also use NT7 for live trading. Unfortunately, if I want to use Jigsaw DOM replay, I have to use NT8. It's more trouble than it's worth.

[nepenthean]

what instrument(s) do you trade jerry?

I am trying out NT8 playback but when I try to bid on jigsaw's DOM nothing happens. I am not able to execute trades at all. NT7 is not open.

Thanks mate.

[JerryA]

what instrument(s) do you trade jerry?

I am trying out NT8 playback but when I try to bid on jigsaw's DOM nothing happens. I am not able to execute trades at all. NT7 is not open.

Thanks mate.

I trade the ZB and 6J. I don't trade off the Jigsaw DOM - I use the NT DOM because I like to use ATM strategies.
You might ask Pete on the Jigsaw thread at futures.io - here's the link: https://futures.io/trading-reviews-v...-anything.html

[tmfdouglas]

I gave up on nt8 a while ago, I occasionally test it again to see if bugs have been resolved, but generally just find new ones.

Most recent bug I have found is automated strategies cancelling stop losses immediately when there is no reason for the strategy to do so. Ninjatrader have identified the cause although how that got through any form of internal testing, which I’m not confident exists, amazes me.

Very much seems that paying customers are used as testers to save ninjatrader money. In any scenario that is unacceptable, but when real trading capital is at stake it’s an entirely different story.

Also if NinjaTrader support are using the number of bug reports or forum posts to gauge how the platform is doing this is by no means accurate, I know many NinjaTrader users who have stopped bothering to either report bugs or post on the forums as so little progress is being made.

[sledge]

Patrick,

I'm a little frustrated this morning because of losing basically an hour worth of work because NT crashed on me twice.

Any backtest I run in Ninja has different live results. Particularly with Renko bar types. I've added slippage (which still doesn't make sense to me how you guys calculate that) and tried to code a worse price fill in, and still the tests show positive results but the actual results are a loser.

I'm at the point now where I don't think any backtest that is run in NinjaTrader can be trusted. My question then is, are there any methods which produce something somewhat similar to a live result. And can Renko-based aggregations provide any type of testing that is somewhat similar to live results?

NT_PG is right. Renko Bars have been discussed to almost death.

https://ninjatrader.com/support/forum/showpost.php?p=392116&postcount=31

[FMtrader]

I gave up on nt8 a while ago, I occasionally test it again to see if bugs have been resolved, but generally just find new ones.

Most recent bug I have found is automated strategies cancelling stop losses immediately when there is no reason for the strategy to do so. Ninjatrader have identified the cause although how that got through any form of internal testing, which I’m not confident exists, amazes me.

Very much seems that paying customers are used as testers to save ninjatrader money. In any scenario that is unacceptable, but when real trading capital is at stake it’s an entirely different story.

Also if NinjaTrader support are using the number of bug reports or forum posts to gauge how the platform is doing this is by no means accurate, I know many NinjaTrader users who have stopped bothering to either report bugs or post on the forums as so little progress is being made.

Have to agree with the above. After working through every single beta and subsequent releases I have given up! I am simply no longer prepared to spend 100s of hours of my time in trying to get to the bottom of all the issues I have experienced. I simply check the forum every now and then and when things finally appear to be sorted I might return to using NinjaTrader. Combine this with the minimal brokerage options for trading FOREX in Australia and no options for trading markets such as the Nikkei emini means NT can no longer be the platform of choice for me.

[NinjaTrader_PatrickG]

Thanks for the feedback.

We release a new version of NinjaTrader 8 in monthly intervals typically. Each of these updates contains dozens of improvements and changes.

Although the NinjaTrader 8 userbase continues to grow, the bug reports have continued to decrease.

We are ruthlessly hunting any remaining bugs. Therefore, if you run into any unexpected behavior it is paramount that you contact us so we may investigate. The best way to do this would be to write into PlatformSupport[AT]NinjaTrader[DOT]com so we can get specific information from you.

The original issue reported here was half expected (Renko playback accuracy), but the crashing would not be. CMillz, Please clarify - do any crashes occur when using the 'Sample MA Crossover'? At this point it would be best for you to write in so we can obtain your diagnostic files to learn more about your NinjaTrader.

[tmfdouglas]

Patrick, thanks for the generic response which doesn’t directly address, or even acknowledge, the points raised on the most recent replies to this topic.

I see no point in continuing this discussion, similar to NinjaTrader 8, it’s going round in circles and is doomed to fail.

[LangdonJohn]

Yes, I tried it, but I wasn't impressed because I don't know much about it and I'm used to working physically