Backtest Inconsistency

[albazzaztariq]

I've found something very odd happening. When I increase the order quantity, my profit and loss changes greatly. For instance, whereas short trades had a sizeable profit, when I increase the size fivefold, the short trades now become net losers and the long trades are insanely profitable.

[NinjaTrader_Bertrand]

albazzaztariq, what instrument(s) are you testing and which slippage / commission would you apply to results? Would the outcomes change if you tested without any costs?

[albazzaztariq]

I'm using an ES Continuation (ES ##-##) with a $1.30 commission per lot. Why should outcomes change if I test without any commission? The magnified profits/losses should offset this.

Update: I just tried with 0 commissions and I have the same problem. For instance, overall there was a positive cumulative profit on both long and short trades, and a positive cumulative profits on All Trades. When I change from 1 lot to 5 lots, now the cumulative profit is twice as much, but negative instead of positive. The cumulative profit on the long trades is astronomical 713% and the short trades now have a negative cumulative profit.

[albazzaztariq]

I'm using an ES Continuation (ES ##-##) with a $1.30 commission per lot. Why should outcomes change if I test without any commission? The magnified profits/losses should offset this.

Update: I just tried with 0 commissions and I have the same problem. For instance, overall there was a positive cumulative profit on both long and short trades, and a positive cumulative profits on All Trades. When I change from 1 lot to 5 lots, now the cumulative profit is twice as much, but negative instead of positive. The cumulative profit on the long trades is astronomical 713% and the short trades now have a negative cumulative profit.

Something very strange is going on. I had a integer variable that held quantity that I threw as a parameter to ExitShort()/ExitLong() and EnterShort()/EnterLong(). After I took it out and backtested, everything was ok. What's odd is that when I put it back in, I was unable to replicate the problem.
Some very strange things are going on in NinjaTrader. This is people's money you're dealing with and if you guys don't have the manpower to keep this thing stable as well as develop and create new enhancements, you guys should consider adding more people to your company.
I'm going live with a strategy in ES futures on Tuesday/Wednesday and the volume could be magnified pretty quickly if my boss wants us to. I surely hope I don't keep running into these issues. I've tried to keep my strategies to calculate on the close of bars and checked over the trade records from backtesting as well as watched it with real-time market data. I've done this because I've read online others are having issues with trades exiting before entry and all sorts of weird stuff.
I really think NinjaTrader is a godsend in terms of far simplifying building and backtesting strategies. But if the results are not accurate, is it really more harm than help?

[NinjaTrader_Bertrand]

Thanks for the update and comments albazzaztariq - we sure are very heavily working with a dedicated team on enhancements to NinjaTrader and our next major platform update. Any issue we can get to reproduce is being closely reviewed by our development and product management team, so if you run into it for any reason again please don't hesitate to get in touch with us so we could work with you on a reproducible example. I've asked about commissions as in issue cases it's always to get more simpler in terms of variables in the setup and isolate out what's contributing. For the cumulative profit in % terms, generally it's important to keep in mind that results seen would be represented with reinvesting.