NT8 possible bug backtesting

[Steven$B]

Hi,

I bumped into a problem using NT8 for backtesting historical data, for comparison I also include code and results for NT7 since NT7 results are as expected.
Using this simple strategy in NT8:
protected override void OnBarUpdate()
{
EnterLongStopMarket(Close[0] + 1);
EnterShortStopMarket(Close[0] - 1);
}
Using the above NT8 system for a backtest, trades are generates as shown in the attached file NT8_trades. We are always in the market, and the following should always be true:
Exit price of trade N == Entry price of trade N+1
However, this is not always the case, see trade 5 and 6 in file NT8_trades (also 13-14, etc).
On trade 6, instead of using the opening price of 350.60, the price 350.75 is used (350.75 is Close[0] - 1), I think this might be a bug in NT8.
For comparison the same system in NT7:
protected override void OnBarUpdate()
{
EnterLongStop(Close[0] + 1);
EnterShortStop(Close[0] - 1);
}
Using the above NT7 system for a backtest, trades are generates as shown in the attached file NT7_trades, all trades are as expected.
Backtest settings can be viewed when opening NT8_trades.
If you want I can provide the historical data used, to reproduce this problem.

Thanks, Steven

[NinjaTrader_Jesse]

Hello Steven,

Thank you for the post.

I wanted to check, are you able to see this using any instrument or only on this specific dataset?

If you are only able to see this on this one specific instrument this could be data related, If you have not yet tested other instruments in a controlled test I would suggest that we complete that test next to confirm this.

Would you be able to run this test again on an instrument we can both view and get data for? I had tested this using the ES 03-17 myself for a period of 1 day and can see equal results on both platforms, additionally I see that this scenario can also happen in NT7 based on that test. This leads me to believe this could potentially be related to the data being used or the data included in the tests.

To test this, could you use a standard instrument like the ES or AAPL and run the test for 1 day or a length of time it takes for you to see the difference in prices? After running this test could you provide an image of both the NT7 and NT8 windows as you did previously?

In my test I can see both NT7 and NT8 reporting the small difference between entry and exit per your observation in certain cases.

I look forward to being of further assistance.

[Steven$B]

Hello Jesse,

thank you for the reply, but I don't think that you are taking the right approach.
First, the problem is not that there is a difference in backtest output between NT7 and NT8, the problem is that NT8 gives (very) wrong backtest results for the used dataset; NT7 is only used to indicate that this is a NT8 related problem.
Second, when using a valid dataset this problem is never data-related but always software related; a NT8 backtest should give correct output not matter what (valid) dataset is used.
The point of checking if software is correct is not feeding it with an al sorts of datasets that give correct output. If you can find one valid dataset that causes your software to generate wrong output, you have proof that there is a problem with the software.

So, instead of asking me to perform additional tests I suggest you to do the following:
- provide me with an email address so I can send you the used data
- perform a backtest using this data to reproduce the error in your own environment
- if you can reproduce the problem using this data contact your development team

If you are unable to reproduce this problem I will be glad to be of further assistance in any kind.

In short, send me an email address and I will send you the used data.
Please keep in mind that in this case you are not helping me, but I am taking the time and energy to point you to a serious problem in the NT8 software.

If I can be of any further help please let me know.

Steven

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

Please feel free to email this item to our platform support [at] ninjatrader.com for further assistance.

Please let me know if I may be of further assistance.