Same code different results

[JohnSteinberg]

Hi,

I upgraded to the latest version today (8.0.11.0) and ran an algo I have been testing with a trading buddy. To my surprise, we got totally (by many orders of magnitude) different results. We both removed the code and re imported, cleaned our cache etc. but the problem persists.

Please see attached screenshots of the versions and different results.

Could you please assist with an answer as to why this might be happening?

Thanks,
John

[NinjaTrader_Jim]

Hello John,

Thanks for writing in.

As far as we know, this issue surrounds diverging results between 8.0.10.0 and 8.0.11.0. We should confirm that the behavior is not due to strategy logic as a first step.

Do you see different results when testing the SampleMACrossOver strategy between NinjaTrader 8.0.10.0 and NinjaTrader 8.0.11.0 with the same settings?

There may be a lot of variables to narrow down which will show difference in strategy results. If there is an issue with the NinjaTrader platform, it should be apparent when testing the SampleMACrossover strategy.

Are you able to narrow down a simplified version of the strategy that only demonstrates the difference in results between either versions of the platform? We could use such a barebones example to report the issue further, or at least provide further input.

I look forward to being of further assistance.

[JohnSteinberg]

Hi Jim,

Thanks for your speedy reply. We will run the MA strategy on both and post the results later today.

Thanks,
John

[sledge]

You could always downgrade to NT 8.0.10.0.

You might have to do some cleanup (rename database file). Of course easiest way would be to rename your My Documents/NT folder and rebuild nt 8.0.10.0... Then see what happens.

I ran a Market Replay Test and it was spot on with my new data.

[JohnSteinberg]

Thanks @sledge, I will do this as a last resort. I am not in a hurry, if we can find a problem using the process and it can help make the platform or community better I would be happy

[JohnSteinberg]

Hi Jim,

We ran the sample ma crossover on both version too, please see results attached. Same problem it seems.

EDIT: We ran the sample MA crossover on a different instrument too (SI) just to try and eliminate data discrepancies, same problem (please see attached).

Thanks,
John

[NinjaTrader_Jim]

Hello John,

Thanks for sharing the results of that test.

I have taken the same test by installing R10 cleanly, running the backtest, installing R11, deleting historical data, and then running the second backtest. My results are lining up. This tells us that the issue is due to underlying data between both versions of the platform on your end.

For the platforms that you are comparing, I would suggest to delete the historical data used for the backtest so it is downloaded anew. You can do this from the Histoical Data window which can be accessed from the Control Center under Tools > Historical Data. Right click the instrument and expiries used to drive the backtest and select Delete.

At this point you can reopen the Strategy Analyzer and run your back test. Performing these steps on both platforms and downloading the data from the same provider will give you the same data to produce the same results. You may wish to connect with the same account at different times to rule out issues with the provider.

After taking these steps, I would suggest to verify the OHLC values of the data series by opening a chart within the Strategy Analyzer and using the middle mouse button to open a mini data box that will tell you these values.

Please let me know if I can be of further help.

[JohnSteinberg]

Thanks for your help Jim. We will give that a shot