Discrepancies Between Backtest and Real Life Algorithmic Calculations

[Vispilio]

Hi,

We have an automated strategy built in house for our prop desk, and the original developer who built it as well as our own investigation has failed to address the following problem so I'm looking for a Ninja Trader expert who can solve the discrepancy:

We have a certain parameter set that alternates depending on the consecutive number of bearish or bullish days. This is calculated correctly during live conditions; however the backtest sometimes misreads the number of bullish and bearish days and applies different paremeters.

Similar problem also exists when we introduce a time delay to our order entries. Normally this should effect only the timing, however order entry levels are also affected during the backtest.

Anyone who can take a look at our strategy under an NDA and solve these issues asap would be hugely appreciated...


Warm Regards,
Vispilio

[NinjaTrader_ZacharyG]

Hello Vispilio,

Thank you for your inquiry.

I would suggest taking a look at the "Discrepancies: Real-Time vs Backtest" section of the NinjaTrader help guide for further information about why there are different results when backtesting a strategy and when running it real-time through a live brokerage account, live market simulation, Market Replay, etc: http://ninjatrader.com/support/helpG...ime_vs_bac.htm

Please, let us know if we may be of further assistance.

[Vispilio]

Hi Zachary,

We've already gone through that link and introduced custom modifications to make the backtest resemble reality as closely as possible. In fact, that's why a lot of developers are at a loss now of fixing problems like this because scripts get increasingly more complicated to account for inherent discrepancies of Ninja Trader backtest methodology.

If you have anyone who can take a look at our algo after signing NDA from your team, that would be hugely appreciated. We intend to use Ninja Trader platform as the backbone of our Fund Management enterprise; however the technical shortcomings of your backtest functionality make it very difficult to create precise simulations...

[NinjaTrader_ZacharyG]

Hello Vispilio,

We would be unable to sign a NDA.

Can you please provide a simplified strategy script that can reproduce what you are seeing so we may test on our end?

[Vispilio]

Hi,

ok I will try to forward you a simplified version, which e-mail address should I send it ?

Can we at least insure that only the developer sees it, I don't want to mail the strategy to general supports so an unlimited number of people has access to it ?

Basically the issue is we have a order trigger delay in minutes, yet we don't see this behavior exhibited correctly on the backtest, yet it sort of works in real life.

We haven't been able to identify so far whether this is a bug in the strategy or whether Ninja Trader backtest algo just can't seem to process correctly a trigger delay, even if it's expressed in minutes (to make sure the minute granularity is respected...)

Let me know..

Best,
Vispilio

[NinjaTrader_PatrickH]

Hello Vispilio,

Is the trigger delay based on a System.Windows.Forms.Timer?
If so, in backtesting the Timer is not going to work as it would in real-time.

The support team works as one, so a script that you send us can be reviewed by any member on the team. If you prefer not to send the file, that is understandable and respected.
So, let's see what other steps we can take here. Please answer the question on Timers above.