Suggestion for improving QA suite

[crokusek]

Hi,

I have a general suggestion for improving the NT8 QA suite.

To help locate possible multi-threading issues, please consider placing artificial random delays throughout the code, especially within the indicators.

For example, a case was sent earlier this week to Shawn where the simple SMA indicator was modified by adding a small delay. When alerts are created on this indicator it causes some error and the chart stops functioning and the tool must be restarted to clear it. It only happens if that delay is there.

protected override void OnBarUpdate()
{
// Add a delay to simulate performing a long calculation like the real case does.
// The time can be adjusted but too small and it doesn't show the bug.
//
Thread.Sleep(15); // or better to use a new Random value each iteration.
//...
}

I offer this suggestion because currently there are few dedicated software tools available to thoroughly exercise/test multi-threaded C# code.

Its like in the days of writing C or C++ before the memory checking tools like "Purify" came along. There used to be tons of memory leaks.

Thanks,
Chris


The related issue:

http://ninjatrader.com/support/forum/showthread.php?t=91352

[NinjaTrader_Brett]

Thanks for the feedback. Its a little chicken and the egg scenario going on as well since we can add tests to make sure we don't have regressions but to randomly try to test in the automated test suite for faulty conditions of this type is very difficult, since you don't know what you don't know. Tests work well for scenarios which have a clear input and a clear output and that is where we will typically would invest effort on developing such tests.

-Brett

[crokusek]

Hi Brett,

It sounds like I can't sway you but I should reply anyway...

The motivation here is that there is evidence of lingering race conditions in a couple different areas (shapes / alerts /chart panel) and it takes lots of effort to isolate, package and report them. So much more efficient to communicate just the true cause--thread timing!

Adding in a random delay to some of the existing indicators/strategies/etc is like installing 100's of custom indicators from the field--all which all operate at different compute/draw times. And the delay could be turned off or on or modulated using some global test setting. This single line of delay code could be added to existing tests requiring almost no investment.

Okay I'm over my 2 cents!
Chris