Charting in NT 7 several times slower than NT 6.5

[bluelou]

Ninja's,
Is there a way to speed up how charts are rendered in NT7?

My charts are all based on custom indicators which call a C# DLL (my own, not a 3rd party DLL) and use tick data. I have all indicators set to MaxBarsBack = 256.

In NT 6.5 a 120 day chart (using 20 ticks/bar) would render in < 20-30 seconds max. In NT 7 this takes 3-5 minutes.

Why is NT7 so much slower and what can be done to speed this up? FWIW, I've also noticed that backtesting is no faster in NT7 vs NT6.5, and may actually be slower.

The only positive I'm seeing so far w/NT7 over NT6.5 is that the memory doesn't appear to leak and build up over time.

[NinjaTrader_Bertrand]

bluelou, are you checking with NT7's latest beta 15? Have found similar occurrences when using the default NT indicators to compare to latest 65?

For backtesting : what scenario are you exactly testing? Have compared speed across versions with for example our SampleMACross strategy?

What broker / datafeed are you using in your testing?

Have you noticed any errors in the log tab on runtime in NT7?

Is there a way to test your setup here on my end?

Thanks

[bluelou]

Bertrand,
Yes, I'm using NT 7.0.0.15

My issue is more chart rendering than backtesting:
FWIW, I'm used to the very slow backtesting in Ninja but the chart rendering is difficult to accept. Why should it take 3-5 minutes in NT7 to render a chart that only requires 30 seconds in NT6.5?

For backtesting:
I'm not sure what you mean by scenario. I'm testing AUDUSD using 20 tick bars (Gain Capital data) from 1/1/10 to 5/1/10. I haven't tested the default indicators. Whenever I've tested Ninja using time-based bars or simple indicators it has run pretty quickly. It's not really relevant to me.

Datafeed/broker:
I'm not connected to a broker or datafeed. That has always slowed down charting and backtesting in the past w/Ninja so I'm just working unconnected and pulling ticks out of the database.

Errors in Log files:
No, I haven't seen any errors in the log tab.

Replicating the test on your end:
I don't think it's possible without sending my DLLs. The only thing I can think of is using LogMeIn and watching it on my desktop.

[bluelou]

I think I've isolated the slow chart rendering issue. If I chart an indicator that doesn't call a DLL then I can chart 120 days worth of 20 tick bars in < 5 seconds. If I call the DLL for a single indicator then this takes about 10-15 seconds.

In my basic template I have 6 indicators and they call different functions within the same DLL. For some reason, calling the 6 indicators simultaneously has a compounding effect in slowing the chart rendering; that's when it takes the 3-5+ minutes to render a chart. I didn't have this problem in NT 6.5. Also, I've noticed that when NT 7 is struggling to render the chart I'm unable to use my web browser. Both NT7 and the web browser are just stuck there hanging for several minutes until the chart is rendered.

[NinjaTrader_JoshP]

bluelou,

Can you please provide a code snippet example of one of your calls? Thank you.

[bluelou]

Josh,
Here's the relevant code from one of my typical indicators. There are 2 calls to the C# DLL here.

As I mentioned previously, the chart rendering isn't too bad, < 10-20 seconds, when calling a single indicator of this type on 120 days of 20 tick bars in NT7. However, with NT 6.5 I could call a template of 6 indicators which are very similar to the code below and get the chart to render within 30-45 seconds. Now, with NT7, the same chart rendering takes 3-5 minutes. What is causing this 5-fold increase in rendering time?

protected override void OnBarUpdate()
{
double ret = Calc(CurrentBar);
Result.Set(ret);
Value.Set(ret);
}

public double Calc(int CurrentBar)
{
if(wvResults == null)
{
wvResults = new double[StandardLength + 1];
prices = new double[(int) Math.Pow(2, WvLevel)];
}

if(CurrentBar < prices.Length)
return 0;

for(int i = wvResults.Length - 1; i > 0; i--)
wvResults[i] = wvResults[i - 1];

wvResults[0] = Belle3.Algorithms3.Wv(Close, WvLevel, 3, Belle3.WvType.RH);
//1st call to C# DLL


if(CurrentBar >= prices.Length + wvResults.Length)
return Belle3.Algorithms3.Matcher(wvResults, 1, 1, N, M, StandardLength, constant).Two;
//2nd call to C# DLL

return 0;
}

[NinjaTrader_JoshP]

bluelou,

Is it possible for you to send me the DLL, the 6.5 version and the 7 version of your script calling the DLL? Thank you.

[bluelou]

Josh,
The indicators are exactly the same for both NT6.5 and NT7. All I did was copy the non-NinjaScript generated code from 6.5 to 7. Also, the only Ninja code I use are the event handlers, the rest is home grown. I'm not sure that I want to send out the DLL. What can you tell me given what you have so far w/o looking at the DLL?

[NinjaTrader_JoshP]

bluelou,

I was just checking if you were somehow referencing an indicator DLL which does not seem to be the case. If it was an indicator there could have been a potential of creating many instances of the indicator resulting in many many calculations.

Unfortunately at this point in time since neither looks to be the case we would not be able to analyze unless we can replicate it on our end. Perhaps you can simplify your DLL code significantly to some dummy logic and see if you can replicate that way.

[bluelou]

I'm not sure. Please explain. What is an indicator DLL versus a regular DLL?

[NinjaTrader_Bertrand]

bluelou, this was in reference to indicator assemblies vs external custom DLL's. If you can't provide the setup for us to reproduce here, please try removing the calls to the custom DLL's and then check how this affects your experienced performance.

[bluelou]

Maybe you can help by answering a different question.

I'm fairly certain that the multiple concurrent calls (6 indicators in 1 template) to the DLL are what is slowing down NT7 charting. The DLL code itself has been optimized. It can't run any faster. The multiple concurrent DLL calls were acceptably slow in NT 6.5 (< 45 seconds to render a chart) and were quite fast in OpenQuant (< 5-10 seconds).

This leads me to believe that there is something different about the way NT7 either manages tick bars or processes DLL calls.

So, my question is are there other known issues w/slow chart rendering in NT7 and/or are other users experiencing slower charting or strategy performance with custom DLLs in NT7?

--------------
Regarding data management: Let's say I use constant 20 tick bars for AUDUSD. My understanding is that I still can't create a 20-tick bar object for backtesting in NT7. Would NT7 run a backtest more quickly if I created a 20-tick bar data set outside of NT7 and then imported this into NT7? By doing so, the data set would be 1/20th of it's former size. Would this help?

[NinjaTrader_Bertrand]

bluelou, unfortunately I'm not aware of similar issues, so I could not correlate your case.

Concerning the data loading - NinjaTrader 7 would create a cached series internally already per interval request / session template / type.

You should see a significant performance gain on retesting on the same interval / type / session template then.