Nt8 lagging

[NinjaTrader_Brett]

jphiggs, an1952, mbesha,

I'd like to look more closely into your cases, as you have time to work through this can you please reach out to me to platformsupport at ninjatrader dot com with ATTN: Brett and reference this forum post so that I could work with you to better understand what is going on and take next steps to isolate the issues and get them resolved.

With NinjaTrader 7 and NinjaTrader 8 we have a completely unrestricted development environment for third party scripts. This is incredibly powerful as to what can be created has no limit, however comes at the cost of allowing any code to run in the NInjaTrader runtime. All it takes is one line of code to cause a negative performance impact which would cause lagging. As NinjaTrader must finish processing before it can move on to the next tick.

1. Isolate Third Party Scripts one by one to see if the problem persists. I'll suggest a simple scientific approach to take.
2. Typically we will find the offender in that process and then I will work with you to contact the vendor to report the issue and work towards resolution.
3. If in fact we isolate all third party scripts and there is still an issue and the computer specs meet our recommended specifications then this becomes a high priority issue for the development team to understand. Which is why I want to work with you to understand these cases.

Thank You.

[brucerobinson]

Progress & feedback with 8

Over the last couple of weeks I've been trying to migrate from 7 to 8 & experienced several of the issues reported here.

I thought it may be helpful (encouraging) to report some progress & my 'journey'.

My comments are subjective & not comparing like for like, & Support may disagree, and have good reason.

I have a 7 workspace with 6 Futures, each 2 Sons, approx 24 charts from day to 15 tick, & over 300 indicators total. Many are simple to do things 7 can't which are now part of 8 without indicators. It took many hundreds of hours to refine, but runs flawlessly.

My intention starting out migrating was to start recreating my 7 workspace simplified, bare minimum using only Ninja's indicators.

1st issue - lagging cross hairs (local). PatrickH fixed by adding NT to graphics card driver. Given that 8 can & really needs to use the card, and most PCs of 8 spec will have one it should be more visible, even a 'splash screen' during install.

2nd issue - I used the 'migrate from 7'. I believe this contributed to problems. My 7 workspace is big (nearly 3gb). Long & short - I had a plethora of 'unexplained' errors, freezes, crashes, log errors, historical data server errors, etc, etc. Uninstall, reinstall, delete Documents ninja folder you name it. Almost as last resort, fresh clean install, nothing imported from 7, since - no crashes, freezes, unknown errors. Others have reported similar importing from 7. My 7 runs flawlessly. Import to 8, potential for problems to build upon. It's my opinion - but my conclusion is, after fixing using graphics driver, and building 8 from a clean install, no problems. I did look at .NET as next step, but wasn't required.

Again my opinion, but one thing neither 7 or 8 cope well with is making changes to settings etc 'on-the-fly' especially doing so in periods of high activity (particularly with complex setups). Unfortunately, building a new workspace is nothing but this activity.

My 8 setup is simpler than my 7 as yet - fastest chart is 1 Min, & no tick charts, but I do have 10 instruments & 30 data series running, several intervals, 6 Doms, only 6 indicators per chart only Ninja ones.

I am in Europe & trade Dax from its open & others until NY Close. I have used 7 until NY Open, then switched to 8. The Overnight Session was highly volatile and fast on all markets. My 7 setup ran smoothly. I switched to 8 at the Open & it coped with everything the market threw at it live but in SIM (I don't harbour a death wish), which, today is probably the most it is likely to. It seemed more responsive than 7, a little more 'immediacy', responsive.

Despite 'working it quite hard' changing intervals, dragging charts around, etc, not a single error in the log with over 100 multi-lot ATM strategy & manually added trades in blistering conditions. Ram & CPU never under stress & my trading station is mid-range (4 core AMD 3.8 GHz, 16 Gb, 2Gb graphics)

Charts comparisons & hence overall are not like-for-like because 1-Min way slower than 15-Tick. I've tried 10s and have seen some cross hair & chart lag (I believe this may be due to refresh limit constraint), & some other things to investigate.

In a few days, I may cautiously add Ninja's Price Line, one chart at a time....
Hope this is helpful to some
Kind regards

[imalil]

I'm having the same issues as everyone here. The lag was untradeable today-15 20 point difference at one time. I also have CPU and RAM issues when the market gets volatile. I'm using my own developed indicator which uses level 2 data, checks max/min about 10 bars back of indicator-calculated data. I assume this contributes to my CPU problem but I don't know how it contributes to lag.

.NET version is 4.7.1 and I have no idea how to change it.

My question is to jphiggs, an1952, mbesha, silver99:

Do you have custom made or 3rd party indicators running on NT8, get these performance issues, and also have the SAME or similar indicators running on a different platform with no performance issues?

[NinjaTrader_Brett]

Thanks for the posts. A little detail for those of you interested.

In principle there are two separate events which must ultimately take place for you to see market data on your screen:

1) We must process the data. (E.g. calculating that new SMA indicator value)
2) We must render the data. (E.g. drawing that new SMA line value)

NinjaTrader 8 is multi-threaded and we create separate threads for these task. Simply spoken we create threads for processing data (step 1) and separate threads for rendering data (step 2). This allows us to distribute the load across modern CPU's. We reference these internally by the names of 'instrument' thread and 'UI' thread, where on a quad core computer you will have 4 UI threads and 4 instrument threads. We will randomly assign windows to a UI thread for rendering and randomly assign instruments to an instrument thread for processing.

In the case where market data being delayed and the chart is still responsive this is indicative that there is a problem in step 1 referenced above, were a single calculation is taking way too long and the instrument thread falls behind. It must finish that calculation before it moves on to the next tick.

In the case where you cannot manipulate the chart or the chart says its not responding, this is indicative of a problem in step 2.

I just got off a phone call from one customer who had a third party indicator added to his chart and that chart was delayed but still responsive. As soon as I removed the third party indicator the chart instantly caught up and was responsive again. We will run the setup for a day or two to make sure we have pinpointed the exact third party responsible and then I will reach out to the third party to check on what could be done.

In the next few weeks I'll be collecting more data from customers and isolating more cases. I'll also be looking at putting some more details on how exactly data is processed in NinjaTrader which might be helpful for those of you trying to troubleshoot these issues or those of you developers trying to resolve an issue of this type that has been isolated to your code.

For those of you that have not reached out to me with ATTN: Brett in the subject line and reference this post please do so. As I will want to perform similar steps on your install to help isolate the issue. In the case above the issue was related to a third party however your case may be different so we need to work through each case individually and separately.

[imalil]

I always thought this might be the case. And I believe if I remove the indicator I made, things will improve. But I also know the results my indicator calculates are necessary for me to compete in the market. So, quick question for Brett:

My developed indicator uses level 2 data, multiple series, checks max/min of indicator-calculated data X bars back. It also does other things which are no big deal.

Are calculations like this always CPU taxing and should not be done, or is my indicator most likely poorly written?

Thank you.

[NinjaTrader_Brett]

Basic calculations and math operations are always very fast for the computer. I would not expect to see any issues with this. Where large performance decrease can comes in are things like looping over large number of data points on each tick or expensive collection allocations like calling .Add() on a list when the list is out pre-allocated memory, etc. Where sometimes we have to implement an alternative more performance oriented method to achieve the goal.

We do in our help guide have a section dealing with some best practices on performance here: https://ninjatrader.com/support/help..._practices.htm

Can you please take a look to see if it will help, if your still having issues and relatively small/simple as you mention and your not sensitive to sharing it I could offer to take a brief look to offer some feedback. You could email it to me privately if you wished referenced this forum post to platformsupport at ninjatrader dot com ATTN: Brett.

[silver99]

Chris,

I feel your pain. After 2 days of NT8 running smoothly, mine was lagging again today. Very few charts open, very few indicators. Frustrated and don't know what to do about NT8 at this point. I take back my .NET framework 4.7 post - that wasn't it......

[NinjaTrader_Brett]

Silver99,

I'm available tomorrow morning to review your setup, so I could look into whats going on. Please reach out to me at platformsupport at ninjatrader dot com if you would like to do so and reference this forum post, ATTN: Brett in subject line and a phone number for me to call on.

Thank You.

[silver99]

Brett,

Thank you for following up with me in trying to resolve my chart lagging issue. I have been looking at the latency in my internet connection and found out that it is extremely high, >1000ms, a majority of the time. I am disconnecting all devices except my trading computer to see if this resolves my chart lagging issue. Thank you for your followup and determination to get my machine running properly. Same appreciation to Patrick for his support help, both of you are professionals.

[RonnyKeller]

Trading the ES, my trading chart (5-minutes) is usually set to display horizontal lines for each point (Chart Properties / Horizontal grid lines = Fixed, interval type = points, interval value = 1). With the huge swings over the last days I happened to notice huge lagging, especially in cursor movement (no matter whether using cross hair or simple mouse pointer). Today's chart shows a range of 186 points on my setup, thus printing 186 horizontal lines, yes, verrry narrow ones).

Once I changed my chart property to interval value = 10 and therefore decreasing the horizontal grid to less than 20, all lagging was gone immediately. Happened to see a similar behavior on my 60-minutes chart where horizontal lines are set to "automatic" - changing to fixed bigger distance / less lines got rid of lagging there as well.

Perhaps you may want to try on your charts whether actively reducing the amount of NT's horizontal grid lines might help for you as well. Since "automatic" is the default setting, and since NT seems to print more grid lines than it can handle, you may face lot of lagging simply due to this setting, just a guess. Hope it helps.

[NinjaTrader_Brett]

Thanks for posting your experience. You may in fact have been hitting an actual rendering "wall" there. This primarily is a function of the pure rendering speed of the video card. What video card do you have currently in that setup? Additionally, were these dashed lines by chance? I do know that dashed lines are very much more challenging to render the just solid lines for the video card.

[RonnyKeller]

Not dashed but dotted horizontal grid lines, width=1, so Yes, some rendering work to process.

My PC runs an i7-6700K CPU @ 4.00 GHz, with 16 GBs RAM, and my graphics card is a NVIDIA GeForce GTX 1070, data fed through a 1 Gbit fibre connection - this setup is way more than needed.

Just wanted to mention my experience to help others; seeing that the "automatic" setup draws too many lines to handle was kind of a surprise and I guess, many users have many charts with that setting active - so it might provide a little tweak there.

For me it's only a minor issue at the moment since the 1-point spacing didn't make sense anyway over the last days with those exceptionally big swings.

[imalil]

I use tick charts, smallest around 500. I don't draw hor or vert lines at all. I have an indicator that draws two vert lines per day which I know is not the issue. Based on what I'm reading, it simply sounds like NT8 cannot handle the kind of volatility we've been experiencing. There are too many of us with the exact same symptoms. I'll work on my workspace, but most of the people here have much more powerful computers than me and we're all having the same problems, so I don't have much hope.

[brucerobinson]

Thanks RonnyKeller re: (grid) lines. To confirm a) the same experience, b) consequently I have zero vertical or horizontal grid lines of any sort on any of my charts - Current & Prior OHL & Session breaks, that's it, nothing else, nada c) I have a way over-spec video card also - 2Gb GTX960, that could, I believe if I were so inclined which I'm not, render whatever mega texture level 3D whatever-it's-all-about superfine granular badass game feature-rich enhancement pack with cyborgs attacking me at speed of light and still keep up, and my internet latency is 14ms

To be clear, I have 3x NT OHL indicators, 1x NT Prior Day OHL indicator, 1 x NT VOL indicator, 1 NT PricelineWH indicator (only added yesterday - it is the only one set to Update every tick - don't believe it has affected performance).
NO other indicators, NO third-party indicators. None. Nada. Zip

Just for info.

Kind regards
Bruce

[NinjaTrader_Ray]

Thanks RonnyKeller re: (grid) lines. To confirm a) the same experience, b) consequently I have zero vertical or horizontal grid lines of any sort on any of my charts - Current & Prior OHL & Session breaks, that's it, nothing else, nada c) I have a way over-spec video card also - 2Gb GTX960, that could, I believe if I were so inclined which I'm not, render whatever mega texture level 3D whatever-it's-all-about superfine granular badass game feature-rich enhancement pack with cyborgs attacking me at speed of light and still keep up, and my internet latency is 14ms

To be clear, I have 3x NT OHL indicators, 1x NT Prior Day OHL indicator, 1 x NT VOL indicator, 1 NT PricelineWH indicator (only added yesterday - it is the only one set to Update every tick - don't believe it has affected performance).
NO other indicators, NO third-party indicators. None. Nada. Zip

Just for info.

Kind regards
Bruce

Thanks Bruce.

Can you tell me if the CurrentDayOHL indicator parameter "Plot current value only" is checked true?