Multi-Instrument Market Analyzer Column hangs Market Analyzer

[Matheusfx]

Hi, I'm facing a hanging issue with a multi-instrument Market Analyzer column, which I believe comes from a deadlock. I've attached a Market Analyzer template and the code for the custom column below.

FYI, for some weird reason, the hang doesn't always happen, and I've found that in some computers it doesn't hang most of the time, and in others it hangs almost always.

The code simply adds 28 forex instruments, and calculates an EMA when the column reaches State.Transition. I've also setup multiple of these columns on a Market Analyzer and added the 28 instruments it uses as rows. After doing this, Market Analyzer gets stuck in a "Loading..." or "Calculating..." state forever. None of the columns update anymore, and even some charts get affected. Even closing NT8 gets affected, it stays open in the background forever, and needs to be force closed with the Task Manager.

I'm using the latest version of NT8 and the "My NinjaTrader" data provider.

I'd really appreciate some help on finding a workaround or getting this issue fixed!

[NinjaTrader_ChrisL]

Hello Matheusfx thanks for your post.

This script works properly on my installation with a Kinetick connection. There is likely an issue with data loading on your end. Please reduce the script to narrow down the data loading issue, loading one instrument at a time. Once you have confirmed that one instrument is loading, add the next one. Once you have found the instrument where the script stops loading, try loading that onto a blank chart to see if your data feed is providing that instrument.

I look forward to assisting.

[Matheusfx]

The data provider is providing all of the instruments, everything loads and updates as expected without the multi-instrument column. I suspect the underlying issue is related to this one: https://ninjatrader.com/support/foru...angs-the-chart

Also, switching to a FXCM Demo connection causes the same issue.

[NinjaTrader_ChrisL]

Hello Matheusfx, thanks for your reply.

Have you tried reducing the amount of data series being added? What if you try N-1, N-2, N-3 added series? Is there any point where it does load? If you do find a certain point where the multiple series will load and when you add another series, it does not load, I will be happy to test the same thing on a test FXCM connection.

Best regards.

[Matheusfx]

I've narrowed down the problem, the reduced examples are attached in this post.

The hang starts happening after I add the EURGBP data series to the column, removing it seems to make it stop hanging the application. I also couldn't reproduce it without the AUDCAD line inserted into the Market Analyzer template. I'm not sure if it's due to these currency pairs specifically, or the number of instruments added, but there is definitely something going wrong internally inside NT8. Another thing that seems to be important to cause this is calling an Indicator. If I remove the EMA call that's in State.Transition, I can't reproduce the issue, even with the 28 currency pairs inserted.

A tip to reproduce this without relying too much on luck, is repeatedly pressing F5 to reload the scripts in Market Analyzer, as fast as you can. It hangs after a few reloads on my machine.

I'd really appreciate some help on this issue, this has been plaguing my workflow for months.

[NinjaTrader_ChrisL]

Hello Matheusfx,

On the line calling the EMA I was getting a bars ago error because all internal price series indexes are -1. Please check to see if you are getting any errors in the Log tab of the Control Center as well. I changed the use of the Closes array to BarsArray to target the bars series:

CurrentValue = EMA(BarsArray[BarsInProgress], 20)[0];

Please let me know if I can assist any further.

[Matheusfx]

Hello Matheusfx,

On the line calling the EMA I was getting a bars ago error because all internal price series indexes are -1. Please check to see if you are getting any errors in the Log tab of the Control Center as well. I changed the use of the Closes array to BarsArray to target the bars series:

CurrentValue = EMA(BarsArray[BarsInProgress], 20)[0];

Please let me know if I can assist any further.

I did your suggested change and my Market Analyzer still hangs after a few reloads. I'm not even sure how it would have any internal price series indexes at -1 during State.Transition, as it's only supposed to be called after processing all historical data.

During my last test run I've got a few errors on my Log, but I'm not sure if they happened right before it hanged, or if they were already there, but here they are:

Error in realtime market data handling: System.InvalidOperationException: Collection was modified; enumeration operation may not execute. at System.ThrowHelper.ThrowInvalidOperationException( ExceptionResource resource) at System.Collections.Generic.List`1.Enumerator.MoveN extRare() at NinjaTrader.NinjaScript.NinjaScriptBase.SetState(S tate state) at NinjaTrader.Gui.MarketAnalyzer.MarketAnalyzerGrid. HandleBarsBackNow(PendingBarsBackState barsBackState) at NinjaTrader.Gui.MarketAnalyzer.MarketAnalyzerGrid. OnRealTimeDataTimerTick(Object sender, EventArgs e) at NinjaTrader.Cbi.Instrument.<>c__DisplayClass155_0. <OnRealtimeDataTimerElapsed>b__1(SubscribedThrea d[] threads, Int32 i, RealtimeEvents evts)

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load EURNZD Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load AUDJPY Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load AUDCAD Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load USDJPY Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load NZDUSD Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load USDCHF Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load USDCAD Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load EURJPY Default: 5 Minute

'MAHangTest' tried to load additional data. All data must first be loaded by the hosting NinjaScript in its configure state. Attempted to load EURUSD Default: 5 Minute

The "tried to load additional data" errors don't make any sense to me, as the AddDataSeries methods are all clearly defined inside the State.Configure state.
As for the "error in realtime market data handling", it's an internal NT8 error which I couldn't debug because NT8's code is obfuscated.

This doesn't seem to be expected behavior, especially because it happens even without the fast reloads, randomly. I'd be really grateful if you could forward this to the development team to see if they're able to find the underlying issue.

[NinjaTrader_ChrisL]

Hello Matheusfx, thanks for the follow up.

I'm sorry but I am not able to reproduce the issue. What is the main purpose of the custom column? What is it supposed to do? Why is it loading all these series just to input a single EMA value from one into CurrentValue at loading time? I will not be able to report anything as a bug unless we have completely ruled out the possibility of custom code causing the problem.

I look forward to hearing from you.

[Matheusfx]

Hello Matheusfx, thanks for the follow up.

I'm sorry but I am not able to reproduce the issue. What is the main purpose of the custom column? What is it supposed to do? Why is it loading all these series just to input a single EMA value from one into CurrentValue at loading time? I will not be able to report anything as a bug unless we have completely ruled out the possibility of custom code causing the problem.

I look forward to hearing from you.

This is just the minimum example I could come up with which causes the hanging issue. In reality, I have a much more complex indicator that needs data from all of these instruments, and the purpose of this column is to display the data that comes from the indicator. It works flawlessly on the chart, but calling it from the Market Analyzer column, just like this example that's calling the EMA, makes it completely hang and generates all the issues I've stated previously.

If you're still not able to reproduce the issue, I recommend trying the first example I sent, which asks for even more data, on another computer. It seems to happen more frequently on some computers than others, and I think it might be related to the number of threads. Even though it's more frequent in some computers, I'm pretty sure it happens on any computer, because I've had multiple people use the original complex indicator, and all of them had this exact issue.

If even then, you still can't reproduce it, I'll record a video of it happening.

[NinjaTrader_ChrisL]

Hello Matheusfx,

Testing the original script on a FXCM account. I still can not replicate what is being reported.

Please test out your script on a new clean development environment:

To create a clean installation to export your NinjaScript objects from please follow the instructions below:

1. Go to Help > License Key and copy and paste your License Key into a text document
2. Export the code you want as the final export as source code
3. Exit the platform
4. Go to (My) Documents > rename the 'NinjaTrader 8' folder to 'NinjaTrader 8.Prev'.
5. Then re-install from the following link: https://ninjatrader.com/GetStarted

When you reinstall it will recreate the user folder you had just renamed with only stock files.

Now you can start the platform and import the source code for the item you want to export as a compiled assembly. Because there are no other items in this installation, only the item you want to export would be included.

To switch between the two installs; shut down NinjaTrader > rename the new installation to 'NinjaTrader 8.Dev' under (My) Documents > then rename the 'NinjaTrader 8.Prev' to 'NinjaTrader 8'. NinjaTrader looks for the user folder named NinjaTrader 8 specifically.