Anyone using Forex Tick Charts?

[reach4thelasers]

I've been plagued with abysmal performance of-late. And I think it might be related to my use of Tick Charts. Does anyone else use them? More than 5 charts open with Tick Timeframes makes Ninjatrader and my PC is unusable.

Anyone else experience this?

[NinjaTrader_EricB]

Hello,

Thank you for your post.

A number of tick-based charts can cause performance issue. To test, I suggest applying our Performance Tips to your workspace, as they will generally able to address steps you can take to improve the behavior you're seeing.

http://ninjatrader.com/support/helpG...ance_tips2.htm

If custom add ons are in use, see the below section of our help guide for more on using and troubleshooting third-party add-ons.

http://ninjatrader.com/support/helpG...ty_add-ons.htm

We look forward to assisting further.

[reach4thelasers]

Hi Eric,

I'm aware of the performance tips and have tried them all. Yes I generally have a lot of tick charts open. Ninjatrader is literally not usable.

I tried installing a RAM Disk program and copying the whole Ninjatrader 8 Folder from Documents to RAM drive and placing a Junction Link in the documents folder so that Ninjatrader was writing to the RAM drive, however this did zilch for performance which would indicate that its not a disk problem.

Could you please find out/provide some more technical information about the nature of the Performance Problem with Tick Charts? I'd like to establish if there's anything I can do to resolve it.

This was supposed to be one of the Big Features of Ninjatrader 8. There was even a blogpost about it:

http://ninjatrader.com/blog/use-tick-bars-gain-new-perspective-markets/

Unfortunately its a bit of a non-feature if we can only have one chart open at a time!!!


Thanks

Kev

[NinjaTrader_EricB]

Thank you for your reply,

It would be best for us to look into what is occurring rather than speculate regarding the tick charts.

Please write into to PlatformSupport[AT]NinjaTrader[DOT]Com along with your log and trace files and a link to this forum post.

Please follow the steps below to manually attach your log and trace files to your response so I may investigate this matter further.

* Open your NinjaTrader folder under Documents.
* Right click on the 'log' and 'trace' folders and select Send To> Compressed (zipped) Folder.
* Send the 2 compressed folders as attachments to this email.
* Once complete, you can delete these compressed folders.

We look forward to assisting further.

[reach4thelasers]

Eric, you said "A number of tick-based charts can cause performance issue." so if this is a known issue then I doubt my logs will show anything.... Do you have any details about why they don't perform well? I'm looking to upgrade my hardware to get around it but I need more info about it...

[NinjaTrader_EricB]

Thank you for your reply,

Generally the tick-based charts can contribute more to performance tissues because more data is being received and processed than with time-based charts.

[reach4thelasers]

Hi Eric

Is is possible that the attached screenshot might be the cause of the performance problems with Forex Tick Charts? 3 Days of Tick bars being loaded!!!!

For an 8-Tick Chart of GBPJPY this is around 50,000 bars. And if I've got 10 charts open then this makes half-a-million bars being maintained internally.

Can you find out what Abstract Data Structure type is being used to store these internally?

The reason I ask is the insert time-complexity for all the classes in the .NET Collections Hierarchy except LinkedList is O(n) unless the data-structure's size is defined in advance. For a use-case like Ninjatrader's bar-updates the size is not known in advance because new items are constantly being added all the time....

What O(n) insert time-complexity means is that it takes 1 Million times longer to insert something to a structure that contains a million items as it does to insert something into a data structure that contains zero items. The more things you add the slower the inserts become.

In unmanaged languages like C, programmers are required to allocate memory in advance. Managed Runtimes like .NET do these things for you automatically, but every data structure in the .NET Collections Hierarchy is an abstract implementation of an Array. Arrays are stored in contiguous memory blocks. So when the internal array for a .NET collection runs out of space the CLR needs to make a new one, and copy each item one-by-one into the new array, then when you keep adding more items and that Array runs out of space it needs to do it again. This causes the performance of inserts to become slower and slower and slower as the size of the collection grows.

I am speculating here, but it makes sense... It would explain why after half a day of using Tick charts I need to quit Ninjatrader and restart, Half a day is another quarter million bars for 10 high-frequency tick charts. Your Solution Architect should know this stuff....

[reach4thelasers]

I genuinely don't know why I bother to try to help - the lengths I've gone to and the time and Money I've wasted trying to do so..,.. You released Ninjatrader 8 into beta 2 years ago. And after the most shambolic beta program I have ever witnessed you released it into production... and still... after 2 years it STILL performs badly, it is still riddled with bugs.... I've had to single-handedly write code and provide steps-to-reproduce for your bugs because your testers are incapable and your engineering team too lazy to do it themselves.... And still after all this time I continue to waste my time and my experience trying to help and provide advice and expertise - something clearly lacking on the Ninjatrader technology team. And yet you cannot even grace me with a response, or a thank you?

Seriously, can I ask, what happened to Dave and Matthew? Why are they no longer with Ninjatrader? And who is doing their job now? Because from what I can see the new policy is

- Lock down all threads from users complaining about bugs, to prevent new posts being added
- Ignore all advice from experienced programmers trying to assist with Ninjatraders problems.

Which frankly is a totally stinking attitude

[NinjaTrader_PatrickG]

This is Patrick responding on behalf of Eric. I am the Platform Support Team Lead and I'm here to make sure all of your questions and concerns are addressed.

We pride ourselves on the quality of our support, and we appreciate your detailed feedback in this thread and in others. I apologize for the delayed response. Your inquiry went a bit deeper into the program than most so I wanted to make sure I had accurate information to provide you.

To answer one of your points directly, NinjaTrader uses a logic-optimized list structure under the hood. There are many reasons why NinjaTrader is fundamentally programmed this way.

In general, millions of data points are going to increase the load on NinjaTrader/your PC and performance slow-down is expected. There are some simple things which can be done in the platform (reducing workspaces, reducing windows, reducing data, other settings mentioned here) to try to mitigate this performance behavior.

If you would like us to look into your unique situation, then we will need additional specific information about your NinjaTrader and PC setup. Please write into PlatformSupport[AT]NinjaTrader[DOT]com with a link to this forum thread and '1734763 ATTN Patrick G' in the body so we can work together to find out if the root of the issue is within NinjaTrader or elsewhere.