AddOn Project and Use of Reflection

[BarzTrading]

I'm working with the NinjaTraderAddOnProject template from this page (Considerations for Compiled Assemblies). I downloaded from the link on that page: "Download Visual Studio Solution for AddOn Development". The project is working fine and I can open the AddOn in NinjaTrader.

That page talks about casting types in compiled assemblies which could require using reflection and the dynamic keyword.

I have some confusion about the interplay between assemblies and types.

I would like to clarify some things:

• Would you agree that any AddOn project should not include a reference to the custom project?
• Would you agree that the whole point of the help page discussing Reflection and the dynamic keyword is because if we want access to ANY class defined in the Custom project from an assembly that is NOT the custom project, then this is the way we have to do it because we don't have those types available while developing?
• Can the same problem occur with code in the Custom project if we export to a compiled assembly?

[NinjaTrader_Jesse]

Hello BarzTrading,

Thank you for your post.

Would you agree that any AddOn project should not include a reference to the custom project?

Correct, the types in the custom project should not be referenced directly by adding it as a reference. If you need to use a type in the custom project, you should use dynamic/reflection and the full type name as a string such as "NinjaTrader.NinjaScript.DrawingTools.HorizontalLi ne".

Would you agree that the whole point of the help page discussing Reflection and the dynamic keyword is because if we want access to ANY class defined in the Custom project from an assembly that is NOT the custom project, then this is the way we have to do it because we don't have those types available while developing?

Yes, that is correct, with no reference to custom the items you would need to indirectly reference them in this way. dynamic gives you a type to work with so you can form objects with properties.

Can the same problem occur with code in the Custom project if we export to a compiled assembly?

Yes, this also applies to the considerations for compiled assemblies page.

I look forward to being of further assistance.

[BarzTrading]

Hi Jesse,

That helps a lot... I still have a few questions...

I implemented the example on the Help page "Considerations for Compiled Assemblies."

I first tested this code in an indicator:

        protected override void OnBarUpdate()
        {

            var lines = DrawObjects.Where(t => t is HorizontalLine).ToList();
            foreach (HorizontalLine line in lines)
            {
                // Print the tag of each Horizontal Line on the chart
                Print(String.Format("Horizontal Line {0} found.", line.Tag));
            }

            .... other code to draw a plot ...

        }

and then this code:

        protected override void OnBarUpdate()
        {
            foreach (dynamic line in DrawObjects.ToList())
            {
                Use ToString().Equals() to detect the object's Type
                if (line.ToString().Equals("NinjaTrader.NinjaScript.DrawingTools.HorizontalLine"))
                {
                    Access the object by assuming that it is the Type we expect
                    Print(String.Format("Horizontal Line {0} detected!", line.Tag));
                }
            }

            .... other code to draw a plot ...

        }

My indicator was in the Custom project in the Indicators folder. I'm trying to validate your reply to my 3rd question above.

In both cases I exported the indicator, then commented it in the Custom project, then imported the compiled assembly with the indicator containing the test code.

I had a couple of draw objects on the chart including a HorizontalLine.

The Indicator worked in both cases.

Is this expected?

When I looked at the contents of the compiled assembly with ILSpy I can see the HorizontalLine type in the assembly.

In fact, it looks like all of the drawing object classes are present in the exported assembly.

However, there are quite a few other types, like all of the built-in indicator types, that are not present in the exported assembly.

I don't quite understand why some types are in the exported assembly, but not others.

Is there another type I could do my test with where we would expect it to fail? Or do I need to do a different test to see the issue?

Thanks,

[NinjaTrader_Jesse]

Hello BarzTrading,

Thank you for your reply.

I believe this is expected because some items are included in the assembly to create the export. There are certain use cases in the platform which would cause the syntax without dynamic to become invalid so it would instead be suggested to use dynamic where shown in the considerations for compiled assemblies page.

I don't quite understand why some types are in the exported assembly, but not others.

The platforms compile and export process is complex and includes what it requires for the export/import to be successful. Generally what is inside the resulting compiled assembly should not be relevant to you as the developer, mainly the platform will require this to include the dll when importing and compiling.

Is there another type I could do my test with where we would expect it to fail? Or do I need to do a different test to see the issue

Is there a specific reason you are trying to make the script fail? I am unsure of a specific example that will fail here, but I can suggest using dynamic as that is known to work for compiled assemblies which reference external types.

I look forward to being of further assistance.

[BarzTrading]

Thanks, Jesse.

I'm just trying to understand when dynamic is required and when it's not. Right now, the distinction still seems partially arbitrary to me.

Let's assume at this point I'm only talking about code created in the Custom project and exported to a compiled assembly. And not anything to do with additional, separate projects. I understand why dynamic is needed in separate projects.

so it would instead be suggested to use dynamic where shown in the considerations for compiled assemblies page

Does this also mean that if I create an Indicator that uses the built-in SMA indicator internally, that I should use dynamic to reference the SMA type?

How is this any different than looping over a list of draw objects as specified in the help page? Both are types defined in the Custom project and used by my own indicator (or ninjascript) that I am exporting.

Are you saying, "if you plan to export any ninjascript or addon from the custom project to a compiled assembly that any classes it uses that are defined in the Custom project should _always_ be referenced using the dynamic key word"? And that the reason is because

There are certain use cases in the platform which would cause the syntax without dynamic to become invalid

and since we don't have a list of these use cases, that we should just cover our bases.

[NinjaTrader_Jesse]

Hello BarzTrading,

Thank you for your reply.

Does this also mean that if I create an Indicator that uses the built-in SMA indicator internally, that I should use dynamic to reference the SMA type?

No, indicators are not listed in the considerations page so this you can use them normally, this is also not a good comparison with the considerations help guide example.

On the considerations page, we are specifically examining comparing equality of objects which reside in different assemblies. The DrawObjects loop really has nothing to do with the test other than it is needed to get the objects we compare. In both examples, the same collection is being used and iterating the same objects.

There can be situations which this comparison will not be true. Using dynamic and string type names will avoid that. A good way to judge what needs to use dynamic would be to use this help guide page as an example rather than a specific list(like a lot of the help guide). The equality example may be seen in other situations where you are comparing objects and seeing they are not equal so this is really just explaining the concept of using dynamic in one known scenario.

How is this any different than looping over a list of draw objects as specified in the help page? Both are types defined in the Custom project and used by my own indicator (or ninjascript) that I am exporting.

In one case you are calling on an indicator to return a value and in the other case you are comparing equality. These are really two separate use cases which I couldn't directly compare.

Are you saying, "if you plan to export any ninjascript or addon from the custom project to a compiled assembly that any classes it uses that are defined in the Custom project should _always_ be referenced using the dynamic key word"? And that the reason is because

If you are comparing equality and the class doesn't exist in the current assembly, most likely yes. There are items like Drawing Objects that exist in the custom assembly which you may also do things like comparing equality which doesn't always succeed in the first direct comparison example.

I look forward to being of further assistance.

[BarzTrading]

Thanks Jesse,

OK. I think this is starting to gel.

Thank you in advance if you actually read all of this. I welcome any corrections or further clarifications.

I will use DR as an abbreviation for dynamic keyword and reflection.

I believe there are a few sources of confusion on our thread:

1) The word AddOn is used in two different ways.
2) I'm unsure if we mean the same thing when using the term compiled assembly.
3) There are actually two distinct scenarios where DR might be needed.

1) Meaning of AddOn

AddOn is generally used to mean anything that extends NinjaTrader including a zip file that was generated through an export.

AddOn is also a class that inherits from AddOnBase.

I think of an "AddOn Project" as a separate .NET project containing the code for an AddOn class inheriting from AddOnBase.

2) Meaning of "Compiled Assembly"

I think perhaps you may think of a compiled assembly as a DLL generated from an export operation. Whereas I have probably also been including a DLL produced by an AddOn project as a compiled assembly.

Which is correct?

It's looking more to me like "compiled assembly" must mean only DLLs produced via export. That's how I'll use it in the rest of this post.

3) Two Distinct Scenarios

* MISSING TYPES:

This scenario occurs in a separate .NET project that does NOT reference the Custom project such as an AddOn project. There are no extra or duplicate types in the generated assembly. The only issue is that a type might not be available at compile time.

* DUPLICATE TYPES:

When we create a compiled assembly through the export operation in NinjaTrader, the resulting DLL contains a bunch of extra duplicate classes, the majority of which are the drawing tools in the Custom project.

It's clear that part or all of the motivation for injecting these duplicate classes is related to the way NinjaTrader uses partial classes. Since partial classes cannot span assemblies, another approach is required.

The duplicates only create problems in certain scenarios as you mentioned.

Here's what I understand so far:

Every compiled assembly imported into the program contains its own copy of the HorizontalLine drawing tool
because it was injected into the assembly during export. (OK. There might be some scenarios where export doesn't inject HorizontalLine, but I haven't seen it yet. It doesn't matter though.)

Additionally, HorizontalLine is also in the Custom project.

This means that we can have not just two copies, but multiple copies of HorizontalLine while running NinjaTrader. In most cases, it doesn't matter that there are additional copies, but there are some cases where it does.

To help explain this, it's important to note that each of these copies of HorizontalLine is seen as a unique and distinct type! Even though each has the exact same namespace, same classname, and same code. Each class is a distinct type because each is further qualified by its assembly.

There appear to be some priority rules that .NET follows regarding which type it uses in different scenarios.

If my indicator code is in a compiled assembly and it uses the HorizontalLine type, it appears the version within that same assembly will be the type that is used because .NET first looks for a match in the same assembly.

Meanwhile if my indicator is accessing the DrawObjects collection, _those_ objects were NOT instantiated in my compiled assembly. Therefore their HorizontalLine is a type from a different assembly. I assume that would be the Custom assembly.

One might ask why the following code works. This is from the "Considerations for Compiled Assemblies" help page:

foreach (dynamic line in DrawObjects.ToList())
{
  // Use ToString().Equals() to detect the object's Type
  if (line.ToString().Equals("NinjaTrader.NinjaScript.DrawingTools.HorizontalLine"))
  {
      // Access the object by assuming that it is the Type we expect
      Print(String.Format("Horizontal Line {0} detected!", line.Tag));
  }
}

Well, when we use ToString() to print the fully qualified type name, it doesn't print the assembly. So regardless of which assembly the HorizontalLine is from, the if statement will be true for any HorizontalLine type regardless of assembly.

And because we used the dynamic keyword, line.Tag is not evaluated until runtime and presumably that would be a valid property for all versions of the HorizontalLine type.

So, what this code does actually, is _avoid_ making any equality comparison between objects.

Now, I think we know that anything in the DrawObjects collection is going to be a type from the Custom assembly, so we're not really worried about trying to validate which assembly version we have in this situation.

All of this also explains why my test detailed previously did not fail as I expected. First, while I thought it was working, it wasn't really, it just didn't throw an exception:

I used the following code:

var lines = DrawObjects.Where(t => t is HorizontalLine).ToList();
foreach (HorizontalLine line in lines)
{
    //Print the tag of each Horizontal Line on the chart
    Print(String.Format("Horizontal Line {0} found.", line.Tag));
}

When I debugged this code, I could see that the first line did not return any items and did not throw an exception, either.

Why? Looks like maybe the CIL (Common Intermediate Language) code that was generated from this line of code was able to distinguish the two different HorizontalLine types and just treat them as two different types.

So, the where clause failed to find any types in DrawObjects that matched, even though there were some on the chart.

Now here is the code from the help guide:

foreach (HorizontalLine line in DrawObjects.ToList())
{
    //Print the tag of each Horizontal Line on the chart
    Print(String.Format("Horizontal Line {0} found.", line.Tag));
}

It actually throws an exception. And I achieved this reliably.

I don't know exactly why it throws. You'd think that it would also just say that it found no matching HorizontalLine objects in the DrawObjects list like the where clause above did.

I'm going to assume that this boils down to how each line is represented in CIL. Looks like Linq probably does more complete type comparison by including the assembly qualification, while "foreach" or the "in" keyword perhaps ignores the assembly of each type and then we get an error further up the line when the runtime determines the types don't actually agree.

Now that I know (most of) the _why_ of this issue it makes a lot more sense. But the help page was little help, to be honest. All it did was let me know there was some problem I needed to worry about but did not help me understand it enough to know when and why I would need to implement these techniques.

Now, after some guidance from your replies, a lot of testing and looking inside the compiled assemblies, I understand the scope of this issue.

One important thing is simply knowing that I'll get an error message to let me know when the type cannot be cast and that will be when I need to implement one of these documented techniques.

And that error message will look something like this:

Indicator 'Demo SMA': Error on calling 'OnBarUpdate' method on bar 0: [A]NinjaTrader.NinjaScript.DrawingTools.HorizontalLin e cannot be cast to [B]NinjaTrader.NinjaScript.DrawingTools.HorizontalLin e. Type A originates from 'NinjaTrader.Custom, Version=8.0.17.2, Culture=neutral, PublicKeyToken=null' in the context 'LoadFrom' at location 'C:\Users\srlau\Documents\NinjaTrader 8\bin\Custom\NinjaTrader.Custom.dll'. Type B originates from 'DemoSMA, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null' in the context 'LoadFrom' at location 'C:\Users\srlau\Documents\NinjaTrader 8\tmp\6da5801570c046379b75162b0682c60d\DemoSMA.dll '.

If a note were added to the help page with this sample error it would be extremely helpful. The user would then know that they don't have to worry about this issue until they see this flavor of error when testing their compiled assembly.

But I think adding 1) A clearer definition of "compiled assembly" and 2) Adding a short paragraph explaining _why_ there are duplicate classes would also be extremely helpful to others in my shoes.

Thanks again,

Steve

[NinjaTrader_Jesse]

Hello BarzTrading,

Thank you for your extended reply.

Before we dive further into this reply I wanted to check, do you have a specific situation your running into or a specific question surrounding the details?

From what I can tell the majority of this reply details your testing using the non-suggested syntax from the considerations page. While this is detailed I don't see a specific problem or question here and we are looping back to using the code which is not suggested to demonstrate a known problem. I understand you want to better know when to use dynamic, but as noted the help guide only mentions this one known case which is common. If you are seeing other cases where something similar happens, this would generally be the solution to use dynamic, however, you can feel free to report those so we can review and document them if needed.

If you feel the help guide does not provide enough information here about using dynamic or reflection, I would suggest doing further research in external C# education materials as this will also help with understanding when this will be needed in your development.


I will try to briefly expand on the bolded questions you provided below:

1) Meaning of AddOn

Addon is a blanket term used for importable items however there is also a base class named AddOnBase in NinjaScript which can be used to create extended tools.


2) Meaning of "Compiled Assembly"

A compiled assembly is simply compiled code as a .dll. This is not a specific NinjaTrader term, this comes from C# and how C# creates compiled assemblies. An assembly can come from NinjaTrader or from Visual studio, they both export to .dll. NinjaTrader does have its own export process separate from visual studio which makes its own dlls which are different than the visual studio export dll due to the context of where it was exported from. These both lead to the same end or being used in NinjaTrader.



Please let me know if I may be of additional assistance.

[BarzTrading]

Thanks for the reply, Jesse.

Before we dive further into this reply I wanted to check, do you have a specific situation your running into or a specific question surrounding the details?

No, I'm a vendor with NinjaTrader and I am working on a training course related to the use of Visual Studio with NinjaTrader. My vendor username for this forum is "Steve L". I was posting under BarzTrading before becoming a vendor so just continued. Although, it might help give you guys more context to see me posting as a vendor.

From what I can tell the majority of this reply details your testing using the non-suggested syntax from the considerations page. While this is detailed I don't see a specific problem or question here and we are looping back to using the code which is not suggested to demonstrate a known problem.

My motivation for including all the details is two fold. First, to help others who land on this page and second to validate whether I'm understanding correctly.

If you feel the help guide does not provide enough information here about using dynamic or reflection, I would suggest doing further research in external C# education materials as this will also help with understanding when this will be needed in your development.

Well, yes, I did, but that does not negate my opinion that the help guide would be much better if it contained a bit more explanation about why this issue arises. It is not just a .NET issue. Having duplicate types in assemblies is not typical. It is an artifact of the NinjaTrader design.

A compiled assembly is simply compiled code as a .dll. This is not a specific NinjaTrader term ...

OK. Well, that's how I was interpreting it initially, but you have two very distinctly different scenarios that are being conflated.

A separate, .NET project such as an AddOn project does not have the issue with duplicate types. When the help page does not clarify that duplicate types only apply to compiled assemblies that were generated by exporting, it confuses things.

I feel I have a good handle on this issue at this point, but if I've said something incorrect in this post or the previous I would welcome any clarification.

Thanks for your time,

Steve