Change Ninjatrader Folder Location

[carnitron]

So that appears to work, thank you jeronymite. The doc is somewhat ambiguous on exactly what works when and what are the consequences of not following the rules, and I have been interpreting what is on that page differently, but I can see how one could conclude DataLoaded as being the "safest" place to add this event handler. Strangely, OnFundamentalData() even delivers data successfully after being added from the Configure or Active states.

Honestly though, using an API incorrectly should not break the compiler, lol. And really, if adding an event handler at phases that are too early is capable of producing bizarre, catastrophic consequences like that, then NT8 should prevent it. Throw an exception, log an error to the output window, and then prevent the addition. Something that humans can easily understand and react to. A simple error message would have turned a highly mysterious 3 day problem into a 3 minute no-brainer. I cannot imagine adding that validation code would be very hard or time consuming either.

My 2 cents anyway. Regardless, thank you again for the help, much appreciated.

NinjaTrader_Jim, at this point, several things:

1. Can you validate that you see the behavior with the sample AddOn I have provided?
2. Is there a good explanation of how and why something like this could even happen?
3. Presuming that we are able to validate the behavior, will there be a fix? At minimum, is there a way you could put in a request to add validation code to NT8 as suggested above, to save future customers from these issues? (This isn't for me, it's just for the betterment of the NT ecosystem, which is good for everyone here.)

Cheers.

[NinjaTrader_Jim]

Hello carnitron,

Thanks for providing the example, exactly what we need to help.

You need to make sure you add protective code for subscribing and unsubscribing events in OnStateChange.

When I add null checks to ensure FundamentalData is subscribed when null in State.Configure, and unsubscribed when not null in State.Terminated, the code works after recompiling without the issue.

The issue is coming up after recompiling because the NinjaScript Lifecycle for the AddOn is reiterated after we compile (see note #3 below, this would be what causes the code to encounter that issue.)

https://ninjatrader.com/support/help...fecycle_of.htm

Including null checks is needed to properly manage resources in OnStateChange and is a good practice in general.

NinjaTrader does have try catches implemented to trap errors in common methods, which we seen in the Log tab and the platform does not crash on the error. I.E. "Indicator encountered error in OnBarUpdate..."

I think this case would be similar to an infinite loop, or infinite recursion. It would still be able to take the platform down, and in order to catch it, we would have to be observant of the last change of code that presented the issue. Otherwise, it is always possible to do a clean environment test to verify the specific part involved.

Here is what I tried without issue:

else if (State == State.Configure)
{
    Print("State.Configure Reached in Test Script");
    Instrument instrument = Instrument.GetInstrument("AMD");

    if (m_fundamentalData != null)
        return;

    m_fundamentalData = new FundamentalData(instrument);
    m_fundamentalData.Update += OnFundamentalData;
}
else if (State == State.Terminated)
{
    if (m_fundamentalData == null)
        return;
    m_fundamentalData.Update -= OnFundamentalData;
}

[carnitron]

Apologies for the delayed response, Jim. I'm been trying to get to this all week, but this work thing keeps intervening, lol.

So this is interesting and definitely related But I actually now have logging statements (my own simple ones that call Print()) around that remove in Terminate, like so:

Log("EvilAddOn; State.Terminated; m_fundamentalData.Update; Before Remove ");

m_fundamentalData.Update -= OnFundamentalData;

Log("EvilAddOn; State.Terminated; m_fundamentalData.Update; After Remove ");

And on the 2nd compile we hang after we the first log statement but not the 2nd. In other words, the remove is probably blocking on something, stopping the terminate and blocking the compile button thread, which therefore prevents the new compile from actually starting, even though the UI makes it appear as if it has.

How do I synchronize here? I can't just lock on the m_fundamentalData object, b/c some other thread might not be using that as its synchronizing primitive. Like some NT8 thing is in the middle of sending an event here, and in the middle of that, I do a remove. A classic multi-threading conflict for lists. Is there a locking mechanism the NT8 internals respect that I can use too?

If not, what do you recommend?

Other comments:
1. Your second not null check in the above is supposed to be a null check?
2. FWIW, I suspect it is possible to have a watchdog thread that kills rogue addons that fail to terminate so the compiler can continue, hopefully with good error messaging, so users can tell what is actually happening.
3. Is there a way to reload addons without hitting the compile button?

Cheers

[NinjaTrader_Jim]

Hello carnitron,

I had mixed up the null check from a copy/paste, but as I go back I now see the issue. Even with null checks properly added.

This may have to do with the platform starting and then starting a fundamental data subscription too soon, but I will be reporting this so QA and Development can have a check.

AddOn windows typically use the constructor and CleanUp method of an NTTabPage class to subscribe + unsubscribe, and this works well. If you handle this code in an AddOn window (or create a custom object, and handle the same way) you won't have such issues and you will be able to include a "reload" mechanism.

I've made modifications to your example to show some practices to follow and how you can "reload" your AddOn.

[carnitron]

Thank you for that example. I have migrated my app over to activating and deactivating from the menu, and I think that is just a safer approach all the way around. It keeps all this "delicate" code out of the startup and teardown path NT8 does with AddOns when I recompile. Plus I get to control when my addon subscribes to data or not, which is actually useful for keeping my output log free of extra spam from prior instances. So I'm over to that and I think I'm good at this point.

I hope you are able to find traction with QA and Dev as I definitely feel like there are least two issues to fix here:
1. It should not be possible to stop compilation with an AddOn that becomes blocked in code run on the main "restart" thread.
2. Unsubscribing from fundamental data should not block indefinitely, under any circumstances, or there should be a synchronization mechanism coders can use to prevent contention when they modify the Update delegate list.

One further clarification of one point we've discussed here, if I may. So I can "reload" my addon now with my menu item as per your guidance. That's awesome. But I'm looking for clarification on NT8's UI. Right now, it feels like the model that NT8 uses for running a new version of my code is:

1. I save code in VS2022, NT8 automatically detects the save and attempts to compile. If the compile is successful, it reloads the AddOn automatically.
2. I modify the code in the NT8 editor and then hit the compile button (which saves the code automatically). If the compile is successful, it reloads the AddOn automatically.

Are these the only ways of "activating" code in NT8? Now that I think I understand it, it's actually a super low friction iteration path, which is cool. I originally assumed that I would maybe build my AddOn in VS2022 and then use some kind of "Reload" UI to restart it. It initially struck me as odd and not super discoverable that hitting the compile button was the way to reload an AddOn and I just want to confirm that I'm not missing other alternatives.

P.S. Is there a way to configure the NinjaScript Editor to use spaces instead of tabs? It's not a huge detail, I just need to keep cleaning tabs up in VS2022 whenever I make quick little fixes in the NT8 editor.

Ok, I think my barrage of questions may in fact be at its end, hah. Thanks again for all the help.

[NinjaTrader_Jim]

Hello carnitron,

1. I save code in VS2022, NT8 automatically detects the save and attempts to compile. If the compile is successful, it reloads the AddOn automatically.
2. I modify the code in the NT8 editor and then hit the compile button (which saves the code automatically). If the compile is successful, it reloads the AddOn automatically.

#1 - As long as the NinjaScript Editor is open, saving in Visual Studio, or dropping a file into bin/Custom will recompile. The NinjaScript Editor does this specifically for indicator wrappers, but saving with the NinjaScript Editor open becomes a good habit for consistency.

Either #1 or #2 would be how to bring in new code, yes.

P.S. Is there a way to configure the NinjaScript Editor to use spaces instead of tabs? It's not a huge detail, I just need to keep cleaning tabs up in VS2022 whenever I make quick little fixes in the NT8 editor.

There is not such a feature, no.

I'll update this thread after I have heard further details.

[carnitron]

Thanks for pursuing this Jim, much appreciated.

[Bend70]

Hi, please add me for the ability to specify where NT keeps the user files

I've wasted several days now (literally!) trying to recover from NT and Onedrive clashing. The fix to deselect the folder is only possible once the folder had been created, so that's something of a catch 22 and NT should be doing much better here. If I hadn't already purchased this would be a compelling reason to prefer other platforms

The problem is worsened since I have many years of data, which is many tens of thousands of individual files. Once Onedrive gets in a mess it takes over 24 hours to resync itself, very frustrating!

I'm currently mid way through the current lash up to get everything working on a new machine, one wasted day already. The idea that onedrive isn't universal on PCs by now and therefore ok to be 'unsupported' seems crazy!

Please could you update us on the current status of this 'feature request'? Thanks

[Bend70]

As follow up to the above, I thought I'd got close to a workaround, but now NT will not run, presumably since I've 'hidden' the \onedrive\documents\NinjaTrader folder by deselecting it in the onedrive setting as the previous instructions. Several iterations of uninstall/reinstall/deleting all NT files, syncing onedrive (in case its still messing something up) etc and I still can't get NT running

I'm on a new Windows 11 PC, and there seems no way to get NT installed away from Onedrive cleanly?
This seems the only possible process:
PC running onedrive (as pretty ubiquitous) happily
Exit or pause onedrive
Install NT, creates onedrive\documents\NinjaTrader8 folder
NT runs, but I also want Onedrive running for when I am not using NT
So, I need to deselect onedrive\documents\Ninjatrader8 in onedrive, so that it becomes (not onedrive)\documents\Ninjatrader8
To delect any folder in onedrive, onedrive must be running (to access the settings, no other way to do that?)
So I have to start onedrive, then go to settings and try to deselect the folder, but onedrive tells me the folder can't be unselected until the sync is completed
once sync is complete, I can then deselect onedrive\documents\NinjaTrader8 to put it 'outside' onedrive, phew
NT now fails to start (clicking the icon results in nothing happening)
I read the support to reinstall NT, so did that, but no change, NT won't start. Reboot etc no difference
Onedrive also now reports that there is an error because there are two versions of onedrive...

I remember it took several days a couple years ago to get NT and Onedrive to co-exist on the same machine (not run at the same time, I appreciate that is problematic)
Once I managed, my Windows10 machine ran fine, and I love NT
But here I am again, two days in and still no running NT

Seems all that's necessary to fix this for presumably everyone who runs Onedrive (most PC users?) is to allow a non-onedrive location to be specified in install?

Back to trying to get NT running, any ideas for help would be much appreciated. Thanks

[Bend70]

Hi, my last two posts are too long to be useful, though describe the difficult reality of trying to exclude the NinjaTrader8 folder from onedrive\documents

I've got back to the solution which I ran with successfully on my old machine for several years. Just pause or exit onedrive when using NT, and leave it running the rest of the time. No excluded folders

The advice in post #2 above, and elsewhere, suggests that for NT to work reliably we either must remove onedrive completely (not acceptable to most) or exclude the NT folder, which seems somewhere between messy and impossible, perhaps worse with Windows 11 than it was on 10?

Is there any reason to do anything other that pause/exit onedrive while running NT please? Thanks

[brenthilburn]

I will add my vote to NEVER change the path where NT installs nor allow it to be changed by the installer.

The people who want that ability have no idea what nightmare will come after that's available not only to NT platform support but the hundreds maybe thousands of vendors who have to support their products as well... The ONE thing you can count on with NT is it's consistently in the same place and has proven to be reliable and stable because of it...

Wait until Joe George Jim decides to bury his NT installation 4 layers deep because he can and then wonder why nothing works properly nor can he get any of his indicators to install properly. Wait until that really NEAT indicator you found on an obscure site won't load because it's a zipped Ninjascript file and it has no idea where NT is because you put it in a different place... Wait until the .msi and .exe indicator installation files no longer work because they were expecting NT to be in a specific place and now you're just going to have to wait for the indicator vendor to "get around" to correcting it...

The best solution is for Microsoft not to write a program that is so utterly intrusive that you can't shut it off while you use your computer the way YOU want to use it... Or maybe you could act like a professional who has purchased professional tools who uses them to make money and understand the stability of your platform should be your first concern and not being able to install it where you want... Because guess what... Microsoft will bone you again in the future just like they always do... At least NT is trying to help you work around the issue your O/S provider has caused...

Again... NEVER CHANGE it Ninjatrader... NEVER...

[kramnor2001]

For now I created Hyper V Virtual and installed Ninjatrader there. I don't have onedrive installed there so it solves the problem.

[tullips]

I have given up using onedrive, and went to use Google Drive where I have a bit more control. I do not want to blame Microsoft or NinjaTrader, it is just not liking OneDrive. Now that I am using Google Drive I have no issues anymore.

[QuantKey_Bruce]

Personally, I find OneDrive to be a complete pain, and uninstall it immediately upon encountering each new PC. Presto, NinjaTrader works fine.

[QuantKey_Bruce]

Also, I wish that NinjaTrader did not store its transient files like db -> cache (and quite a few of the others) in the Documents folder. That is not what the Documents folder is for, and the decision to put things like this there has led to the continuous OneDrive debacle. Storing templates or workspaces there is one thing, but these files do not belong. Yet, I agree with brenthilburn that if it is suddenly changed it will be a nightmare, so the right thing to do is to change this during a significant version upgrade e.g. NinjaTrader Desktop 8.2 or 9.0. Don't change it during the 8.0 or 8.1 version series. That way, vendors will have a chance to handle it properly while they are going about supporting the new version. Way back when, it was not the greatest decision to put that stuff in Documents. Yet, here we are, and changing now could capsize the boat that is floating currently.