Global packages no longer work after node.js update with homebrew

[publicarray]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?

When node and yarn are installed with homebrew yarn global bin returns a path that is tied to the a specific version of node. If node is updated with brew the symlinks are no longer valid, i.e global packages with binaries need to be reinstalled.

~ ❯ yarn global bin
/usr/local/Cellar/node/7.2.0/bin

Related issue: Homebrew/homebrew-core#7283

If the current behavior is a bug, please provide the steps to reproduce.

I've used brew reinstall to emulate an update, I'm not sure how different it is to brew upgrade node.

~ ❯ brew install node
~ ❯ node --version
v7.2.0
~ ❯ brew install yarn
~ ❯ yarn global bin
/usr/local/Cellar/node/7.2.0/bin
~ ❯ yarn global add bower
~ ❯ bower --version
1.8.0
~ ❯ brew reinstall node --HEAD
~ ❯ node --version
v8.0.0-head
~ ❯ yarn global bin
/usr/local/Cellar/node/HEAD-11b0872/bin
~ ❯ bower --version
command not found: bower

What is the expected behavior?

~ ❯ bower --version
1.8.0

Please mention your node.js, yarn and operating system version.

~ ❯ yarn --version
0.17.9
~ ❯ node --version
v7.2.0
~ ❯ sw_vers
ProductName: Mac OS X
ProductVersion: 10.12.1
BuildVersion: 16B2555

[nicgordon]

The same issue exists for node installed via nvm:

~ $ which node
/Users/nicgordon/.nvm/versions/node/v7.2.0/bin/node
~ $ yarn global bin
/Users/nicgordon/.nvm/versions/node/v7.2.0/bin
~ $ yarn global add gatsby
~ $ which gatsby
gatsby not found

However adding this prefix flag symlinks the installed module into /usr/local/bin and fixes the issue for me

~ $ yarn global add gatsby --prefix /usr/local
~ $ which gatsby
/usr/local/bin/gatsby

Checking the symlink shows me that the module was actually installed here:

/Users/nicgordon/.config/yarn/global/node_modules/.bin/gatsby

Found this solution here #1877

[publicarray]

Thank you, however I personally think that your fix is more of a work around and not a permanent solution. I was looking to start a discussion of how this could be handled within yarn or brew so that yarn global packages are not effected on a node.js version change. On the other hand npm global packages function fine after a brew upgrade node.

[kamui545]

I had the same issue several times too.
It would be nice to have this working without the prefix setting

[gutenye]

same issue here, is there a permanent solution?

[OmgImAlexis]

Got the same issue here.

[iddan]

same issue

[bradley-curran]

Yarn adds a symlink of the executables to node's bin directory.
In my instance it was /usr/local/Cellar/node/7.2.1.
The actual place where the global packages are installed is ~/.config/yarn/global/node_modules/.

When I updated to node 7.4.0 from 7.2.1 the symlinks weren't added to the new version. I'm guessing you're seeing the same thing.

The problem is, there's no way of using yarn right now to fix this.

You could write a script that does something along the lines of:

• Get the global packages directory for yarn
• Find all the files that are executable
• Symlink all the files into your current node directory

I'll write an npm package later that does all this.

[bradley-curran]

Hey guys,

I've written an npm package that should fix your yarn-installed global packages.

$ npm install -g fix-yarn-global-packages
$ fix-yarn-global-packages

That should fix all your global packages. Let me know if it doesn't work for you.

https://www.npmjs.com/package/fix-yarn-global-packages

[publicarray]

Thanks @bradley-curran but I personally prefer to just add alias yarn-global='yarn global --prefix /usr/local' to my dotfiles until this is fixed.

[bradley-curran]

I've been doing some more digging and it seems the issue is that yarn doesn't read all npm config files that npm itself uses.

Yarn should read all files defined in npm's npmrc documentation

Currently yarn only reads config files from:

• ./.npmrc
• ~/.npmrc
• <node directory>/.npmrc (<node directory> = /usr/local/Cellar/node/7.6.0/ for me)

The config files that also need to be read include:

• $PREFIX/etc/npmrc ($PREFIX is usually /usr/local)
• path/to/npm/itself/npmrc

The problem is that these files can't be found easily using the existing architecture afaict.
Ideally the existing config loading would be replaced with delegating to npm config ls -l and parsing the result rather than trying to read all the files.

Quick Fix

• Open ~/.npmrc in your favourite text editor.
• Add the line prefix = /usr/local

The Real Fix

Read all configuration files, ideally by delegating to npm config, rather than reading existing INI files. I'd be happy to contribute, though I'm not really sure who I can discuss implementation details with.

[publicarray]

@bradley-curran thanks so much for looking into this, from now on I'll use the npmrc file. (I didn't know that the npmrc existed until now). Just a small word of warning that npm will also install global packages to the prefix set in in the npmrc and no longer in /usr/local/Cellar/node/7.6.0/ as previously.

[mkutny]

Agree it should not install into /usr/local/Cellar/node/version .

As a temporary fix, it's now possible to yarn config set prefix /usr/local -g

Although this prefix should be set by default on yarn installation.

[publicarray]

Similar issue: #2639

[publicarray]

Thanks @mkutny, is the -g flag supposed to do something? for me the config file that is modified is always ~/.yarnrc

[mkutny]

@publicarray , I just did it according to the doc which says yarn config set <key> <value> [-g|--global]

It seems that in this particular case it doesn't matter but for options which could be set for either local or global scope, it does.

[publicarray]

@mkutny thanks, I should have read the docs 📖

[HenrikWL]

Can't this be fixed by having the homebrew formula do

yarn config set prefix /usr/local/

as a post-install?

[bradley-curran]

You can, if you're willing to assume that the prefix is /usr/local on all machines, which I'm not sure can be assumed.

[HenrikWL]

Good point. Within the context of Homebrew formulae, I do believe the global Homebrew-prefix is available in a variable. It would be better to use that.

[vdh]

Please stop advocating the /usr/local prefix workaround. Some of us prefer to stick with the versioned bin directory (e.g. /usr/local/Cellar/node/7.9.0/bin) and keep /usr/local/bin clean. Editing $PATH is a whole lot easier than cleaning up dead symlinks from an overloaded /usr/local/bin directory.

[mkutny]

@vdh , it's not about which solution "is a whole lot easier". It's all about what is the right thing to do.

If you read the thread carefully there are currently several issues with global packages:

• they are not immediately available after yarn global add (they need to be manually symlinked)
• global packages won't work after node update ($PATH needs to be updated)
• global packages will be removed if obsolete node version is purged (editing $PATH won't help at all)

Binaries in /usr/local/bin not only solve all these issues but are also consistent with Unix way of doing things in general and with Homebrew in particular.

[HenrikWL]

@vdh the solution to dead symlinks in /usr/local/bin is to fix the problem that causes dead symlinks in there, not to preserve broken behaviour in yarn. Like @mkutny says, there are several issues with yarn global the way it works now. I expect it to work like npm -g but it doesn't. If that's by design then that's ok, I'll just stick with npm -g for global packages and keep using yarn for project packages (for which it works great).

[partounian]

I'm not sure if this is related or not, but with zsh, I have an export on startup with the path from yarn global bin, but on startup I always get command not found: yarn how it calling yarn does work. On top of this with global packages that have their own command, I get command not found when trying that call them, but it all works fine when I reload zsh in the same terminal window.

[publicarray]

@partounian I recommend you open a new issue because your problem is unrelated. Is it possible that you are calling yarn before you initialised the $PATH? e.g.

~/.zshrc:

yarn global bin
export PATH=$(brew --prefix)/bin:$PATH # run this first

[Daniel15]

Adding ~/.config/yarn/global/node_modules/.bin to your $PATH is probably a reasonable workaround for now.

The issue is that Yarn symlinks the executables into the wrong directory (the Node.js directory rather than somewhere reasonable). It doesn't really make sense to use the Node.js directory for these scripts, and on many systems that directory is not even writable by regular users. I fixed this for Windows in #3233 but we still need fixes for other operating systems. #3458 seems reasonable as long as the user has write access to /usr/local/bin, although maybe something like ~/.yarn/bin would be better.

[KishanBagaria]

TL;DR Use a default directory for Windows/macOS. Ask user to set directory on Linux/others.

Windows and macOS
/usr/local/bin is writable in a default macOS installation and that's where Homebrew writes stuff. When node/npm is installed from Homebrew, Homebrew edits the prefix config in .npmrc so that npm links the global binaries in /usr/local/bin too.

For Windows and macOS, we can use %LocalAppData%\Yarn\bin and /usr/local/bin as default, because that's guaranteed to work out-of-the-box (unless the user is on a broken/unusual installation).

On macOS, Yarn global binaries will work without extending the PATH since /usr/local/bin is already present in default PATH. Although, the directories in default Windows PATH all need admin privileges, so Chocolatey or the user will have to extend the PATH with %LocalAppData%\Yarn\bin.

Linux et al.
For Linux and other unrecognized platforms, we shouldn't assume anything (any of the parent directories of process.execPath isn't reasonable or guaranteed to be writable as @Daniel15 suggested). We can test if /usr/local/bin is writable before using it. But there's thousands of configurations of Linux distros with different versions and different defaults. I dunno if that directory will work on all of them.
Instead, we can ask the user to set the prefix explicitly. Running any yarn global command on these platforms should throw an error and say something like

error you need to first run yarn config set prefix <global_prefix_dir>
info see https://yarnpkg.com/en/docs/cli/global

Or we can show an interactive inquirer.js prompt where the user can select one among many commonly-used prefixes:

* /usr/local/bin
* /usr/bin
* /bin
* ~/n/bin
* ~/bin
* <enter custom prefix>: ______

The current behaviour has been copied from npm which was written about 5 years ago and has seen no updates: https://github.com/npm/npm/blob/771c66884a0a1851cffb3bf132828f6ccef1c09c/lib/config/defaults.js#L92-L105

[BYK]

When node/npm is installed from Homebrew, Homebrew edits the prefix config in .npmrc so that npm links the global binaries in /usr/local/bin too.

Shall we honor this setting too?

Although, the directories in default Windows PATH all need admin privileges, so Chocolatey or the user will have to extend the PATH with %LocalAppData%\Yarn\bin.

I think this is a really subpar experience for Windows users. Node puts itself into the path along with npm right, why can't we use those directories directly?

[Daniel15]

I think this is a really subpar experience for Windows users

The Windows installer (and Chocolatey package) automatically update the path, so this is mostly a non-issue.

Node puts itself into the path along with npm right, why can't we use those directories directly?

On most systems, it would require you to run Yarn as administrator, as Node is normally installed at C:\Program Files\nodejs and Node.js doesn't expose the Win32 API for requesting elevation (getting admin rights after the app has started, like "sudo")

[BYK]

@Daniel15 oh, okay so this clears my Windows-side questions :)

[BYK]

Well, with the exception of, when/if we install yarn via NPM or as a direct script. I'm not sure if we need elevated permissions when running npm install -g do we?

[Daniel15]

I'm not sure if we need elevated permissions when running npm install -g do we?

I think it'd depend on where Node.js is installed and what the permissions are on that directory.

[Daniel15]

Well, with the exception of, when/if we install yarn via NPM or as a direct script.

We could probably update Yarn so that it edits the path on first run. However I don't know if Node.js exposes an API for that. It's got extremely limited ffi capabilities without dealing with complex native modules.

[BYK]

Did some reading on https://docs.npmjs.com/files/folders#prefix-configuration and looks like the approach in the diff is correct. Just concerned about not using the npm folder (we use Yarn instead). I think this is fine until we get complaints.

[nodox]

sudo yarn global add ... worked but I'm not sure that should be the best solution

[diegoazh]

I had the same issue in ubuntu after upgrade node version with nvm, in my dev proyects I fix it removing .lock file en running yarn cache clean and reinstall its. I think that if you remove the .lock file inside of ~/.config/yarn/global/ (in ubuntu) and reinstall all packages fix the error with global packages; also I recomend defining install location with yarn config set prefix ~/.yarn like this docs suggests