Support unpublished lerna packages – ignore symlinked packages or not found failures

[timoxley]

What is the current behavior?
Currently, if a dependency is not found in the registry, yarn will error:

error Couldn't find package "unpublished-pkg" on the "npm" registry.

This is a problem if you're using lerna to manage internal packages, since running any yarn commands inside a packages/* dependency will fail. This means you currently cannot add/remove dependencies to a package.json/yarn.lock file using yarn directly if the package depends on an unpublished local dependency. The only way to get any dependencies to install at all for such packages is to use lerna bootstrap.

steps to reproduce:

#!/bin/bash

set -e

rm -Rf testpkg && true
mkdir testpkg
cd testpkg
yarn init --yes
lerna init

# create an unpublished package "unpublished-pkg"
(
  mkdir -p packages/unpublished-pkg
  cd packages/unpublished-pkg
  yarn init --yes
)

# create another package with a dependency on "unpublished-pkg"
(
  mkdir -p packages/depends-on-unpublished-pkg
  cd packages/depends-on-unpublished-pkg
  echo '{"name":"depends-on-unpublished-pkg", "version": "0.0.0", "dependencies": {"unpublished-pkg": "*"}}' > package.json
)

yarn
lerna bootstrap --npm-client=yarn

# try run any yarn command from depends-on-unpublished-pkg
(
  cd packages/depends-on-unpublished-pkg
  yarn add mkdirp # will fail
)

What is the expected behavior?

Ideally, yarn would simply ignore any existing symlinked packages, or could be told to do so.

yarn add mkdirp --ignore-symlinked

Or there would be a way to tell yarn to ignore particular packages:

yarn add mkdirp --ignore=unpublished-pkg
# or maybe
yarn add mkdirp --no-check
# or maybe
yarn add mkdirp --ignore-not-found
# etc

Ideally this could be set from either the commandline or as a persistent package-wide config via package.json or a .yarn file:

{
  "yarn": {
    "ignore-symlinked": true
  }
}

Very open to other ideas or workarounds.

---

Issue originally reported on lerna repo, but I now feel this is more of a yarn concern: lerna/lerna#501

[timoxley]

Possible this should be an rfc and/or is perhaps superseded by yarnpkg/rfcs#34

[richburdon]

@timoxley did you find a solution for this?

I agree this issue will be covered by the workspaces project (#3294), but that seems like a longterm project; a small fix here could be a really big win.

lerna works really well today, except for this issue that prevents it working well with yarn. I.e., I use lerna with yarn to manage my monorepo, but ironically still need to use npm to manage local package deps.

I don't think you'd need a config file change -- it seems yarn could easily respect the already present symlnks?

[bestander]

Yarn workspaces are available for this use case

[richburdon]

Hi @bestander. Every month or so I try to check in to see if #3294 has come out of RFC phase. I can't find anything here: https://yarnpkg.com/en/docs. I can't find anything on Google either except for others eagerly awaiting it: https://medium.com/spektrakel-blog/yarn-workspaces-are-coming-477c3707f482?

Are there any docs?

yarn help workspace

Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.

Returns 404

No mention here either: https://yarnpkg.com/en/docs/cli/

[bestander]

Hey, @richburdon.
I am writing a blog post about what stage workspaces are in now, how to use them and what is the future. Planning to publish it at the end of this week at https://yarnpkg.com/blog/.

Workspaces are out of RFC and the available in 0.28 release, still tentatively calling it "experimental" to allow for breaking changes when the solution is refined.
Check out Jest for how it can be used jestjs/jest#3906.

[richburdon]

Thanks @bestander; really looking forward to this.

I looked here (jestjs/jest#3906) but don't see any docs on usage or the config. Is there a sample project? If not, do you have a timeframe in mind for your blog post.

Thanks again for the work.

[jasonmacdonald]

I've tried turning this on in an test app and even with workspaces enabled yarn still complains when trying to call yarn add <pkg> --save inside a workspace package. Has this not been addressed?

Don't get me wrong, the auto hoisting being done by yarn now is nice, but Lerna did that for me too. The biggest problem we've always had is that when using yarn you can't install package dependencies inside sub packages without yarn complaining about unpublished packages. This forces you to either use npm install (which doesn't work with NPM 5 either now) or hand editing the package.json. I hate this break for a normal devleoper workflow of installing packages.

Am I missing some config I can set to get yarn to ignore workspace packages?

[bestander]

yarn add <pkg> from a workspace folder should work fine, there are tests that cover this feature.
Could you raise an issue with repro steps?

[jasonmacdonald]

@bestander It seems there's already an issue for it #3973