Excessive Cache Usage for Local Dependencies

[mitsuhiko]

What is the current behavior?

When you yarn install a local dependency instead of using something like npm pack it would appear yarn copies the entire target path into the cache folder and then installs a packed version from that path later. I have a local folder which has about 800MB of node_modules and other stuff in it and a package.json which references about 8KB of data. When you yarn add that path as dependency yarn copies 800MB of useless data into the cache folder. This also makes it appear "stuck" (shows 0/1 without a progress bar) for about 3 minutes on my mac.

Steps to reproduce

Just check out any npm package locally and add useless files to it. When you add a local dependency with yarn add to that path it should show that behavior.

A real world repo with that issue is sentry-cli (https://github.com/getsentry/sentry-cli). If you build it with cargo build && cargo build --release and then try to yarn add the path the cache fills with stuff and local dependency adding takes a llong time.

What is the expected behavior?

Local dependencies should probably not be cached at all and if they are, less data should be cached.

node.js, yarn and operating system versions

$ yarn --version
0.24.4

$ uname -a
Darwin herzog.local 16.5.0
Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017;
root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64

[bestander]

Thanks for reporting this.
Indeed the file: dependency just copies the whole folder into cache every time you install.

We could start with:

1. clearing the cache after using it because this cache is not used anymore after the installation (PR is welcome!).
2. Filtering the files to copy into cache folder based on files property in package.json (Another PR is welcome)

Besides that we are working on Workspaces #3294 that will address local development via symlinks.
A lighter alternative is link: specifier that is in the works yarnpkg/rfcs#34.

[BYK]

I think we should treat local dependencies simply as "cached" and skip the copying to cache step. This would be a nice improvement. I think this is a sister ticket to #3344.

[BYK]

Actually, looks like #3344 covers this so closing in favor of that.