Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage of bzlmod causes cache invalidations due to module version being in file path #20997

Closed
v3n opened this issue Jan 23, 2024 · 1 comment
Closed
Assignees
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@v3n
Copy link

v3n commented Jan 23, 2024

Description of the feature request:

Remove module version from the visible module path as it causes frequent cache invalidations when modules updates. This is particularly problematic in C++ where changes in include paths due to module versions can propagate a cache invalidation over most of the build graph; despite no actual dependent files changing.

Which category does this issue belong to?

External Dependency

What underlying problem are you trying to solve with this feature?

No response

Which operating system are you running Bazel on?

Mac OS

What is the output of bazel info release?

release 6.4.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@meteorcloudy meteorcloudy added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label Jan 23, 2024
@meteorcloudy meteorcloudy changed the title Usage of bzlmod causes cause invalidations due to module version being in file path Usage of bzlmod causes cache invalidations due to module version being in file path Jan 23, 2024
@meteorcloudy meteorcloudy added P2 We'll consider working on this in future. (Assignee optional) area-Bzlmod Bzlmod-specific PRs, issues, and feature requests and removed untriaged labels Jan 23, 2024
copybara-service bot pushed a commit that referenced this issue Jan 26, 2024
These cause failures when relying on `depGraph` containing `ModuleKey`s for all modules, which is the case in production.

Work towards #20997

Closes #21037.

PiperOrigin-RevId: 601822357
Change-Id: Ifcad9d7b73835491c1f1fca975e05834057a6825
bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 26, 2024
These cause failures when relying on `depGraph` containing `ModuleKey`s for all modules, which is the case in production.

Work towards bazelbuild#20997

Closes bazelbuild#21037.

PiperOrigin-RevId: 601822357
Change-Id: Ifcad9d7b73835491c1f1fca975e05834057a6825
github-merge-queue bot pushed a commit that referenced this issue Jan 26, 2024
These cause failures when relying on `depGraph` containing `ModuleKey`s
for all modules, which is the case in production.

Work towards #20997

Closes #21037.

Commit
27419e3

PiperOrigin-RevId: 601822357
Change-Id: Ifcad9d7b73835491c1f1fca975e05834057a6825

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
copybara-service bot pushed a commit that referenced this issue Feb 5, 2024
The name can instead be set centrally in `BzlmodRepoRuleFunction`. This removes some unnecessary glue code and paves the way for rewriting canonical repo names based on knowledge of the full dep graph.

Work towards #20997

Closes #21026.

PiperOrigin-RevId: 604256186
Change-Id: Ia49123f016d1512074b04ec458f017f4bbb88233
iancha1992 pushed a commit to iancha1992/bazel that referenced this issue Feb 5, 2024
The name can instead be set centrally in `BzlmodRepoRuleFunction`. This removes some unnecessary glue code and paves the way for rewriting canonical repo names based on knowledge of the full dep graph.

Work towards bazelbuild#20997

Closes bazelbuild#21026.

PiperOrigin-RevId: 604256186
Change-Id: Ia49123f016d1512074b04ec458f017f4bbb88233
iancha1992 pushed a commit to iancha1992/bazel that referenced this issue Feb 5, 2024
The name can instead be set centrally in `BzlmodRepoRuleFunction`. This removes some unnecessary glue code and paves the way for rewriting canonical repo names based on knowledge of the full dep graph.

Work towards bazelbuild#20997

Closes bazelbuild#21026.

PiperOrigin-RevId: 604256186
Change-Id: Ia49123f016d1512074b04ec458f017f4bbb88233
Wyverald pushed a commit that referenced this issue Feb 6, 2024
The name can instead be set centrally in `BzlmodRepoRuleFunction`. This removes some unnecessary glue code and paves the way for rewriting canonical repo names based on knowledge of the full dep graph.

Work towards #20997

Closes #21026.

PiperOrigin-RevId: 604256186
Change-Id: Ia49123f016d1512074b04ec458f017f4bbb88233
github-merge-queue bot pushed a commit that referenced this issue Feb 6, 2024
The name can instead be set centrally in `BzlmodRepoRuleFunction`. This
removes some unnecessary glue code and paves the way for rewriting
canonical repo names based on knowledge of the full dep graph.

Work towards #20997

Closes #21026.

PiperOrigin-RevId: 604256186
Change-Id: Ia49123f016d1512074b04ec458f017f4bbb88233

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
Wyverald pushed a commit that referenced this issue Feb 12, 2024
The canonical repository name of a module `rules_foo` is now `rules_foo~` instead of e.g. `rules_foo~1.2.3` if there is only a single version of the module in the entire dep graph. This also applies to overrides, which previously used the canonical repository name `rules_foo~override`.

This improves cacheability of actions across module version changes and also prevents the output base from filling up with outdated module and extension repositories. See the long comment in `ModuleKey#getCanonicalRepoName` for a detailed explanation of why this particular scheme was chosen.

The change includes a bump of the lockfile version as the canonical repo names of label attributes in extension usages are affected.

RELNOTES: The scheme for generating canonical repository names has changed to improve cacheability of actions across dependency version updates. Note that canonical names are not considered to be public API and can change at any time. See https://bazel.build/external/module#repository_names_and_strict_deps for advice on how to avoid hardcoding canonical repository names.

Fixes #20997

Closes #21035.

PiperOrigin-RevId: 605730371
Change-Id: Ica1be1ba5493d3636248a79a6549a0927021bef9
Wyverald pushed a commit that referenced this issue Feb 12, 2024
The canonical repository name of a module `rules_foo` is now `rules_foo~` instead of e.g. `rules_foo~1.2.3` if there is only a single version of the module in the entire dep graph. This also applies to overrides, which previously used the canonical repository name `rules_foo~override`.

This improves cacheability of actions across module version changes and also prevents the output base from filling up with outdated module and extension repositories. See the long comment in `ModuleKey#getCanonicalRepoName` for a detailed explanation of why this particular scheme was chosen.

The change includes a bump of the lockfile version as the canonical repo names of label attributes in extension usages are affected.

RELNOTES: The scheme for generating canonical repository names has changed to improve cacheability of actions across dependency version updates. Note that canonical names are not considered to be public API and can change at any time. See https://bazel.build/external/module#repository_names_and_strict_deps for advice on how to avoid hardcoding canonical repository names.

Fixes #20997

Closes #21035.

PiperOrigin-RevId: 605730371
Change-Id: Ica1be1ba5493d3636248a79a6549a0927021bef9
github-merge-queue bot pushed a commit that referenced this issue Feb 13, 2024
The canonical repository name of a module `rules_foo` is now
`rules_foo~` instead of e.g. `rules_foo~1.2.3` if there is only a single
version of the module in the entire dep graph. This also applies to
overrides, which previously used the canonical repository name
`rules_foo~override`.

This improves cacheability of actions across module version changes and
also prevents the output base from filling up with outdated module and
extension repositories. See the long comment in
`ModuleKey#getCanonicalRepoName` for a detailed explanation of why this
particular scheme was chosen.

The change includes a bump of the lockfile version as the canonical repo
names of label attributes in extension usages are affected.

RELNOTES: The scheme for generating canonical repository names has
changed to improve cacheability of actions across dependency version
updates. Note that canonical names are not considered to be public API
and can change at any time. See
https://bazel.build/external/module#repository_names_and_strict_deps for
advice on how to avoid hardcoding canonical repository names.

Fixes #20997

Closes #21035.

PiperOrigin-RevId: 605730371
Change-Id: Ica1be1ba5493d3636248a79a6549a0927021bef9

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
@iancha1992
Copy link
Member

A fix for this issue has been included in Bazel 7.1.0 RC1. Please test out the release candidate and report any issues as soon as possible. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants