proposal: cmd/go: support local directive in go.mod

[liubog2008]

Why

One go.mod is not reasonable for monorepo

For example, we have a repo with structure as below

- cmd
- pkg
  - server
  - client
  - api
- go.mod
- go.sum

If the repo consumers hope to import client and api pkg, they have to import the whole module and indirectly import dependencies of server pkg.

Only three choices for this situation.

1. New git repo for client and api
2. New go.mod file and new git tag for client and api, e.g client/v0.1.0
3. New go.mod file and use replace directive

For choice 1, we have to manage more git repos and submit more PRs for a feature.
For choice 2, we have to tag api and client, then change the go.mod version and tag main module.
For choice 3, replace directive has some problems now.

replace directive for monorepo

The below lines are used in many go.mod files.

replace github.com/xxx/yyy/api => ./api
replace github.com/xxx/yyy/client => ./client

However, go install is not worked if replace directive exists(See #44840, #40276)

Another problem of replace is if client also imports api and replaces it. consumers have to use replace in go.mod again to ensure api module's version.

require (
    github.com/xxx/yyy/api v0.0.1
    github.com/xxx/yyy/client v0.0.1
)

# If omit this line, the below error will be thrown
# go: github.com/xxx/yyy/client@v0.0.1 requires
#.     github.com/xxx/yyy/api@v0.0.0-00010101000000-000000000000: invalid version: unknown revision 000000000000
replace github.com/xxx/yyy/api => github.com/xxx/yyy/api v0.0.1

Design

1. Add local directive for all modules which are imported from same git repo.
2. Use same version as the main module to import local modules.

In main module

module github.com/xxx/yyy

local (
    github.com/xxx/yyy/client
    github.com/xxx/yyy/api
)

in client module

module github.com/xxx/yyy/client

local (
    github.com/xxx/yyy/api
)

in consumer module

module github.com/aaa/bbb

require (
    github.com/xxx/yyy/client v0.0.1
    github.com/xxx/yyy/api v0.0.1
)

go mod graph for consumer

github.com/aaa/bbb github.com/xxx/yyy/client@v0.0.1
github.com/aaa/bbb github.com/xxx/yyy/api@v0.0.1
github.com/xxx/yyy/client@v0.0.1 github.com/xxx/yyy/api@v0.0.1

Compatibility

No compatibility problem

Unresolved Issues

Git tag client/v0.0.1 will be conflict with v0.0.1in semantic. There are two solutions:

1. client/v0.0.1 will override the v0.0.1 tag.
2. Throw error if they are both used. (I prefer)

Alternatives

1. Just omit version in require as the local module. (also good to me)
2. Change the behavior of replace directive.

[ianlancetaylor]

CC @bcmills

[beoran]

Go 1.18 has workspaces. Could they be used in your use case?

[mvdan]

Also note that this is otherwise a duplicate of #26640.

[liubog2008]

Go 1.18 has workspaces. Could they be used in your use case?

go.work seems a local workspace and not be pushed to remote repo in best practice? If I split monorepo to three git repos, go.work can be easily used for me to modify my code.

[liubog2008]

Also note that this is otherwise a duplicate of #26640.

I feel they are not duplicated.

[bandesz]

I would like to add a further issue to consider here.

We have a similar project structure, but we also use vendoring in the root project. Using the example from the description, this means the pkg/client module is copied to vendor (but with full module path of course), and whenever you make a change to pkg/client during development, you have to run go mod vendor.

[liubog2008]

Any feedback about this proposal?
I think it's useful for many go projects to removereplace in their go.mod.
For example, the popular KV store ETCD.
We can remove replace in https://github.com/etcd-io/etcd/blob/main/go.mod#L5-L15 and use go install to install etcdctl

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

As envisioned above, this feature would be incompatible with modules. The specific goal seems to be to have something that is like replace but that applies during go install. But applying during go install means applying during any kind of version selection decisions, and it would impose a constraint of the form "use exactly this version" instead of "use this version or newer". Allowing that new kind of constraint changes version selection from trivial to NP-complete, which we're not going to do.

A separate problem would be how to infer the actual version to use. The go command does not fundamentally operate on Git repositories, and there is no way to express "use the same Git commit for m/sub as for m". So even if that constraint were not computationally problematic, it still depends on breaking down abstractions that we currently have.

To work on a collection of related modules in a single repository and have the convenience of having them all bound together while working in a git clone of the repo, the answer is go.work.

The answer for making go install work is to tag the sub-modules, then update the appropriate require lines in the command module, then tag the command module.

[rsc]

This proposal has been declined as infeasible.
— rsc for the proposal review group