x/tools/go/packages: support for fake GOROOTs

[xieyuschen]

This issue is the continuation of discussions in CL 615396.

When I tried to migrate x/tools/go/ssa/interp away from the deprecated loader package in its test cases in CL 615016, I attempted to create a fake GOROOT with some mocked std libs so packages.Load could load the faked std libs, which will be complemented by the go/ssa/interp interpreter later.

Even though @adonovan has provided a better approach to use overlay and custom GOROOT in x/tools/go/ssa/interp doesn't make sense, however, there isn't a conclusion regarding "whether we should support the usage of fake GOROOTs." Probably we can discuss here more about whether we should support/recommend the usage of fake GOROOTs.

For me, I have tried to add some libraries with my own toolchain under GOROOT because I want to use internal std package features but do not want to use the go linkname directive (which is locked down in 1.23 and higher versions, #67401).

The second point to discuss (related to the custom GOROOT) is whether it's desirable for users to add new packages under GOROOT and let the compiler treat them as standard libraries as well?

@timothy-king @matloob @adonovan

[gabyhelp]

Related Issues and Documentation

• x/tools/go/packages: loading should report error when use custom GOROOT #69606

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[timothy-king]

This is currently blocking[/making it difficult] to transition away from using x/tools/go/loader. In particular, x/tools/ssa/interp has tests which do a moderately sophisticated replacement of real std packages.

One possible avenue that was explored was to copy the GOTOOLDIR into the fake GOROOT. (For reference, GOTOOLDIR is derived from GOROOT and cannot be set.) It is expected that the job of someone faking a GOROOT needs to mimic some expected structure of a real GOROOT. But copying the binaries in this directory or symlinking the directory [unclear if these would be executed] seemed like a bit too much for a unit test.

To be explored is whether one can look at a real GOROOT directory, walk that for its file structure and use overlays to remove the undesired files/directories.

Another possibility is that the go list invoked by go/packages is told to derive GOTOOLDIR in a different way. Perhaps based on itself/the GOROOT values from within go/packages instead of the go/packages.Config.Env. Flag? Environment variable?

[gopherbot]

Change https://go.dev/cl/615016 mentions this issue: go/ssa/interp: migrate interp_test.go away from loader

[xieyuschen]

Hi @timothy-king I have tried your 2 ways mentioned above.

1. I tried to use GOTOOLCHAIN=go1.23.1+path and custom GOROOT but it doesn't work as the std lib and all its toolchains are treated as a bundle. packages.Load loads the std lib under go1.23.1 instead of my custom GOROOT. I think that's how the toolchain supposes to work. But do you think it's reasonable to add a new env var toggle to allow specify a custom stdlib place? Because in my case, usually I would like to copy and manipulate the standard libs in another place instead of modifying the original one.

2. overlay works well and I think this is the best solution for the interp migration. The CL 615016 is ready to review now:)