Go: awkward UX with ./pants check and third-party packages

[Eric-Arellano]

Many third-party packages end up never being used in a Go project. Pants pessimistically uses target generation to create a go_third_party_package target for every package belonging to a third-party module, even if it will never be used:

pants/src/python/pants/backend/go/util_rules/third_party_pkg_test.py

Lines 284 to 299 in f3ffa37

	assert_module(
	"github.com/google/go-cmp",
	"v0.5.6",
	[
	"github.com/google/go-cmp/cmp",
	"github.com/google/go-cmp/cmp/cmpopts",
	"github.com/google/go-cmp/cmp/internal/diff",
	"github.com/google/go-cmp/cmp/internal/flags",
	"github.com/google/go-cmp/cmp/internal/function",
	"github.com/google/go-cmp/cmp/internal/testprotos",
	"github.com/google/go-cmp/cmp/internal/teststructs",
	"github.com/google/go-cmp/cmp/internal/teststructs/foo1",
	"github.com/google/go-cmp/cmp/internal/teststructs/foo2",
	"github.com/google/go-cmp/cmp/internal/value",
	],
	)

This is necessary for dependency inference to work: we need to have a mapping of every possible package used to map back imports.

--

However, this means that ./pants check :: will end up compiling substantially more packages than are actually needed. Not only is this wasted performance, but some packages might not compile such as from us not yet supporting cgo files, c files, etc.

--

It is not acceptable to require the user first do ./pants filter --target-type=go_first_party_package :: | xargs ./pants check.

Instead, I think the solution is to not hook up go_third_party_package to ./pants check. The only way to compile a third-party package is because it's a dependency of a first-party package.

[Eric-Arellano]

cc @tdyas @stuhood @patricklaw . What do you think of the proposed solution? Do you have any other ideas?

[tdyas]

Instead, I think the solution is to not hook up go_third_party_package to ./pants check. The only way to compile a third-party package is because it's a dependency of a first-party package.

I agree with this solution. We are using targets as an implementation detail to make it easier to build third party code, but the user does not expect :: to build that third-party code. At least for me, I presume Pants will only build my first-party code in that case (and whatever is necessary to build that first-party code).

[tdyas]

Pants pessimistically uses target generation to create a go_third_party_package target for every package belonging to a third-party module, even if it will never be used:

Probably save this as a motivating use case for lazy target generation some time in the future.

[Eric-Arellano]

Closing as answered.