Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vendor: add fake imports for glide's benefit #12166

Merged
merged 1 commit into from
Dec 8, 2016
Merged

Conversation

dt
Copy link
Member

@dt dt commented Dec 8, 2016

We've repeatedly encountered issues arising from the fact that entries in glide.yaml are not treated as the, or even as additional, roots in the calculation of the transitive dependency closure.

Previously in #11797, we attempted to clarify the situation by pruning all derivable specs from glide.yaml, so that it would be clear where the dependency roots were indeed coming from.

We however left vendored tools, and their transitive dependencies, there, as they were not derivable from any imports (e.g. some are not importable due to package main).

As we've already found though, simply adding a tool to glide.yaml, or even glide get'ing them, doesn't work: Masterminds/glide#690.

Thus, to reliably vendor tools and their dependencies, we need to introduce artifical imports of them. We get around the package main issue with by hiding the imports in a file that is build-tagged +glide so we never attempt to build it.


The glide up used to test this also picked up the following uninteresting changes:

Note: The etcd bump does include a raft change, but it is test only


This change is Reviewable

@sdboyer
Copy link

sdboyer commented Dec 8, 2016

yep. till glide adds explicit requires support, a la sdboyer/gps#42, this is the easiest workaround. 😞

@tamird
Copy link
Contributor

tamird commented Dec 8, 2016

:lgtm:


Reviewed 5 of 5 files at r1.
Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


build/README.md, line 45 at r1 (raw file):

The [docs](https://github.com/Masterminds/glide) have detailed instructions, but in brief:
* run `./scripts/glide.sh` in `cockroachdb/cockroach`.
* add new dependencies with `./scripts/glide.sh get -s github.com/my/dependency`

seems like we don't advise this at all now


build/tool_imports.go, line 31 at r1 (raw file):

import (
	_ "github.com/client9/misspell/cmd/misspell"

change checkdeps to look at this file? ...or what's your plan for checkdeps?


Comments from Reviewable

@dt
Copy link
Member Author

dt commented Dec 8, 2016

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


build/README.md, line 45 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

seems like we don't advise this at all now

Hm. You need something to fetch the dep, so you can start developing / importing it, and I don't have a better suggestion for now?


build/tool_imports.go, line 31 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

change checkdeps to look at this file? ...or what's your plan for checkdeps?

buildutil will choke on the package main imports AFAIK, so checkdeps can't use this file currently.

That said, this shouldn't makecheckdeps any less correct, and should make glide up more correct, so it still seems to make sense to do.


Comments from Reviewable

@tamird
Copy link
Contributor

tamird commented Dec 8, 2016

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


build/README.md, line 45 at r1 (raw file):

Previously, dt (David Taylor) wrote…

Hm. You need something to fetch the dep, so you can start developing / importing it, and I don't have a better suggestion for now?

would glide.sh install do it?


build/tool_imports.go, line 31 at r1 (raw file):

Previously, dt (David Taylor) wrote…

buildutil will choke on the package main imports AFAIK, so checkdeps can't use this file currently.

That said, this shouldn't makecheckdeps any less correct, and should make glide up more correct, so it still seems to make sense to do.

Wait, what does buildutil have to do with anything? checkdeps is implemented using go list, and this list of imports effectively duplicates the GLOCKFILE.


Comments from Reviewable

@dt
Copy link
Member Author

dt commented Dec 8, 2016

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


build/README.md, line 45 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

would glide.sh install do it?

I don't think so -- install should just fetch whatever is in the lock file, right? up would resolve and fetch things if you already had imports of them, but that's easier if you have them locally to import.


build/tool_imports.go, line 31 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

Wait, what does buildutil have to do with anything? checkdeps is implemented using go list, and this list of imports effectively duplicates the GLOCKFILE.

How are you suggesting we use go list? AFAIK, it needs to be able to load a package to enumerate it.


Comments from Reviewable

@tamird
Copy link
Contributor

tamird commented Dec 8, 2016

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


build/README.md, line 45 at r1 (raw file):

install should just fetch whatever is in the lock file, right?

I don't know, this is what I'm asking you to test.


build/tool_imports.go, line 31 at r1 (raw file):

Previously, dt (David Taylor) wrote…

How are you suggesting we use go list? AFAIK, it needs to be able to load a package to enumerate it.

I have no idea what you're referring to here - the current code in checkdeps already does a go list which includes these package mains.


Comments from Reviewable

@dt dt closed this Dec 8, 2016
@tamird
Copy link
Contributor

tamird commented Dec 8, 2016

I'm going to add the changes I suggested here to avoid the back-and-forth.

@tamird tamird reopened this Dec 8, 2016
@tamird tamird force-pushed the glide branch 4 times, most recently from 59101b2 to fd917c7 Compare December 8, 2016 17:08
We've repeatedly encountered issues arising from the fact that entries
in `glide.yaml` are not treated as the, or even as additional, roots in
the calulation of the transitive dependency closure.

Previously in cockroachdb#11797, we attempted to clarify the situation by pruning
all derivable specs from `glide.yaml`, so that it would be clear where
the dependency roots were indeed coming from.

We however left vendored tools, and their transitive dependencies,
there, as they were not derivable from any imports (e.g. some are not
importable due to `package main`).

As we've already found though, simply adding a tool to `glide.yaml`, or
even `glide get`'ing them, doesn't work.

Thus, to reliably vendor tools and their dependencies, we need to
introduce artifical imports of them. We get around the `package main`
issue with by hiding the imports in a file that is build-tagged `+glide`
so we never attempt to build it.

---

The `glide up` used to test this picked up a test-only raft change,
etcd-io/etcd#6935, and:

- golang/tools@ae1141f...3d92dd6
- cockroachdb/cockroach-go@2c874f1...140a8c5
- etcd-io/etcd@cfd10b4...b713113
- distribution/distribution@67095fb...844b928
- google/go-github@171a931...43e7458
- opencontainers/runc@5974b4c...34f23cb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants