Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix go mod publishing #185

Merged
merged 5 commits into from
Jun 20, 2019
Merged

Conversation

nikhita
Copy link
Member

@nikhita nikhita commented Jun 18, 2019

This PR has the following changes:

  • Display the error that might occur while running a exec.Command.
  • Run go mod download before packaging up a dependency and before updating the require and replace directives. This will help in avoiding packaging up, when the pseudo version is the same but we don't have it in the local cache.
  • Fix the git command used for pseudo version calculation (go mod pseudo version uses wrong timestamp #186)
  • Publish all tags, not just alpha ones.
  • Add a new go binary gomod-zip for creating the zip file that would have the same hash as that of the zip file if it were downloaded by go mod download. The zip file created by gomod differs from the former:
    • there are some extra normalizations: not considering anything in vendor, making sure we don't have directories in the file archive. These can, however, still be taken care of by -x and -D options.
    • The zip file created by go mod download has the 00-00-1980 00:00 timestamp in the file header for all files in the zip archive. This is not a valid UNIX timestamp and cannot be set easily. This is, however, valid in MSDOS. The archive/zip package uses the MSDOS version so we create the zip file using this package.
      • change the initial zip file creation to use git archive: go uses this.
      • add basic normalizations as done by go internally.
      • use the archive/zip package to re-create the zip file from that created using git archive.

For the record: we had to do a GO111MODULE=on go mod clean -modcache first to clear the old zip files in our cache. Did this through kubectl exec, not using this PR.

For additional context, this slack thread has more details.

/hold
doesn't fix the error yet, but I figured I'll open a PR to get my latest changes in

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 18, 2019
@nikhita
Copy link
Member Author

nikhita commented Jun 18, 2019

/uncc @mfojtik
/cc @sttts

@k8s-ci-robot k8s-ci-robot requested review from sttts and removed request for mfojtik June 18, 2019 09:56
@@ -55,30 +55,46 @@ func updateGomodWithTaggedDependencies(tag string, depsRepo []string) (bool, err
rev := commit.String()
pseudoVersion := fmt.Sprintf("v0.0.0-%s-%s", commitTime.UTC().Format("20060102150405"), rev[:12])

downloadCommand := exec.Command("go", "mod", "download")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this always pick the right go version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not always, but it could if the pseudo version doesn't change. Added a comment.

cmd/sync-tags/gomod.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 19, 2019
@nikhita
Copy link
Member Author

nikhita commented Jun 19, 2019

/cc @liggitt

@nikhita nikhita changed the title Run go mod download before packaging a dependency Fix go mod publishing Jun 19, 2019
Dockerfile Outdated Show resolved Hide resolved
cmd/gomod-zip/zip.go Outdated Show resolved Hide resolved
The zip file created by go mod download has extra normalization over
the zip file created by git archive. The normalization process is done below.

While the normalization can also be achieved via a simple zip command, the zip file
created by go mod download has the `00-00-1980 00:00` timestamp in the file header
for all files in the zip archive. This is not a valid UNIX timestamp and cannot be
set easily. This is, however, valid in MSDOS. The `archive/zip` package uses the
MSDOS version so we create the zip file using this package.
@@ -193,11 +193,6 @@ func main() {
continue
}

// temporarily publish only alpha tags
if !strings.Contains(bName, "alpha") {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the alpha tags restriction. PTAL.

@sttts
Copy link
Contributor

sttts commented Jun 20, 2019

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 20, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nikhita, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 20, 2019
@nikhita
Copy link
Member Author

nikhita commented Jun 20, 2019

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 20, 2019
@k8s-ci-robot k8s-ci-robot merged commit 0fec9b1 into kubernetes:master Jun 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants