Go PRs require extra tidying #6795

tbpg · 2020-07-06T17:39:55Z

Which Renovate are you using?

WhiteSource Renovate App (the forking version)

Which platform are you using?

GitHub.com

Have you checked the logs? Don't forget to include them if relevant

No.

What would you like to do?

Have go mod tidy run when making dependency updates. For example, GoogleCloudPlatform/golang-samples#1556 required me updating the PR manually to run go mod tidy on all modules.

I do have postUpdateOptions.gomodTidy configured, but maybe it's not configured correctly? https://github.com/GoogleCloudPlatform/golang-samples/blob/2c581f8b444a3b4cf790d4350ea212d1d9c0f8d0/.github/renovate.json#L7-L9

The text was updated successfully, but these errors were encountered:

rarkins · 2020-07-06T18:05:10Z

Here's the command that was run: docker run --rm --name=renovate_go --label=renovate_child -v \"/mnt/renovate/gh/GoogleCloudPlatform/golang-samples\":\"/mnt/renovate/gh/GoogleCloudPlatform/golang-samples\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -v \"/tmp/renovate-cache/others/go\":\"/tmp/renovate-cache/others/go\" -e GOPATH -e CGO_ENABLED -w \"/mnt/renovate/gh/GoogleCloudPlatform/golang-samples/functions/firebase/upper\" renovate/go:latest bash -l -c \"git config --global url.\\\"https://x-access-token:**redacted**@github.com/\\\".insteadOf \\\"https://github.com/\\\" && go get -d ./...\"

This confirms that go mod tidy isn't being run. I'll dig into it more now.

rarkins · 2020-07-06T19:07:40Z

go mod tidy was run, as configured
Strangely, go.sum wasn't updated at all, despite go get and go mod tidy being run
This PR is old/closed so I can't really do much now

Best if you raise a new issue here or @ me in the repo the next time you have a PR that looks wrong, and before you close it.

tbpg · 2020-07-06T20:06:05Z

Thank you for the speedy response and for looking into it!

Got it. I'll close this and let you know the next time it happens (probably next Monday, as that's when updates are configured to happen).

tbpg · 2020-07-20T19:11:44Z

Reopening to track the current PR with this issue: GoogleCloudPlatform/golang-samples#1573

Thank you for the help with this! Let me know if there are any adjustments or anything I should try.

rarkins · 2020-07-21T05:26:30Z

Noting a few things:

This repo doesn't just run go mod tidy, it also runs goimports -w. I'm not sure yet what that part does or if it matters.

Here's how it's run:

Another point: the repo is running Go 1.14 on all go.mod files while the go.mod files mostly specify go1.11 or go1.13, which is what we run. Go has changed quite a lot around modules in 2 years so there can definitely be a difference between go1.11 and go1.14 tidying, but that may not be the cause.

Most likely solution: run go mod tidy again.

I'm also wondering: why should we run go get at all - maybe go mod tidy on its own does what we need?

tbpg · 2020-07-26T23:34:52Z

Thank you for investigating!

goimports -w . only modifies *.go files -- not go.sum. The go.sum changes only come from go mod tidy.

Go 1.14 vs other versions could cause inconsistencies in go.sum files. Would it be possible to upgrade the Go version used? Go 1.15 will be out in ~1 month, too.

Yes, you should run go get. go get can also modify the version of other dependencies to meet new requirements. So, it's the best way to change the version of any dependency. So, no, go mod tidy doesn't do everything needed.

rarkins · 2020-07-27T05:09:02Z

Would it be possible to upgrade the Go version used?

Renovate follows whatever Go version that is specified in the go.mod files. For example: https://github.com/GoogleCloudPlatform/golang-samples/blob/5520efc715afa2e11ed8f86d0e851a899372f716/go.mod#L3

I'm thinking though that Renovate should ideally have a feature to "upgrade" that value anyway.

Thanks for the clarification re go get and go mod tidy.

tbpg · 2020-07-28T12:34:57Z

Renovate follows whatever Go version that is specified in the go.mod files.

Gotcha. I missed that. Often, the development version and the compatibility version are different. Is there any way we could specify the version of Go to use as part of the config?

I'm not sure "upgrading" it is a good idea. That value indicates the minimum Go version needed to use the module. So, it's reasonable for it to stay the same for a long time.

rarkins · 2020-07-28T12:54:08Z

I wasn't aware that it means the minimum version - thank you for clarifying. In such case we may be better using the latest version of Go if that is also compatible.

We also plan to allow configurable versions in the future but it's not supported yet

tbpg · 2020-07-28T14:43:08Z

Yes, I think latest would make the most sense.

Sounds good with configuring. If it always used the latest, I don't think we'd need the ability to configure.

rarkins · 2020-08-04T11:21:00Z

Waiting on #6909

tbpg · 2020-08-17T19:53:57Z

Friendly nudge -- looks like #6909 is fixed.

rarkins · 2020-08-17T19:56:32Z

Yes, I guess it's as simple as adding one more go mod tidy to our command sequence to see if that fixes it.

renovate-release · 2020-08-17T20:16:53Z

🎉 This issue has been resolved in version 22.19.1 🎉

The release is available on:

GitHub release
22.19.1

Your semantic-release bot 📦🚀

rarkins · 2020-08-17T20:18:14Z

I'll put this live tomorrow

tbpg · 2020-08-18T11:26:59Z

Thank you!

rarkins · 2020-08-18T11:45:41Z

Can you tick the rebase checkbox to try your existing PR again?

tbpg · 2020-08-18T14:19:15Z

Unfortunately, it looks like they still aren't tidy. Perhaps the new release wasn't fully rolled out yet?

rarkins · 2020-08-18T14:34:51Z

It's rolled out. Here's the command that was run: docker run --rm --name=renovate_go --label=renovate_child -v "/mnt/renovate/gh/GoogleCloudPlatform/golang-samples":"/mnt/renovate/gh/GoogleCloudPlatform/golang-samples" -v "/tmp/renovate-cache":"/tmp/renovate-cache" -v "/tmp/renovate-cache/others/go":"/tmp/renovate-cache/others/go" -e GOPATH -e CGO_ENABLED -w "/mnt/renovate/gh/GoogleCloudPlatform/golang-samples/run/logging-manual" renovate/go:1.12.17 bash -l -c "git config --global url.\"https://**redacted**@github.com/\".insteadOf \"https://github.com/\" && go get -d ./... && go mod tidy && go mod tidy"","msg":"Executing command","time":"2020-08-18T13:30:55.764Z"} {"level":20,"branch":"renovate/all","cmd":"docker run --rm --name=renovate_go --label=renovate_child -v "/mnt/renovate/gh/GoogleCloudPlatform/golang-samples":"/mnt/renovate/gh/GoogleCloudPlatform/golang-samples" -v "/tmp/renovate-cache":"/tmp/renovate-cache" -v "/tmp/renovate-cache/others/go":"/tmp/renovate-cache/others/go" -e GOPATH -e CGO_ENABLED -w "/mnt/renovate/gh/GoogleCloudPlatform/golang-samples/run/logging-manual" renovate/go:1.12.17 bash -l -c "git config --global url.\"https://**redacted**@github.com/\".insteadOf \"https://github.com/\" && go get -d ./... && go mod tidy && go mod tidy"

rarkins · 2020-08-18T14:36:35Z

When I check out that PR and run go mod tidy I get this:

diff --git a/go.mod b/go.mod
index e60a1d4..0d6a58f 100644
--- a/go.mod
+++ b/go.mod
@@ -22,7 +22,6 @@ require (
        github.com/fluent/fluent-logger-golang v1.5.0
        github.com/go-sql-driver/mysql v1.5.0
        github.com/gofrs/uuid v3.3.0+incompatible
-       github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
        github.com/golang/protobuf v1.4.2
        github.com/gomodule/redigo v2.0.0+incompatible
        github.com/google/go-cmp v0.5.1
diff --git a/go.sum b/go.sum
index 7ea2449..8566c1c 100644
--- a/go.sum
+++ b/go.sum
@@ -203,6 +203,7 @@ github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORN
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/lib/pq v1.8.0 h1:9xohqzkUwzR4Ga4ivdTcawVS89YSDVxXMa3xJX3cGzg=
 github.com/lib/pq v1.8.0/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
 github.com/linkedin/goavro/v2 v2.9.8 h1:jN50elxBsGBDGVDEKqUlDuU1cFwJ11K/yrJCBMe/7Wg=
 github.com/linkedin/goavro/v2 v2.9.8/go.mod h1:UgQUb2N/pmueQYH9bfqFioWxzYCZXSfF8Jw03O5sjqA=

Surely we don't need another tidy?

rarkins · 2020-08-18T14:38:11Z

Something else that is perhaps contributing is that we're not actually running the go commands once, we're running them 35 times for the 35 go.mod files. Maybe they interfere with each other somehow?

tbpg · 2020-08-18T16:05:51Z

That command looks OK to me. I was concerned about the working directory because go commands work on the module in the current directory, but that seems correct with the -w flag to docker run. I don't think another go mod tidy would make a difference... I would think one would do the trick.

Each go.mod should be completely independent from the others. When you first add a "nested" go.mod it may lead to changes in the go.mod in the higher directory because the higher one doesn't have to track as many deps. But, I don't think that's the issue here.

I tried running the docker command locally (modifying it to mount pwd instead of /mnt/renovate/gh/GoogleCloudPlatform/golang-samples), but I ran into a volume write permission error.

I don't think we need the go get -d ./.... That doesn't affect go mod tidy (tidy will download what it needs). But, again, I don't think that would cause the problem.

sluongng · 2020-08-27T11:06:03Z

Im also facing a similar issue

ERROR: Error updating branch: Command failed: go mod tidy
       go: k8s.io/klog/v2/v2@v2.3.0: reading https://proxy.golang.org/k8s.io/klog/v2/v2/@v/v2.3.0.mod: 410 Gone
        server response: not found: k8s.io/klog/v2/v2@v2.3.0: invalid version: unknown revision v2/v2.3.0
        (repository=redacted/my-repo, branch=renovate/major-2-k8s-modules)
       "err": {
         "killed": false,
         "code": 1,
         "signal": null,
         "cmd": "go mod tidy",
         "stdout": "",
         "stderr": "go: k8s.io/klog/v2/v2@v2.3.0: reading https://**redacted**@v/v2.3.0.mod: 410 Gone\n\tserver response: not found: k8s.io/klog/v2/v2@v2.3.0: invalid version: unknown revision v2/v2.3.0\n",
         "message": "Command failed: go mod tidy\ngo: k8s.io/klog/v2/v2@v2.3.0: reading https://**redacted**@v/v2.3.0.mod: 410 Gone\n\tserver response: not found: k8s.io/klog/v2/v2@v2.3.0: invalid version: unknown revision v2/v2.3.0\n",
         "stack": "Error: Command failed: go mod tidy\ngo: k8s.io/klog/v2/v2@v2.3.0: reading https://**redacted**@v/v2.3.0.mod: 410 Gone\n\tserver response: not found: k8s.io/klog/v2/v2@v2.3.0: invalid version: unknown revision v2/v2.3.0\n\
n    at ChildProcess.exithandler (child_process.js:303:12)\n    at ChildProcess.emit (events.js:315:20)\n    at ChildProcess.EventEmitter.emit (domain.js:483:12)\n    at maybeClose (internal/child_process.js:1021:16)\n    at Process.ChildPr
ocess._handle.onexit (internal/child_process.js:286:5)"
       }

Running go mod tidy inside /tmp/renovate/repo/.../my-repo works fine.

$ renovate --version
23.5.0

rarkins · 2020-08-27T11:24:17Z

Not the same issue, please see if you can reproduce it in a public repo and raise a separate bug report.

sluongng · 2020-08-27T12:00:25Z

@rarkins thanks, raised #7121

Commands: go get -u -d -t ./... go mod tidy Renovate still seems to have trouble with Go modules: renovatebot/renovate#6795

HonkingGoose · 2020-10-27T18:25:50Z

Closing this issue as it's a duplicate of #6213.

tbpg closed this as completed Jul 6, 2020

tbpg reopened this Jul 20, 2020

rarkins changed the title ~~Go mod tidy is not run even though it's configured~~ Go PRs require extra tidying Jul 21, 2020

rarkins transferred this issue from renovatebot/config-help Jul 21, 2020

rarkins added manager:gomod Go Modules type:bug Bug fix of existing functionality priority-2-high Bugs impacting wide number of users or very important features labels Jul 21, 2020

rarkins closed this as completed in 03b3f09 Aug 17, 2020

renovate-release added the released label Aug 17, 2020

rarkins reopened this Aug 18, 2020

rarkins removed the released label Aug 18, 2020

Hexcles added a commit to web-platform-tests/wpt.fyi that referenced this issue Sep 14, 2020

Upgrade Go dependencies

a3ad04f

Commands: go get -u -d -t ./... go mod tidy Renovate still seems to have trouble with Go modules: renovatebot/renovate#6795

Hexcles mentioned this issue Sep 14, 2020

Upgrade Go dependencies web-platform-tests/wpt.fyi#2152

Merged

Hexcles added a commit to web-platform-tests/wpt.fyi that referenced this issue Sep 14, 2020

Upgrade Go dependencies (#2152)

35b4e74

Commands: go get -u -d -t ./... go mod tidy Renovate still seems to have trouble with Go modules: renovatebot/renovate#6795

HonkingGoose mentioned this issue Oct 27, 2020

Golang update PRs sometimes need more tidying #6213

Closed

HonkingGoose closed this as completed Oct 27, 2020

github-actions bot locked as resolved and limited conversation to collaborators Dec 15, 2020

HonkingGoose added the duplicate This issue is closed as a duplicate of another issue label Jan 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Go PRs require extra tidying #6795

Go PRs require extra tidying #6795

tbpg commented Jul 6, 2020

rarkins commented Jul 6, 2020

rarkins commented Jul 6, 2020

tbpg commented Jul 6, 2020

tbpg commented Jul 20, 2020

rarkins commented Jul 21, 2020

tbpg commented Jul 26, 2020

rarkins commented Jul 27, 2020

tbpg commented Jul 28, 2020

rarkins commented Jul 28, 2020

tbpg commented Jul 28, 2020

rarkins commented Aug 4, 2020

tbpg commented Aug 17, 2020

rarkins commented Aug 17, 2020

renovate-release commented Aug 17, 2020

rarkins commented Aug 17, 2020

tbpg commented Aug 18, 2020

rarkins commented Aug 18, 2020

tbpg commented Aug 18, 2020

rarkins commented Aug 18, 2020

rarkins commented Aug 18, 2020

rarkins commented Aug 18, 2020

tbpg commented Aug 18, 2020

sluongng commented Aug 27, 2020

rarkins commented Aug 27, 2020

sluongng commented Aug 27, 2020

HonkingGoose commented Oct 27, 2020

Go PRs require extra tidying #6795

Go PRs require extra tidying #6795

Comments

tbpg commented Jul 6, 2020

Which Renovate are you using?

Which platform are you using?

Have you checked the logs? Don't forget to include them if relevant

What would you like to do?

rarkins commented Jul 6, 2020

rarkins commented Jul 6, 2020

tbpg commented Jul 6, 2020

tbpg commented Jul 20, 2020

rarkins commented Jul 21, 2020

tbpg commented Jul 26, 2020

rarkins commented Jul 27, 2020

tbpg commented Jul 28, 2020

rarkins commented Jul 28, 2020

tbpg commented Jul 28, 2020

rarkins commented Aug 4, 2020

tbpg commented Aug 17, 2020

rarkins commented Aug 17, 2020

renovate-release commented Aug 17, 2020

rarkins commented Aug 17, 2020

tbpg commented Aug 18, 2020

rarkins commented Aug 18, 2020

tbpg commented Aug 18, 2020

rarkins commented Aug 18, 2020

rarkins commented Aug 18, 2020

rarkins commented Aug 18, 2020

tbpg commented Aug 18, 2020

sluongng commented Aug 27, 2020

rarkins commented Aug 27, 2020

sluongng commented Aug 27, 2020

HonkingGoose commented Oct 27, 2020