Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Wolfi images #47182

Merged
merged 58 commits into from
Apr 24, 2023
Merged

Build Wolfi images #47182

merged 58 commits into from
Apr 24, 2023

Conversation

willdollman
Copy link
Contributor

@willdollman willdollman commented Jan 31, 2023

We have packages, we have base images, so let's build some actual runnable images!

This PR is my main branch for building Wolfi-based versions of our first-party docker images.

Progress tracked in https://github.com/sourcegraph/security/issues/447

Notes for reviewers

Reviewing does not need to be comprehensive (so don't worry about inspecting every single build script or dockerfile!), as I'll be doing a full verification of each image prior to merging. Helpful feedback would be on any patterns you think could be improved.

One area that could use improvement is duplication in the wolfi-images/ directory - we could use some YAML templating here.

For each image there's a:

  • build-wolfi.sh alongside the standard build.sh script. Changes are minimal, and in most cases just change docker build -f Dockerfile [...] to Dockerfile.wolfi. In a few cases they remove unneeded build scripts, such as install-ctags.sh
  • Dockerfile.wolfi which contains the bulk of the changes

The apko build definitions are in the wolfi-images/ directory. Simple images without additional package requirements rely on wolfi-images/sourcegraph.yaml. Images that require packaged deps have their own .yaml file.

Test plan

  • Green CI
  • These changes will not be made live until each image undergoes full validation

@willdollman willdollman marked this pull request as ready for review April 20, 2023 13:01
@sourcegraph-bot
Copy link
Contributor

sourcegraph-bot commented Apr 20, 2023

Codenotify: Notifying subscribers in CODENOTIFY files for diff 2228b36...b4e44bb.

Notify File(s)
@bobheadxi docker-images/cadvisor/Dockerfile.wolfi
docker-images/cadvisor/build-wolfi.sh
docker-images/prometheus/Dockerfile.wolfi
docker-images/prometheus/build-wolfi.sh
enterprise/dev/ci/internal/ci/pipeline.go
enterprise/dev/ci/internal/ci/wolfi_operations.go
enterprise/dev/ci/scripts/wolfi/build-base-image.sh
@efritz cmd/worker/Dockerfile.wolfi
cmd/worker/build-wolfi.sh
cmd/worker/build.sh
enterprise/cmd/migrator/build-wolfi.sh
enterprise/cmd/precise-code-intel-worker/Dockerfile.wolfi
enterprise/cmd/precise-code-intel-worker/build-wolfi.sh
enterprise/cmd/worker/Dockerfile.wolfi
enterprise/cmd/worker/build-wolfi.sh
@indradhanush cmd/repo-updater/Dockerfile.wolfi
cmd/repo-updater/build-wolfi.sh
cmd/repo-updater/build.sh
enterprise/cmd/repo-updater/build-wolfi.sh
@keegancsmith cmd/searcher/Dockerfile.wolfi
cmd/searcher/build-wolfi.sh
cmd/searcher/build.sh
cmd/symbols/Dockerfile.wolfi
cmd/symbols/build-wolfi.sh
cmd/symbols/go-build-wolfi.sh
@sashaostrikov cmd/repo-updater/Dockerfile.wolfi
cmd/repo-updater/build-wolfi.sh
cmd/repo-updater/build.sh
@sourcegraph/delivery docker-images/postgres-12-alpine/Dockerfile.wolfi
docker-images/postgres-12-alpine/build-wolfi.sh
docker-images/postgres-12-alpine/rootfs/postgres-wolfi.sh

@sourcegraph-bot
Copy link
Contributor

sourcegraph-bot commented Apr 20, 2023

Codenotify: Notifying subscribers in OWNERS files for diff 2228b36...b4e44bb.

Notify File(s)
@sourcegraph/dev-experience enterprise/dev/ci/internal/ci/pipeline.go
enterprise/dev/ci/internal/ci/wolfi_operations.go
enterprise/dev/ci/scripts/wolfi/build-base-image.sh

@jhchabran jhchabran requested a review from davejrt April 20, 2023 13:24
Copy link
Member

@jhchabran jhchabran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great to finally see this land 😊

I've noticed that in some cases, the bazel building process is not ported. I think that's fine, because we'll need to update those things anyway because wolfi isn't running musl, so we got to tweak a few of them anyway.

And we haven't fully landed those either, so we can update them afterward, I don't think it would be wise to delay landing and testing this because of it.

export CGO_ENABLED=0

echo "--- go build"
pkg="github.com/sourcegraph/sourcegraph/cmd/frontend"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You didn't port the bazel build in that one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just been tracking what the build.sh script on main does, and for this image it doesn't use bazel - https://github.com/sourcegraph/sourcegraph/blob/main/cmd/frontend/build.sh

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's because in our branch, it's done in a build-bazel.sh

@willdollman willdollman enabled auto-merge (squash) April 24, 2023 12:35
@willdollman willdollman merged commit 3db318d into main Apr 24, 2023
@willdollman willdollman deleted the wolfi/build-wolfi branch April 24, 2023 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants