Build image with ko #2955

johngmyers · 2023-01-03T00:56:18Z

Issue

N/A

Description

Uses ko to build the LBC image.

Reduces build time significantly and increases build reliability. Previously builds would take 15 minutes to over 2 hours and would fail about 3 out of 4 times, both because of the go mod download in the Dockerfile.

Checklist

N/A Added tests that cover your change (if possible)
N/A Added/modified documentation as required (such as the README.md, or the docs directory)
Manually tested
Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

Backfilled missing tests for code in same general area 🎉
Refactored something and made the world a better place 🌟

codecov-commenter · 2023-01-03T01:02:21Z

Codecov Report

Base: 54.07% // Head: 54.01% // Decreases project coverage by -0.06% ⚠️

Coverage data is based on head (28b7f84) compared to base (a92e689).
Patch has no changes to coverable lines.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2955      +/-   ##
==========================================
- Coverage   54.07%   54.01%   -0.06%     
==========================================
  Files         144      144              
  Lines        8301     8312      +11     
==========================================
+ Hits         4489     4490       +1     
- Misses       3484     3494      +10     
  Partials      328      328

Impacted Files	Coverage Δ
pkg/config/controller_config.go	`15.38% <0.00%> (-2.48%)`	⬇️
pkg/ingress/model_builder.go	`63.90% <0.00%> (-0.32%)`	⬇️
pkg/service/model_builder.go	`88.29% <0.00%> (+0.12%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

johngmyers · 2023-01-03T05:26:33Z

/retest

olemarkus · 2023-01-10T09:20:55Z

I strongly support this! Building LBC is a bit prohibitive right now.

kishorj · 2023-01-11T01:47:51Z

Thanks for the contribution, I will look into this changes post the v2.4.6 patch release.

johngmyers · 2023-01-12T18:28:15Z

/retest

kishorj

I have two items I'm trying to work through at the moment:

ability to save to tarball, the --tarball flag to build command appears to be the way to go
Configuring a base image. The ko build appears to use a debian based image, I'm not certain if there is a fixed image per ko version, or it depends on the latest one available at the time of the build. I'm also interested to see how we can address the base image CVEs.

kishorj · 2023-01-20T00:41:43Z

hack/install-ko.sh

+
+if ! command -v ko &> /dev/null; then
+  cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1
+  go install github.com/google/ko@v0.12.0


It failed for me with the following error:

go: downloading github.com/go-openapi/analysis v0.21.3 ../../go/pkg/mod/github.com/go-openapi/validate@v0.22.0/spec.go:23:2: github.com/go-openapi/analysis@v0.21.3: verifying module: checksum mismatch downloaded: h1:vR88pR69D/jGh02vPbT4qoxiG+p9e5uT75JJG3O1JIU= sum.golang.org: h1:CPEa+B2oYCkb+lIKB4xP6Ork8Gvh0GNg9dm/twI3+QA=

I was able to install v0.11.2 and run the tests

You're apparently not the first to run across this. I found go-openapi/analysis#81

Filed ko-build/ko#940

Shall I back off to v0.11.2?

lets back off to v0.11.2, we can update in future

kishorj · 2023-01-20T00:42:39Z

helm/aws-load-balancer-controller/templates/deployment.yaml

@@ -153,8 +153,6 @@ spec:
          value: "{{ $value }}"
        {{- end }}
        {{- end }}
-        command:
-        - /controller


Is there a standard entrypoint for ko built container images?

Yes. See the last sentence of https://ko.build/get-started

Thanks. The entrypoint change will introduce backwards incompatibility with the existing manifests, we will release note it.

kishorj · 2023-01-20T00:54:58Z

scripts/ci_e2e_test.sh

-    n=$((n + 1))
-    sleep 2
-  done
+  make docker-push IMG=${CONTROLLER_IMAGE_NAME} IMG_PLATFORM=linux/amd64


would this not affect the multi-arch tests?

The old code specified --platform linux/amd64 so I preserved that behavior.

johngmyers · 2023-01-20T01:38:58Z

The base image is cgr.dev/chainguard/static and it takes what is the latest at the time of build.

Information on overriding the base image is at https://ko.build/configuration/#overriding-base-images

I doubt we'd be faster at fixing CVEs than Chainguard.

johngmyers · 2023-01-20T01:42:00Z

Did you want a make target to create a tarball? Should that be instead of or in addition to pushing to a registry?

kishorj · 2023-01-21T00:54:04Z

The base image is cgr.dev/chainguard/static and it takes what is the latest at the time of build.

Information on overriding the base image is at https://ko.build/configuration/#overriding-base-images

I doubt we'd be faster at fixing CVEs than Chainguard.

Lets use public.ecr.aws/eks-distro-build-tooling/eks-distro-minimal-base-nonroot:2022-07-27-1658910674.2 as the base image. Since the base image is internal, it will help us meet the SLAs for addressing the CVEs.

kishorj · 2023-01-21T00:56:43Z

Did you want a make target to create a tarball? Should that be instead of or in addition to pushing to a registry?

This is a separate requirement for publishing the official ECR images. We can revisit it when it is time to publish the v2.5.0 images.

kishorj · 2023-01-21T00:58:00Z

Please preserve the existing ldflags and the build flags to the extent possible/necessary, especially the version information.

johngmyers · 2023-01-21T21:35:49Z

Is there any particular reason to use CGO? It doesn't appear to be needed.

johngmyers · 2023-01-21T23:06:41Z

This configuration removes a whole swath of attack surface by disabling CGO.

kishorj · 2023-01-24T00:08:28Z

Is there any particular reason to use CGO? It doesn't appear to be needed.

We don't need, lets remove it

kishorj · 2023-01-24T00:10:14Z

/lgtm

k8s-ci-robot · 2023-01-24T00:10:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johngmyers, kishorj

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kishorj]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Build image with ko * Downgrade to ko v0.11.2 * Use eks-distro-minimal-base-nonroot as base image * Specify ko build options

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 3, 2023

k8s-ci-robot requested review from kishorj and M00nF1sh January 3, 2023 00:56

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 3, 2023

johngmyers force-pushed the ko branch 7 times, most recently from 7c1a05a to e65c2fa Compare January 3, 2023 03:52

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 12, 2023

Build image with ko

caa7140

johngmyers force-pushed the ko branch from e65c2fa to caa7140 Compare January 12, 2023 15:57

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 12, 2023

kishorj reviewed Jan 20, 2023

View reviewed changes

johngmyers mentioned this pull request Jan 20, 2023

Dependency checksum mismatch ko-build/ko#940

Closed

Downgrade to ko v0.11.2

0ad697e

Use eks-distro-minimal-base-nonroot as base image

824effa

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 21, 2023

Specify ko build options

28b7f84

johngmyers force-pushed the ko branch from 6979eca to 28b7f84 Compare January 21, 2023 22:57

kishorj added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Jan 24, 2023

kishorj approved these changes Jan 24, 2023

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 24, 2023

k8s-ci-robot assigned kishorj Jan 24, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 24, 2023

k8s-ci-robot merged commit bfa2860 into kubernetes-sigs:main Jan 24, 2023

johngmyers deleted the ko branch January 24, 2023 01:59

johngmyers mentioned this pull request Jan 24, 2023

Add johngmyers as reviewer #2999

Merged

jayanthvn mentioned this pull request Feb 8, 2023

Consider moving docker buildx -> ko aws/amazon-vpc-cni-k8s#2258

Closed

This was referenced Apr 14, 2023

🥳 aws-load-balancer-controller v2.5.0 Automated Release! 🥑 aws/eks-charts#932

Closed

🥳 aws-load-balancer-controller v2.5.0 Automated Release! 🥑 aws/eks-charts#933

Merged

johngmyers mentioned this pull request Aug 23, 2023

Add johngmyers as approver #3356

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build image with ko #2955

Build image with ko #2955

johngmyers commented Jan 3, 2023

codecov-commenter commented Jan 3, 2023 •

edited

Loading

johngmyers commented Jan 3, 2023

olemarkus commented Jan 10, 2023

kishorj commented Jan 11, 2023

johngmyers commented Jan 12, 2023

kishorj left a comment

kishorj Jan 20, 2023

johngmyers Jan 20, 2023

johngmyers Jan 20, 2023

johngmyers Jan 20, 2023

kishorj Jan 20, 2023

kishorj Jan 20, 2023

johngmyers Jan 20, 2023

kishorj Jan 20, 2023

kishorj Jan 20, 2023

johngmyers Jan 20, 2023

johngmyers commented Jan 20, 2023

johngmyers commented Jan 20, 2023

kishorj commented Jan 21, 2023 •

edited

Loading

kishorj commented Jan 21, 2023

kishorj commented Jan 21, 2023

johngmyers commented Jan 21, 2023

johngmyers commented Jan 21, 2023

kishorj commented Jan 24, 2023

kishorj commented Jan 24, 2023

k8s-ci-robot commented Jan 24, 2023

Build image with ko #2955

Build image with ko #2955

Conversation

johngmyers commented Jan 3, 2023

Issue

Description

Checklist

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

codecov-commenter commented Jan 3, 2023 • edited Loading

Codecov Report

johngmyers commented Jan 3, 2023

olemarkus commented Jan 10, 2023

kishorj commented Jan 11, 2023

johngmyers commented Jan 12, 2023

kishorj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johngmyers commented Jan 20, 2023

johngmyers commented Jan 20, 2023

kishorj commented Jan 21, 2023 • edited Loading

kishorj commented Jan 21, 2023

kishorj commented Jan 21, 2023

johngmyers commented Jan 21, 2023

johngmyers commented Jan 21, 2023

kishorj commented Jan 24, 2023

kishorj commented Jan 24, 2023

k8s-ci-robot commented Jan 24, 2023

codecov-commenter commented Jan 3, 2023 •

edited

Loading

kishorj commented Jan 21, 2023 •

edited

Loading