From 0135eb19d4569364fa26c71da0f0c3613f62021c Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Wed, 30 Mar 2022 13:18:27 +0300 Subject: [PATCH 1/5] Add proposal for adding Helm OCI support to Flux Source API Signed-off-by: Stefan Prodan --- rfcs/helm-oci/README.md | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 rfcs/helm-oci/README.md diff --git a/rfcs/helm-oci/README.md b/rfcs/helm-oci/README.md new file mode 100644 index 0000000000..a57d3db1e8 --- /dev/null +++ b/rfcs/helm-oci/README.md @@ -0,0 +1,110 @@ +# RFC-xxxx Flux OCI support for Helm + +**Status:** provisional + +**Creation date:** 2022-03-30 + +**Last update:** 2022-03-30 + +## Summary + +Given that Helm v3.8 supports [OCI](https://helm.sh/docs/topics/registries/) for package distribution, +we should extend the Flux Source API to allow fetching Helm charts from container registries. + +## Motivation + +Helm OCI support is one of the most requested feature in Flux +as seen on this [issue](https://github.com/fluxcd/source-controller/issues/124). + +### Goals + +- Add support for fetching Helm charts stored as OCI artifacts with minimal API changes to Flux. +- Make it easy for users to switch from HTTP/S Helm repositories to OCI repositories. + +### Non-Goals + +- Introduce a new API kind for referencing charts stored as OCI artifacts. + +## Proposal + +Introduce an optional field called `type` to the `HelmRepository` spec. + +When not specified, the `spec.type` field defaults to `Default` which preserve the current `HelmRepository` API behaviour. + +When the `spec.type` field is set to `OCI`, the `spec.url` field must be prefixed with `oci://` (to follow the Helm conventions). +For `oci://` URLs, source-controller will use the Helm SDK and the `oras` library to connect to the OCI remote storage. +For authentication, the controller will use Kubernetes secrets of `kubernetes.io/dockerconfigjson` type. + +### User Stories + +#### Story 1 + +> As a developer I want to use Flux `HelmReleases` that refer to Helm charts stored +> as OCI artifacts in GitHub Container Registry. + +First create a secret using a GitHub token that allows access to GHCR: + +```sh +kubectl create secret docker-registry ghcr-charts \ + --docker-server=ghcr.io \ + --docker-username=$GITHUB_USER \ + --docker-password=$GITHUB_TOKEN +``` + +Then define a `HelmRepository` of type `OCI` and reference the `dockerconfig` secret: + +```yaml +apiVersion: source.toolkit.fluxcd.io/v1beta2 +kind: HelmRepository +metadata: + name: ghcr-charts + namespace: default +spec: + type: OCI + url: oci://ghcr.io/my-org/charts/ + secretRef: + name: ghcr-charts +``` + +And finally in Flux `HelmReleases`, refer to the ghcr-charts `HelmRepository`: + +```yaml +apiVersion: helm.toolkit.fluxcd.io/v2beta1 +kind: HelmRelease +metadata: + name: podinfo + namespace: default +spec: + interval: 60m + chart: + spec: + chart: my-app + version: '1.0.x' + sourceRef: + kind: HelmRepository + name: ghcr-charts + interval: 1m # check for new OCI artifacts every minute +``` + +### Alternatives + +We could use introduce a new API type e.g. `HelmOCIRepository`. That is considered unpractical, +as there is no benefit for users in having a dedicated kind instead of a `type` filed in the current +`HelmRepository` API. Adding a `type` filed to the spec follows the Flux Bucket API design, where +the same Kind servers different implementations: AWS S3 vs Azure Blob vs Google Storage. + +## Design Details + +In source-controller we'll add a new predicate for indexing `HelmRepositories` based on the `spec.type` field. + +When the `spec.type` field is set to `OCI`, the `HelmRepositoryReconciler` +will set the `HelmRepository` Ready status to `False` if the URL is not prefixed with `oci://`, +otherwise the Ready status will be set to `True`. + +The current `HelmChartReconciler` will use the `HelmRepositories` with `type: Default`. +For `type: OCI` we'll introduce a new reconciler `HelmChartOCIReconciler` that uses `oras` to download charts +and their dependencies. + +### Enabling the feature + +The feature is enabled by default. From 606078c1b3890a2b9842f846bf6df710b7cd4298 Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Wed, 30 Mar 2022 14:31:25 +0300 Subject: [PATCH 2/5] Add chart update automation to Git Signed-off-by: Stefan Prodan --- rfcs/helm-oci/README.md | 57 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 53 insertions(+), 4 deletions(-) diff --git a/rfcs/helm-oci/README.md b/rfcs/helm-oci/README.md index a57d3db1e8..6d9fc87c2c 100644 --- a/rfcs/helm-oci/README.md +++ b/rfcs/helm-oci/README.md @@ -16,6 +16,9 @@ we should extend the Flux Source API to allow fetching Helm charts from containe Helm OCI support is one of the most requested feature in Flux as seen on this [issue](https://github.com/fluxcd/source-controller/issues/124). +With OCI support, Flux users can automate chart updates to Git in the same way +they do today for container images. + ### Goals - Add support for fetching Helm charts stored as OCI artifacts with minimal API changes to Flux. @@ -86,12 +89,58 @@ spec: interval: 1m # check for new OCI artifacts every minute ``` +#### Story 2 + +> As a platform admin I want to automate Helm chart updates based on a semver ranges. +> When a new patch version is available in the container registry, I want Flux to open a PR +> with the version set in the `HelmRelease` manifests. + +Given that charts are stored in container registries, you can use Flux image automation +and patch the chart version in Git, in the same way Flux works for updating container image tags. + +Define an image policy using semver: + +```yaml +apiVersion: image.toolkit.fluxcd.io/v1beta1 +kind: ImagePolicy +metadata: + name: my-app + namespace: default +spec: + imageRepositoryRef: + name: my-app + policy: + semver: + range: 1.0.x +``` + +Then add the policy marker to the `HelmRelease` manifests in Git: + +```yaml +apiVersion: helm.toolkit.fluxcd.io/v2beta1 +kind: HelmRelease +metadata: + name: podinfo + namespace: default +spec: + interval: 60m + chart: + spec: + chart: my-app + version: 1.0.0 # {"$imagepolicy": "default:my-app:tag"} + sourceRef: + kind: HelmRepository + name: ghcr-charts + interval: 1m +``` + ### Alternatives -We could use introduce a new API type e.g. `HelmOCIRepository`. That is considered unpractical, -as there is no benefit for users in having a dedicated kind instead of a `type` filed in the current -`HelmRepository` API. Adding a `type` filed to the spec follows the Flux Bucket API design, where -the same Kind servers different implementations: AWS S3 vs Azure Blob vs Google Storage. +We could introduce a new API type e.g. `HelmRegistry` to hold the reference to auth secret, +as proposed in [#2573](https://github.com/fluxcd/flux2/pull/2573). +That is considered unpractical, as there is no benefit for users in having a dedicated kind instead of +a `type` filed in the current `HelmRepository` API. Adding a `type` filed to the spec follows the Flux +Bucket API design, where the same Kind servers different implementations: AWS S3 vs Azure Blob vs Google Storage. ## Design Details From 20d7d0c78ac4371cc1130df0490004ba8514c180 Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Fri, 1 Apr 2022 15:21:10 +0300 Subject: [PATCH 3/5] Add image registry example to story 2 Signed-off-by: Stefan Prodan --- rfcs/helm-oci/README.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/rfcs/helm-oci/README.md b/rfcs/helm-oci/README.md index 6d9fc87c2c..b03fba6cf9 100644 --- a/rfcs/helm-oci/README.md +++ b/rfcs/helm-oci/README.md @@ -75,7 +75,7 @@ And finally in Flux `HelmReleases`, refer to the ghcr-charts `HelmRepository`: apiVersion: helm.toolkit.fluxcd.io/v2beta1 kind: HelmRelease metadata: - name: podinfo + name: my-app namespace: default spec: interval: 60m @@ -98,10 +98,19 @@ spec: Given that charts are stored in container registries, you can use Flux image automation and patch the chart version in Git, in the same way Flux works for updating container image tags. -Define an image policy using semver: +Define an image registry and a policy for the chart artifact: ```yaml apiVersion: image.toolkit.fluxcd.io/v1beta1 +kind: ImageRepository +metadata: + name: my-app + namespace: default +spec: + image: ghcr.io/my-org/charts/my-app + interval: 1m0s +--- +apiVersion: image.toolkit.fluxcd.io/v1beta1 kind: ImagePolicy metadata: name: my-app @@ -120,7 +129,7 @@ Then add the policy marker to the `HelmRelease` manifests in Git: apiVersion: helm.toolkit.fluxcd.io/v2beta1 kind: HelmRelease metadata: - name: podinfo + name: my-app namespace: default spec: interval: 60m From e06fa24616ec182e8fbe045f616571844523f04e Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Fri, 1 Apr 2022 15:29:50 +0300 Subject: [PATCH 4/5] Add dedicated reconcilers to the design docs Signed-off-by: Stefan Prodan --- rfcs/helm-oci/README.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/rfcs/helm-oci/README.md b/rfcs/helm-oci/README.md index b03fba6cf9..5962625012 100644 --- a/rfcs/helm-oci/README.md +++ b/rfcs/helm-oci/README.md @@ -22,7 +22,8 @@ they do today for container images. ### Goals - Add support for fetching Helm charts stored as OCI artifacts with minimal API changes to Flux. -- Make it easy for users to switch from HTTP/S Helm repositories to OCI repositories. +- Make it easy for users to switch from [HTTP/S Helm repositories](https://github.com/helm/helm-www/blob/416fabea6ffab8dc156b6a0c5eb5e8df5f5ef7dc/content/en/docs/topics/chart_repository.md) + to OCI repositories. ### Non-Goals @@ -148,18 +149,23 @@ spec: We could introduce a new API type e.g. `HelmRegistry` to hold the reference to auth secret, as proposed in [#2573](https://github.com/fluxcd/flux2/pull/2573). That is considered unpractical, as there is no benefit for users in having a dedicated kind instead of -a `type` filed in the current `HelmRepository` API. Adding a `type` filed to the spec follows the Flux +a `type` field in the current `HelmRepository` API. Adding a `type` field to the spec follows the Flux Bucket API design, where the same Kind servers different implementations: AWS S3 vs Azure Blob vs Google Storage. ## Design Details -In source-controller we'll add a new predicate for indexing `HelmRepositories` based on the `spec.type` field. +In source-controller we'll add a new predicate for filtering `HelmRepositories` based on the `spec.type` field. -When the `spec.type` field is set to `OCI`, the `HelmRepositoryReconciler` -will set the `HelmRepository` Ready status to `False` if the URL is not prefixed with `oci://`, -otherwise the Ready status will be set to `True`. +The current `HelmRepositoryReconciler` will be renamed to `HelmRepositoryDefaultReconciler`, +it's scope remains unchanged, and it will handle only objects with `type: Default`. + +We'll introduce a new reconciler named `HelmRepositoryOCIReconciler`, that will handle +objects with `type: OCI`. This reconciler will set the `HelmRepository` Ready status to +`False` if the URL is not prefixed with `oci://`, otherwise the Ready status will be set to `True`. + +The current `HelmChartReconciler` will be renamed to `HelmChartDefaultReconciler`, +it's scope remains unchanged, and it will handle only objects that refer to `HelmRepositories` with `type: Default`. -The current `HelmChartReconciler` will use the `HelmRepositories` with `type: Default`. For `type: OCI` we'll introduce a new reconciler `HelmChartOCIReconciler` that uses `oras` to download charts and their dependencies. From 30f977a7cb275b57fb23628b609d4170aa483477 Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Wed, 13 Apr 2022 10:27:11 +0300 Subject: [PATCH 5/5] Assign RFC-0002 to Helm OCI proposal Signed-off-by: Stefan Prodan --- rfcs/{helm-oci => 0002-helm-oci}/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename rfcs/{helm-oci => 0002-helm-oci}/README.md (98%) diff --git a/rfcs/helm-oci/README.md b/rfcs/0002-helm-oci/README.md similarity index 98% rename from rfcs/helm-oci/README.md rename to rfcs/0002-helm-oci/README.md index 5962625012..878055fc68 100644 --- a/rfcs/helm-oci/README.md +++ b/rfcs/0002-helm-oci/README.md @@ -1,10 +1,10 @@ -# RFC-xxxx Flux OCI support for Helm +# RFC-0002 Flux OCI support for Helm -**Status:** provisional +**Status:** implementable **Creation date:** 2022-03-30 -**Last update:** 2022-03-30 +**Last update:** 2022-04-13 ## Summary