Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add unicorn flavor to uds-core #507

Merged
merged 36 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
50add89
wip: add istio unicorn flavor
mjnagel Jun 24, 2024
20c5dd8
wip: grafana chainguard
mjnagel Jun 24, 2024
6d779b1
wip: keycloak chainguarded
mjnagel Jun 24, 2024
003218e
wip: loki chainguarded
mjnagel Jun 24, 2024
657a8b4
add unicorn flavor for `kube-prometheus-stack`
noahpb Jun 24, 2024
8ca4373
add unicorn flavor for `promtail`
noahpb Jun 24, 2024
dfffcab
wip: authservice chainguarded
mjnagel Jun 24, 2024
c36d4d3
wip: metrics-server chainguarded
mjnagel Jun 24, 2024
9fd558f
add unicorn flavor for `velero`
noahpb Jun 24, 2024
2f02a38
wip: hacky neuvector chainguarding
mjnagel Jun 24, 2024
4db2ea2
wip: enforcer/manager cert
mjnagel Jun 24, 2024
4ef6f39
wip: temporarily revert neuvector
mjnagel Jun 25, 2024
09d408b
chore: cleanup comment
mjnagel Jun 25, 2024
77de91a
chore: neuvector upstream
mjnagel Jun 25, 2024
0903e60
ci: add tests for unicorn
mjnagel Jun 25, 2024
ab4c9eb
ci: pin sha for chainctl
mjnagel Jun 25, 2024
4d15e39
Update src/neuvector/chart/templates/_helpers.tpl
mjnagel Jun 25, 2024
1ad7f39
Update .github/workflows/publish.yaml
mjnagel Jun 25, 2024
f731cdb
Update src/neuvector/zarf.yaml
mjnagel Jun 25, 2024
041e96b
chore: rebase, resolve conflicts
mjnagel Jul 8, 2024
1ea4702
Merge branch 'main' into chainguard-flavor
mjnagel Jul 9, 2024
f66d215
chore: update grafana [skip ci]
mjnagel Jul 9, 2024
b3a5ba1
chore: unicorn publish, renovate update
mjnagel Jul 10, 2024
ea08e61
chore: echo
mjnagel Jul 10, 2024
27ae723
publish change
mjnagel Jul 10, 2024
057b76d
Revert "publish change"
mjnagel Jul 10, 2024
b0fd7a9
fix: workflow conditional
mjnagel Jul 10, 2024
4d9d2ae
fix: id-token on snapshot
mjnagel Jul 10, 2024
e98374e
fix: reorder conditionals for unicorn
mjnagel Jul 10, 2024
bcfd7bf
rebase, fix conflicts
mjnagel Jul 11, 2024
dc68b8d
Merge branch 'main' into chainguard-flavor
mjnagel Jul 11, 2024
64987e3
docs: add flavor documentation
mjnagel Jul 11, 2024
f1f72ba
docs: clarifications
mjnagel Jul 12, 2024
bfbc451
chore: simplify logic around destination repository
mjnagel Jul 12, 2024
c829bde
docs: unicorn flavor alpha
mjnagel Jul 12, 2024
17f09f2
Merge branch 'main' into chainguard-flavor
mjnagel Jul 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/actions/setup/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ inputs:
registry1Password:
description: 'IRON_BANK_ROBOT_PASSWORD'
required: true
chainguardIdentity:
description: "ID for Chainguard Identity"
required: true

runs:
using: "composite"
Expand Down Expand Up @@ -40,6 +43,12 @@ runs:
run: echo "${{ env.REGISTRY_PASSWORD }}" | uds zarf tools registry login -u "${{ env.REGISTRY_USERNAME }}" --password-stdin registry1.dso.mil
shell: bash

- name: Chainguard Login
if: ${{ inputs.chainguardIdentity != '' }}
uses: chainguard-dev/setup-chainctl@fc62b08dfd3179dd694b50f672bc371f878fbd1e # v0.2.1
with:
identity: ${{ inputs.chainguardIdentity }}

- name: GHCR Login
if: ${{ inputs.ghToken != '' }}
env:
Expand Down
40 changes: 26 additions & 14 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,14 @@ jobs:
publish-uds-core:
strategy:
matrix:
flavor: [upstream, registry1]
flavor: [upstream, registry1, unicorn]
runs-on: "uds-ubuntu-big-boy-8-core"
name: Publish packages

permissions:
contents: read
packages: write
id-token: write # This is needed for OIDC federation.

steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
Expand All @@ -29,6 +30,7 @@ jobs:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}
chainguardIdentity: ${{ secrets.CHAINGUARD_IDENTITY }}

- name: (Snapshot) Get snapshot version using git commit short sha and date
if: ${{ inputs.snapshot }}
Expand Down Expand Up @@ -59,7 +61,7 @@ jobs:
ZARF_ARCHITECTURE=arm64 uds run -f tasks/create.yaml k3d-slim-dev-bundle --no-progress
fi

# Standard Package by default tests what's in the Istio Package
# Standard Package by default tests full core
- name: Test amd64 Bundle
if: ${{ !inputs.snapshot }}
run: |
Expand All @@ -70,21 +72,31 @@ jobs:
if: ${{ always() && !inputs.snapshot }}
uses: ./.github/actions/debug-output

- name: Publish Standard Package
if: ${{ !inputs.snapshot }}
run: uds run -f tasks/publish.yaml standard-package --set FLAVOR=${{ matrix.flavor }} --no-progress
# Determine repository to publish to
- name: Determine destination repository
id: repo
run: |
repo=ghcr.io/defenseunicorns/packages
# Publish unicorn flavor to private repository
if [ "${{ matrix.flavor }}" = "unicorn" ]; then
repo+=/private
fi
repo+=/uds
# Publish snapshots to snapshot repository
if [ "${{ inputs.snapshot }}" = "true" ]; then
repo+=/snapshots
fi

- name: Publish Upstream Flavored Bundles
if: ${{ !inputs.snapshot && matrix.flavor != 'registry1' }}
run: uds run -f tasks/publish.yaml bundles --no-progress
echo "repo=${repo}" >> "$GITHUB_OUTPUT"
echo "Publishing packages and bundles to ${repo}"

- name: (Snapshot) Publish Standard Package
if: ${{ inputs.snapshot }}
run: uds run -f tasks/publish.yaml standard-package --set FLAVOR=${{ matrix.flavor }} --set TARGET_REPO="ghcr.io/defenseunicorns/packages/uds/snapshots" --set VERSION="${SNAPSHOT_VERSION}" --no-progress
# Publish package and bundle to destination repository
- name: Publish Standard Package
run: uds run -f tasks/publish.yaml standard-package --set FLAVOR=${{ matrix.flavor }} --set TARGET_REPO=${{ steps.repo.outputs.repo }} --no-progress

- name: (Snapshot) Publish Upstream Flavored Bundles
if: ${{ inputs.snapshot && matrix.flavor != 'registry1' }}
run: uds run -f tasks/publish.yaml bundles --set TARGET_REPO="ghcr.io/defenseunicorns/packages/uds/snapshots" --set VERSION="${SNAPSHOT_VERSION}" --no-progress
- name: Publish Upstream Flavored Bundles
if: ${{ matrix.flavor == 'upstream' }}
run: uds run -f tasks/publish.yaml bundles --set TARGET_REPO=${{ steps.repo.outputs.repo }} --no-progress

- name: Save logs
if: always()
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/pull-request-conditionals.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ jobs:
strategy:
matrix:
package: ${{ fromJSON(needs.check-paths.outputs.packages) }}
flavor: [upstream, registry1]
flavor: [upstream, registry1, unicorn]
test_type: [install]
include:
- package: all
Expand All @@ -75,6 +75,10 @@ jobs:
- package: all
flavor: upstream
test_type: upgrade
# Commented out until unicorn flavor has a published release
# - package: all
# flavor: unicorn
# test_type: upgrade
uses: ./.github/workflows/test.yaml
with:
package: ${{ matrix.package }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/snapshot-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ jobs:
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/publish.yaml
with:
snapshot: true
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/tag-and-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/publish.yaml
with:
snapshot: false
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ on:

permissions:
contents: read
id-token: write # This is needed for OIDC federation.

jobs:
test:
Expand All @@ -53,6 +54,7 @@ jobs:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}
chainguardIdentity: ${{ secrets.CHAINGUARD_IDENTITY }}

- name: Test a single source package
if: ${{ inputs.package != 'all' && inputs.test_type == 'install' }}
Expand Down
23 changes: 23 additions & 0 deletions docs/deployment/flavors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: Published Flavors
type: docs
weight: 2
---

UDS Core is published with multiple variations (Zarf flavors). Each flavor uses a separate source registry for the images. Each flavor is used as the suffix on the OCI tags for packages. For production use cases we recommend the `registry1` or `unicorn` flavors as these images tend to be more secure than their `upstream` counterparts.

{{% alert-note %}}
Demo and dev bundles (`k3d-core-demo` and `k3d-core-slim-dev`) are only published from the upstream flavor.
{{% /alert-note %}}

### Flavors

| Flavor | GHCR Location | Image Source |
| --------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `registry1` | `ghcr.io/defenseunicorns/packages/uds` | [Ironbank](https://p1.dso.mil/services/iron-bank) - DoD hardened images (only supports amd64 architecture currently) |
| `upstream` | `ghcr.io/defenseunicorns/packages/uds` | Various sources, typically DockerHub/GHCR/Quay, these are the default images used by helm charts |
| **ALPHA** `unicorn` | `ghcr.io/defenseunicorns/packages/private/uds` | Industry best images designed with security and minimalism in mind |

{{% alert-note %}}
The `unicorn` flavored packages are only available in a private repository. These packages are available for all members of the Defense Unicorns organization/company, if you are outside the organization [contact us](https://www.defenseunicorns.com/contactus) if you are interested in using this flavor for your mission.
{{% /alert-note %}}
2 changes: 1 addition & 1 deletion docs/deployment/uds-deploy.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Deploy UDS Core
type: docs
weight: 2
weight: 3
---

## Prerequisites
Expand Down
19 changes: 19 additions & 0 deletions docs/development/flavor-specific-dev.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: Flavor Specific Development Notes
type: docs
weight: 5
---

Specific flavors of UDS Core have access and architecture restrictions when used for development work. The `upstream` flavor is generally recommended for development as it does not have any restrictions or requirements.

### Registry1

The `registry1` flavor uses images from [Ironbank](https://p1.dso.mil/services/iron-bank) which can only be pulled with authentication. Developers can self-register on [P1 SSO](https://login.dso.mil/) and retrieve a pull token for auth from [registry1's Harbor](https://registry1.dso.mil/). (In upper right corner, click --> User Profile, then click the Copy icon next to CLI secret, and use this for `docker login`.)

Images in `registry1` historically only supported `amd64` architectures. While some images do now support `arm64` architecture, uds-core only supports `amd64` for the `registry1` flavor. If developing on an `arm64` machine you will need to use a virtualization layer or an external dev box.

### Unicorn

The `unicorn` flavor uses images primarily from a private Chainguard repository. These images can be pulled by any developers in the Defense Unicorns organization once added to the Chainguard repository. Local authentication should be done with [chainctl](https://edu.chainguard.dev/chainguard/administration/how-to-install-chainctl/), specifically using the [credential helper](https://edu.chainguard.dev/chainguard/administration/how-to-install-chainctl/#configure-a-docker-credential-helper) for a seamless experience.

Developers outside of the Defense Unicorns organization/company will be unable to pull these images directly and should rely on CI testing for validation of this flavor. [Contact us](https://www.defenseunicorns.com/contactus) if you have a need to pull these images and develop on this flavor in particular.
11 changes: 0 additions & 11 deletions docs/development/uds-development-maintenance.md

This file was deleted.

8 changes: 8 additions & 0 deletions renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@
"depNameTemplate": "docker.io/neuvector/controller",
"datasourceTemplate": "docker"
},
{
"fileMatch": ["^src/neuvector/values/unicorn-values\\.yaml$"],
"matchStrings": [
"registry: cgr.dev\\s+tag:\\s+[\"]?(?<currentValue>[^\"]*)[\"]?"
],
"depNameTemplate": "cgr.dev/du-uds-defenseunicorns/neuvector-controller-fips",
"datasourceTemplate": "docker"
},
{
"fileMatch": ["^src/neuvector/values/registry1-values\\.yaml$"],
"matchStrings": [
Expand Down
3 changes: 3 additions & 0 deletions src/authservice/values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
image:
repository: cgr.dev/du-uds-defenseunicorns/authservice-fips
tag: "1.0.1"
13 changes: 13 additions & 0 deletions src/authservice/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,16 @@ components:
- values/registry1-values.yaml
images:
- registry1.dso.mil/ironbank/istio-ecosystem/authservice:1.0.1-ubi9

- name: authservice
required: true
only:
flavor: unicorn
import:
path: common
charts:
- name: authservice
valuesFiles:
- values/unicorn-values.yaml
images:
- cgr.dev/du-uds-defenseunicorns/authservice-fips:1.0.1
21 changes: 21 additions & 0 deletions src/grafana/values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
image:
registry: cgr.dev
repository: du-uds-defenseunicorns/grafana-fips
tag: 11.1.0

initChownData:
image:
registry: cgr.dev
repository: du-uds-defenseunicorns/busybox-fips
tag: 1.36.1

downloadDashboardsImage:
registry: cgr.dev
repository: du-uds-defenseunicorns/curl-fips
tag: 8.8.0

sidecar:
image:
registry: cgr.dev
repository: du-uds-defenseunicorns/k8s-sidecar-fips
tag: 1.27.4
16 changes: 16 additions & 0 deletions src/grafana/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,19 @@ components:
- registry1.dso.mil/ironbank/opensource/grafana/grafana:11.1.0
- registry1.dso.mil/ironbank/redhat/ubi/ubi9-minimal:9.4
- registry1.dso.mil/ironbank/kiwigrid/k8s-sidecar:1.27.4

- name: grafana
required: true
only:
flavor: unicorn
import:
path: common
charts:
- name: grafana
valuesFiles:
- values/unicorn-values.yaml
images:
- cgr.dev/du-uds-defenseunicorns/grafana-fips:11.1.0
- cgr.dev/du-uds-defenseunicorns/busybox-fips:1.36.1
- cgr.dev/du-uds-defenseunicorns/curl-fips:8.8.0
- cgr.dev/du-uds-defenseunicorns/k8s-sidecar-fips:1.27.4
7 changes: 7 additions & 0 deletions src/istio/values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
pilot:
image: cgr.dev/du-uds-defenseunicorns/istio-pilot-fips:1.22.2
global:
proxy_init:
image: cgr.dev/du-uds-defenseunicorns/istio-proxy-fips:1.22.2
proxy:
image: cgr.dev/du-uds-defenseunicorns/istio-proxy-fips:1.22.2
14 changes: 14 additions & 0 deletions src/istio/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,20 @@ components:
- registry1.dso.mil/ironbank/tetrate/istio/proxyv2:1.22.2-tetratefips-v0
- registry1.dso.mil/ironbank/tetrate/istio/pilot:1.22.2-tetratefips-v0

- name: istio-controlplane
required: true
only:
flavor: unicorn
import:
path: common
charts:
- name: istiod
valuesFiles:
- "values/unicorn-values.yaml"
images:
- cgr.dev/du-uds-defenseunicorns/istio-pilot-fips:1.22.2
- cgr.dev/du-uds-defenseunicorns/istio-proxy-fips:1.22.2

- name: istio-admin-gateway
required: true
charts:
Expand Down
3 changes: 3 additions & 0 deletions src/keycloak/values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
image:
repository: cgr.dev/du-uds-defenseunicorns/keycloak
tag: "24.0.5"
14 changes: 14 additions & 0 deletions src/keycloak/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,17 @@ components:
images:
- registry1.dso.mil/ironbank/opensource/keycloak/keycloak:24.0.5
- ghcr.io/defenseunicorns/uds/identity-config:0.5.0

- name: keycloak
required: true
only:
flavor: unicorn
import:
path: common
charts:
- name: keycloak
valuesFiles:
- "values/unicorn-values.yaml"
images:
- cgr.dev/du-uds-defenseunicorns/keycloak:24.0.5 # todo: switch to FIPS image
- ghcr.io/defenseunicorns/uds/identity-config:0.5.0
10 changes: 10 additions & 0 deletions src/loki/values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
loki:
image:
registry: cgr.dev
repository: du-uds-defenseunicorns/loki
tag: 2.9.8
gateway:
image:
registry: cgr.dev
repository: du-uds-defenseunicorns/nginx-fips
tag: 1.27.0
15 changes: 15 additions & 0 deletions src/loki/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,18 @@ components:
images:
- registry1.dso.mil/ironbank/opensource/grafana/loki:2.9.6
- registry1.dso.mil/ironbank/opensource/nginx/nginx-alpine:1.25.3

- name: loki
required: true
description: "Install Loki using chainguard images"
only:
flavor: "unicorn"
import:
path: common
charts:
- name: loki
valuesFiles:
- ./values/unicorn-values.yaml
images:
- cgr.dev/du-uds-defenseunicorns/loki:2.9.8
- cgr.dev/du-uds-defenseunicorns/nginx-fips:1.27.0
3 changes: 3 additions & 0 deletions src/metrics-server/values/unicorn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
image:
repository: cgr.dev/du-uds-defenseunicorns/metrics-server-fips
tag: "0.7.1"
13 changes: 13 additions & 0 deletions src/metrics-server/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,16 @@ components:
- "values/registry1-values.yaml"
images:
- registry1.dso.mil/ironbank/opensource/kubernetes-sigs/metrics-server:v0.7.1

- name: metrics-server
required: true
only:
flavor: unicorn
import:
path: common
charts:
- name: metrics-server
valuesFiles:
- "values/unicorn-values.yaml"
images:
- cgr.dev/du-uds-defenseunicorns/metrics-server-fips:0.7.1
Loading
Loading