Big Bang Component Extension

adr/0012-bigbang-as-a-noun.md

@@ -0,0 +1,39 @@
+# 12. BigBang as a Noun
+
+Date: 2023-01-18
+
+## Status
+
+Accepted
+
+## Context
+
+One primary application component that end users of Zarf are deploying is [Big Bang](https://repo1.dso.mil/platform-one/big-bang/bigbang).  The installation of BigBang is complicated for several reason:
+
+- It requires Flux to be installed to deploy correctly due to the use of Flux CRDs. 
+- The [images](https://umbrella-bigbang-releases.s3-us-gov-west-1.amazonaws.com/umbrella/1.51.0/package-images.yaml) defined within BigBang are normally a superset of the images needed for any individual deployment.
+- All images that BigBang might need takes 10s of gigabytes of storage to include in a Zarf package.
+- The git repositories defined within BigBang are normally a superset of the git repositories needed for any individual deployment.
+- Injecting a `values.yaml` file into the [default deployment structure](https://repo1.dso.mil/big-bang/bigbang/-/blob/master/base/kustomization.yaml) is complicated.
+and the discovery of which images are needed is a function of the values that are provided to the BigBang chart
+
+
+## Decision
+
+Deployments of BigBang can be managed with a new `bigbang` noun in the zarf.yaml that manages the complexity of the deployment.  This capability will take the values provided to the big bang chart, template them during the package phase to identify which [BigBang packages](https://repo1.dso.mil/big-bang/bigbang/-/blob/master/docs/packages.md) are being configured in the Zarf package.  The code then includes only the git repositories and images needed for the configured packages, and does not include the git repositories and images for packages that would not be deployed.  
+
+
+ The `bigbang` section will provide the following configurations for managing a big bang deployment:
+
+- `version` - Identifies the particular version of Bigbang to deploy, which correspond to git tags in the provided `repo`.  See versions of BigBang [here](https://repo1.dso.mil/big-bang/bigbang/-/releases).  
+- `repo` - Identifies the git repository BigBang is hosted on.  Defaults to https://repo1.dso.mil/platform-one/big-bang/bigbang.git
+- `valuesFrom` - list of local files that get passed to the BigBang helm chart for deployment. 
+- `skipFlux` - boolean to determine if the flux installation for BigBang should be skipped.  Only set this to true if flux has been deployed in a different way already in the cluster.
+
+
+## Consequences
+
+
+- By doing package time rendering and discovery of images for inclusion into the zarf package, the flexibility for deploy time configuration is limited since new parts of BigBang can't be added arbitrarily, since the necessary artifacts to deploy those BigBang packages won't be present in the zarf package
+- BigBang is every changing and improving, and while it is available as open source, we do not control the change in how the deployment is handled as [BigBang 2.0](https://repo1.dso.mil/groups/big-bang/-/epics/217) is progressing.  This creates a burden on the Zarf team to ensure new changes in Big Bang do not break how BigBang is deployed, and a burden to ensure as the way BigBang gets deployed is changed, it does not break older versions of deploying BigBang.
+

docs/4-user-guide/3-zarf-schema.md

@@ -1471,6 +1471,104 @@ Must be one of:
 </blockquote>
 </details>
 
+<details>
+<summary><strong> <a name="components_items_bigbang"></a>bigbang</strong>
+
+</summary>
+&nbsp;
+<blockquote>
+
+**Description:** Configurations for installing BigBang and Flux in the cluster
+
+|                           |                                                                                                          |
+| ------------------------- | -------------------------------------------------------------------------------------------------------- |
+| **Type**                  | `object`                                                                                                 |
+| **Additional properties** | [![Not allowed](https://img.shields.io/badge/Not%20allowed-red)](# "Additional Properties not allowed.") |
+| **Defined in**            | #/definitions/ZarfBigBang                                                                                |
+
+<details>
+<summary><strong> <a name="components_items_bigbang_version"></a>version *</strong>
+
+</summary>
+&nbsp;
+<blockquote>
+
+![Required](https://img.shields.io/badge/Required-red)
+
+**Description:** The version of Big Bang you'd like to use
+
+|          |          |
+| -------- | -------- |
+| **Type** | `string` |
+
+</blockquote>
+</details>
+
+<details>
+<summary><strong> <a name="components_items_bigbang_repo"></a>repo</strong>
+
+</summary>
+&nbsp;
+<blockquote>
+
+**Description:** Override of repo to pull big bang from
+
+|          |          |
+| -------- | -------- |
+| **Type** | `string` |
+
+</blockquote>
+</details>
+
+<details>
+<summary><strong> <a name="components_items_bigbang_valuesFrom"></a>valuesFrom</strong>
+
+</summary>
+&nbsp;
+<blockquote>
+
+**Description:** list of values files to pass to BigBang; these will be merged together
+
+|          |                   |
+| -------- | ----------------- |
+| **Type** | `array of string` |
+
+|                      | Array restrictions |
+| -------------------- | ------------------ |
+| **Min items**        | N/A                |
+| **Max items**        | N/A                |
+| **Items unicity**    | False              |
+| **Additional items** | False              |
+| **Tuple validation** | See below          |
+
+ ## <a name="autogenerated_heading_19"></a>valuesFrom items  
+
+|          |          |
+| -------- | -------- |
+| **Type** | `string` |
+
+</blockquote>
+</details>
+
+<details>
+<summary><strong> <a name="components_items_bigbang_skipFlux"></a>skipFlux</strong>
+
+</summary>
+&nbsp;
+<blockquote>
+
+**Description:** Should we skip deploying flux? Defaults to false
+
+|          |           |
+| -------- | --------- |
+| **Type** | `boolean` |
+
+</blockquote>
+</details>
+
+</blockquote>
+</details>
+
 </blockquote>
 </details>
 
@@ -1495,7 +1593,7 @@ Must be one of:
 | **Additional items** | False              |
 | **Tuple validation** | See below          |
 
- ## <a name="autogenerated_heading_19"></a>ZarfPackageVariable  
+ ## <a name="autogenerated_heading_20"></a>ZarfPackageVariable  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -1597,7 +1695,7 @@ Must be one of:
 | **Additional items** | False              |
 | **Tuple validation** | See below          |
 
- ## <a name="autogenerated_heading_20"></a>ZarfPackageConstant  
+ ## <a name="autogenerated_heading_21"></a>ZarfPackageConstant  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |

packages/big-bang-core/README.md

@@ -8,7 +8,11 @@ This package deploys [Big Bang Core](https://repo1.dso.mil/platform-one/big-bang
 
 ## Known Issues
 
-- Big Bang requires an AMD64 system to deploy as Iron Bank does not yet support ARM.  You will need to deploy to a cluster that is running AMD64.  Specifically, M1 Apple computers are not supported locally and you will need to provision a remote cluster to work with Big Bang currently.
+Lots of new things
+* dynamic finding of images
+* Hard coding patches
+* can only use one values.yaml file
+* update docs here to use a binary instead of the go function.
 
 ## Instructions
 
@@ -18,8 +22,9 @@ This package deploys [Big Bang Core](https://repo1.dso.mil/platform-one/big-bang
 # Clone the binaries
 git clone https://github.com/defenseunicorns/zarf.git
 
-# Change dir
-cd zarf
+# change to the examples folder
+cd zarf/examples/big-bang-core
+
 ```
 
 ### Get K3d components
@@ -32,49 +37,23 @@ Follow instructions on  https://zarf.dev/install/ to get the `zarf` cli
 
 (Optional) Alternatively, build the zarf components from the repo
 ```shell
-# Build zarf components from scratch (NOTE: golang and npm must be installed)
-make init-package
-
-# Add zarf cli from build dir to path
-export PATH=$(pwd)/build:$PATH
+# Create the deploy package and move it to the 'examples/sync' folder
+go run ../../main.go package create
 ```
 
-### Build the deploy package
+### Deploy an EKS cluster
 
 ```shell
-# Change dir
-cd packages/big-bang-core
-
-# Authenticate to the registry with Big Bang artifacts
-set +o history
-export REGISTRY1_USERNAME=<REPLACE_ME>
-export REGISTRY1_PASSWORD=<REPLACE_ME>
-echo $REGISTRY1_PASSWORD | zarf tools registry login registry1.dso.mil --username $REGISTRY1_USERNAME --password-stdin
-set -o history
-
-# Run zarf package command
-zarf package create . --confirm
+eksctl create cluster -f eksctl/demo.yaml
 ```
 
+Now wait 20 min :face_palm:
+
 ### Initialize Zarf
 
 ```shell
-# Start k3d cluster
-k3d cluster create
-
-# Initialize Zarf (interactively)
-zarf init
-# Make these choices at the prompt
-# ? Do you want to download this init package? Yes
-# ? Deploy this Zarf package? Yes
-# ? Deploy the k3s component? No
-# ? Deploy the logging component? No
-# ? Deploy the git-server component? Yes
-
-# (Optional) An alternative approach is to get the zarf init package from the zarf repo releases page or via build
-# Change dir to location of the zarf-init*.tar.zst (such as the build dir) & run the zarf init command with these flags
-cd ../../build
-zarf init --confirm --components git-server
+# Initialize Zarf
+go run ../../main.go init -a amd64 --confirm --components git-server
 
 # (Optional) Inspect the results
 zarf tools k9s
@@ -83,21 +62,27 @@ zarf tools k9s
 ### Deploy Big Bang
 
 ```shell
-# Deploy Big Bang (lightweight version)
-cd ../packages/big-bang-core
-zarf package deploy --confirm $(ls -1 zarf-package-big-bang-core-demo-*.tar.zst) --components big-bang-core-limited-resources
-# NOTE: to deploy the standard full set of components use the flag:
-# '--components big-bang-core-standard'
+# Deploy Big Bang
+./zarf package deploy zarf-package-big-bang-core-demo-arm64-1.47.0.tar.zst --confirm
 
 # (Optional) Inspect the results
 zarf tools k9s
 ```
 
+### See the results
+
+```shell
+kubectl get pods -n flux-system
+kubectl get hr -n bigbang
+kubectl get pods -A
+```
+
+
 ### Clean Up
 
 ```shell
-# Destroy the k3d cluster
-k3d cluster delete
+# Inside the VM
+eksctl delete cluster -f eksctl/demo.yaml --disable-nodegroup-eviction --wait
 ```
 
 ## Services

packages/big-bang-core/eksctl/demo.yaml

@@ -0,0 +1,15 @@
+---
+apiVersion: eksctl.io/v1alpha5
+kind: ClusterConfig
+
+metadata:
+  name: stack-demo
+  region: us-east-1
+
+managedNodeGroups:
+  - name: ng-1
+    instanceType: m5.2xlarge
+    desiredCapacity: 1
+    volumeSize: 150
+    ssh:
+      allow: true # will use ~/.ssh/id_rsa.pub as the default ssh key

packages/big-bang-core/values.gatekeeper.yaml

@@ -0,0 +1,13 @@
+gatekeeper:
+  values:
+    violations:
+      allowedDockerRegistries:
+        enabled: false
+      allowedHostFilesystem:
+        enabled: false
+      bannedImageTags:
+        enabled: false
+      noPrivilegedContainers:
+        enabled: false
+      podsHaveIstio:
+        enabled: false

packages/big-bang-core/values.kyverno.yaml

@@ -0,0 +1,62 @@
+domain: bigbang.dev
+
+# Policy
+gatekeeper:
+  enabled: false
+clusterAuditor:
+  enabled: false
+kyverno:
+  enabled: true
+kyvernopolicies:
+  enabled: true
+  values:
+    policies:
+      disallow-shared-subpath-volume-writes:
+        validationFailureAction: audit
+      restrict-host-ports: 
+        validationFailureAction: audit
+      restrict-capabilities:
+        validationFailureAction: audit
+      restrict-image-registries:
+        validationFailureAction: audit
+      disallow-host-namespaces:
+        validationFailureAction: audit
+      disallow-privileged-containers:
+        validationFailureAction: audit
+      require-non-root-user:
+        validationFailureAction: audit
+      restrict-host-path-mount-pv:
+        validationFailureAction: audit
+
+
+kiali:
+  enabled: false
+
+# logging
+logging:
+  enabled: false
+  engine: plg
+
+loki:
+  enabled: true
+
+promtail:
+  enabled: true
+  values:
+    serviceMonitor:
+      enabled: true
+
+eckoperator:
+  enabled: false
+
+fluentbit:
+  enabled: false
+
+
+# Tracing
+jaeger:
+  enabled: false
+
+
+twistlock:
+  enabled: false