fix: Add gatekeeper-webhook post install hook to Helm chart

[joaoubaldo]

Signed-off-by: Joao Ubaldo me@joaoubaldo.com

What this PR does / why we need it:
This PR adds a postInstall Job/hook to probe gatekeeper-webhook API availability

Which issue(s) this PR fixes:
Fixes #1156

Special notes for your reviewer:
curlimages repository was used but if this is not reasonable I can add a custom Dockerfile and corresponding CI steps to this PR as well.

[sozercan]

@joaoubaldo thanks for the PR! can you run make manifests and commit the changes?

[joaoubaldo]

@sozercan done. I also renamed the hook and its values so that it is clearer what they are about.

[codecov-commenter]

Codecov Report

Merging #2052 (c5a62d5) into master (206bbe9) will decrease coverage by 0.13%.
The diff coverage is 40.00%.

@@            Coverage Diff             @@
##           master    #2052      +/-   ##
==========================================
- Coverage   54.50%   54.37%   -0.14%     
==========================================
  Files         111      111              
  Lines        9470     9470              
==========================================
- Hits         5162     5149      -13     
- Misses       3921     3931      +10     
- Partials      387      390       +3     

Flag	Coverage Δ
unittests	54.37% <40.00%> (-0.14%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...onstrainttemplate/constrainttemplate_controller.go	54.91% <33.33%> (-3.12%)	⬇️
pkg/webhook/policy.go	37.53% <50.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9056cbb...c5a62d5. Read the comment docs.

[ritazh]

Thanks for the PR @joaoubaldo! This PR currently ensures the helm install is not marked complete until the webhook is available. It however, does not ensure the namespace label container only runs when the webhook is available. So we will continue to get those errors. What do you think about adding this curl check as an initContainer to the gatekeeper-update-namespace-label job such that the namespace label container only runs after we confirm the webhook is available?

[joaoubaldo]

Thanks for the review @ritazh. That is a good point. I wasn't considering that the label added by the gatekeeper-update-namespace-label Job was causing the webhook to be called. I will test the initContainer approach and update the PR after.

[joaoubaldo]

@ritazh I have updated the PR to make the webhook probe process be part of the gatekeeper-update-namespace-label Job as an initContainer. Since .Values.postInstall.labelNamespace can be explicitly disabled, I think it still makes sense to run the gatekeeper-probe-webhook-post-install Job to ensure the deployment only succeeds when the webhook is ready - is this reasonable to you? Essentially gatekeeper-update-namespace-label and gatekeeper-probe-webhook-post-install Jobs are mutually exclusive.

Below are the manifests for both scenarios.

$ helm template opa-gatekeeper -n gatekeeper-system --set postInstall.probeWebhook.enabled=true --set postInstall.labelNamespace.enabled=true ...:

---
# Source: gatekeeper/templates/namespace-post-install.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gatekeeper-update-namespace-label
  labels:
    release: opa-gatekeeper
    heritage: Helm
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
---
# Source: gatekeeper/templates/namespace-post-install.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: gatekeeper-update-namespace-label
  labels:
    release: opa-gatekeeper
    heritage: Helm
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
      - update
      - patch
    resourceNames:
      - gatekeeper-system
---
# Source: gatekeeper/templates/namespace-post-install.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: gatekeeper-update-namespace-label
  labels:
    release: opa-gatekeeper
    heritage: Helm
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: gatekeeper-update-namespace-label
subjects:
  - kind: ServiceAccount
    name: gatekeeper-update-namespace-label
    namespace: "gatekeeper-system"
---
# Source: gatekeeper/templates/namespace-post-install.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: gatekeeper-update-namespace-label
  labels:
    app: 'gatekeeper'
    chart: 'gatekeeper'
    gatekeeper.sh/system: "yes"
    heritage: 'Helm'
    release: 'opa-gatekeeper'
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
spec:
  template:
    metadata:
      labels:
        app: 'gatekeeper'
        release: 'opa-gatekeeper'
    spec:
      restartPolicy: OnFailure
      serviceAccount: gatekeeper-update-namespace-label
      nodeSelector:
        kubernetes.io/os: linux
      volumes:
        - name: cert
          secret:
            secretName: gatekeeper-webhook-server-cert
      initContainers:
        - name: webhook-probe-post
          image: "curlimages/curl:7.83.1"
          imagePullPolicy: IfNotPresent
          args:
            - "--retry"
            - "99999"
            - "--retry-max-time"
            - "60"
            - "--retry-delay"
            - "1"
            - "--max-time"
            - "2"
            - "--cacert"
            - /certs/ca.crt
            - "-v"
            - "https://gatekeeper-webhook-service.gatekeeper-system.svc/v1/admitlabel?timeout=2s"
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
              - all
            readOnlyRootFilesystem: true
            runAsGroup: 999
            runAsNonRoot: true
            runAsUser: 1000
          volumeMounts:
          - mountPath: /certs
            name: cert
            readOnly: true
      containers:
        - name: kubectl-label
          image: "openpolicyagent/gatekeeper-crds:v3.9.0-beta.0"
          imagePullPolicy: IfNotPresent
          args:
            - label
            - ns
            - gatekeeper-system
            - admission.gatekeeper.sh/ignore=no-self-managing
            - --overwrite
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
              - all
            readOnlyRootFilesystem: true
            runAsGroup: 999
            runAsNonRoot: true
            runAsUser: 1000

$ helm template opa-gatekeeper -n gatekeeper-system --set postInstall.probeWebhook.enabled=true --set postInstall.labelNamespace.enabled=false ...:

---
# Source: gatekeeper/templates/probe-webhook-post-install.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: gatekeeper-probe-webhook-post-install
  labels:
    app: 'gatekeeper'
    chart: 'gatekeeper'
    gatekeeper.sh/system: "yes"
    heritage: 'Helm'
    release: 'opa-gatekeeper'
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
spec:
  template:
    metadata:
      labels:
        app: 'gatekeeper'
        release: 'opa-gatekeeper'
    spec:
      restartPolicy: Never
      nodeSelector:
        kubernetes.io/os: linux
      volumes:
        - name: cert
          secret:
            secretName: gatekeeper-webhook-server-cert
      containers:
        - name: webhook-probe-post
          image: "curlimages/curl:7.83.1"
          imagePullPolicy: IfNotPresent
          args:
            - "--retry"
            - "99999"
            - "--retry-max-time"
            - "60"
            - "--retry-delay"
            - "1"
            - "--max-time"
            - "2"
            - "--cacert"
            - /certs/ca.crt
            - "-v"
            - "https://gatekeeper-webhook-service.gatekeeper-system.svc/v1/admitlabel?timeout=2s"
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
              - all
            readOnlyRootFilesystem: true
            runAsGroup: 999
            runAsNonRoot: true
            runAsUser: 1000
          volumeMounts:
          - mountPath: /certs
            name: cert
            readOnly: true

[ritazh]

LGTM

[maxsmythe]

LGTM