Validating Admissions Webhook (VAW)

[thedodd]

• blocked by: Implement K8s Controller #53
• using axum for the interface..
• any replica of the Hadron Operator should be allowed to handle VAW requests as all data will be read and cached.
• VAW should query for an object from K8s in order to perform dynamic object validation, ensuring that only changes which are allowed to be made to a CRD are actually accepted. Kube API already provides a copy of the most up-to-date object.
• all Hadron CRDs should hit the VAW before admitting into the cluster. Malformed or invalid data will be rejected.
• all CRs should undergo full dynamic validation to ensure referenced objects exist. This mostly applies to pipelines referencing streams right now.
• finish last few reconciler tasks.

Pipeline Stage Removal

Technically, removing a stage from a pipeline is like removing a partition from a stream, it is tantamount to data loss. Normally we don't want to allow for a simple typo/mistake to delete data from a user's cluster. However, there are cases when it should be allowed and is desired for a stage to be removed.

For such cases, let's have the VAW look for an optional annotation on Pipeline objects, say allow-remove-stage/pipelines.hadron.rs (the value doesn't matter and isn't evaluated). When this is present, the VAW will allow a stage to be removed, else it will reject the change as an error describing the danger, but will also describe how to bypass this validation with the aforementioned annotation.

Also need to update the Pipeline Controller to look for such stage removals and update active pipelines to remove any such references to the removed stage. Adding of new stages should also be updated to be proactively updated.

Cert-Manager Dependency

Document the dependency on the cert-manager.

• update operator's env inputs to mount certs for webhook server, and update code to mount and use certs for server TLS.
• update helm chart to to only require optional cert-manager integration. Require secret key ref otherwise. See Make cert-manager optional #87