From 138222d37af126d56ae7335a6dd50482ab38cb84 Mon Sep 17 00:00:00 2001 From: John Howard Date: Wed, 22 Mar 2023 13:28:33 -0700 Subject: [PATCH 1/6] GEP-1762: In Cluster Gateway Deployments --- geps/gep-1762.md | 281 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 geps/gep-1762.md diff --git a/geps/gep-1762.md b/geps/gep-1762.md new file mode 100644 index 0000000000..2c9846549a --- /dev/null +++ b/geps/gep-1762.md @@ -0,0 +1,281 @@ +# GEP-1757: In Cluster Gateway Deployments + +* Status: Provisional + +## Overview + +Gateway API provides a common abstraction over different implementations, whether they are implemented by cloud load balancers, in-cluster deployments, or other mechanisms. However, many in-cluster implementations have solved some of the same problems in different ways. + +Related discussions: +* [Support cluster-local Gateways](https://github.com/kubernetes-sigs/gateway-api/discussions/1247) +* [Scaling Gateway Resources](https://github.com/kubernetes-sigs/gateway-api/discussions/1355) +* [Manual deployments](https://github.com/kubernetes-sigs/gateway-api/issues/1687) +* [Merging Gateways](https://github.com/kubernetes-sigs/gateway-api/pull/1863/) +* [Per-Gateway Infrastructure](https://github.com/kubernetes-sigs/gateway-api/pull/1757) + +## Goals + +* Provide prescriptive guidance for how in-cluster implementations should behave. +* Provide requirements (tested by conformance) for how in-cluster implementations should behave. + +Note that some changes will be suggestions, while others will be requirements. + +## Non-Goals + +* Provide guidance to how out-of-cluster implementations should behave. Rather, this document aims to bring consistency between these types. + +## Terminology + +This document uses a few terms throughout. To ensure consistency, they are defined below: + +* In-cluster deployment: refers to an implementation that actuates a `Gateway` by running a data plane in the cluster. + This is *often*, but not necessarily, by deploying a `Deployment`/`DaemonSet` and `Service`. +* Automated deployment: refers to an implementation that automatically deploys the data plane based on a `Gateway`. + That is, the user simply creates a `Gateway` resource and the rest is handled behind the scenes by the implementation. +* Manual deployment: refers to an implementation that does NOT automatically deploy the data plane based on a `Gateway`. + This may require, for example, a user to manually create a `Deployment` and `Service` for the data plane, then link this to a `Gateway`. + +## Design + +This GEP does not introduce new API fields, but rather standardizes how implementations should behave when implementing the existing API. + +### Automated Deployments + +A simple `Gateway`, as is configured below is assumed to be an automated deployment: + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: my-gateway +spec: + gatewayClassName: example + listeners: + - name: default + port: 80 + protocol: HTTP +``` + +With this configuration, an implementation: +* MUST mark the Gateway as `Programmed` and provide an address in `Status.Addresses` where the Gateway can be reached on each configured port. +* MUST label all generated resources (Service, Deployment, etc) with `gateway.networking.k8s.io/metadata.name: my-gateway` (where `my-gateway` is the name of the Gateway resource). +* MUST provision generated resources in the same namespace as the Gateway if they are namespace scoped resources. + * Cluster scoped resources are not recommended. +* MUST name all generated resources `my-gateway-example` (`-`). + This is not simply `NAME` to reduce the chance of conflicts with existing resources. + +### Manual Deployments + +When a manual deployment is used, a user must explicitly link their Gateway to existing infrastructure. This is done by specifying `spec.infrastructure.attachTo`. + +For the common case that the "existing infrastructure" is a `Service`: + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: my-gateway +spec: + infrastructure: + attachTo: + kind: Service + name: existing-service + gatewayClassName: example + listeners: + - name: default + port: 80 + protocol: HTTP +``` + +With this configuration, an implementation: +* MUST provide an address in `Status.Addresses` where the Gateway can be reached. + This MUST be derived from the referenced `spec.infrastructure.attachTo`. +* MUST not deploy any resources into the cluster; it is expected that a user will do these actions. +* MUST reject any other `infrastructure` fields (see [GEP-1867](https://gateway-api.sigs.k8s.io/geps/gep-1867/#api)) being configured; these will not be respected. + +### Customizations + +With any in-cluster deployment, customization requirements will arise. + +Some common requirements would be: +* `Service.spec.type`, to control whether a service is a `ClusterIP` or `LoadBalancer`. +* IP in the Service to assign to it. +* Arbitrary labels and annotations on generated resources. +* Any other arbitrary fields; the list is unbounded. Some examples would be: + * CPU and memory requests + * Service `externalTrafficPolicy` + * Affinity rules. + +#### Gateway Type + +This is handled by [GEP-1651](https://github.com/kubernetes-sigs/gateway-api/pull/1653), so won't be described here. + +#### Gateway IP + +For setting a specific IP address in a generated `Service`, the `.Spec.Addresses` field can be used. + +For example: + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: my-gateway +spec: + addresses: + - type: IPAddress + value: 1.1.1.1 + gatewayClassName: example + listeners: + - name: default + port: 80 + protocol: HTTP +``` + +This would generate a `Service` with `clusterIP` or `loadBalancerIP`, depending on the Service type. + +This follows the same behavior as out-of-cluster Gateway implementations. + +Note: this differs from the selection of "manual deployment", which doesn't use `IPAddress` type. +Instead, these Gateways attach to some existing infrastructure (such as Service) which provides an address. + +#### Labels and Annotations + +Labels and annotations for generated resources are specified in `infrastructure`: + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: my-gateway +spec: + infrastructure: + labels: + foo: bar + annotations: + name: my-annotation +``` + +These are both `map[string]string` types, just like in `ObjectMeta`. + +Any labels or annotations here are added to all generated resources. +Note this may mean an annotation intended for a `Service` may end up on a `Deployment` (for example). +This is typically not a concern; however, if an implementation is aware of specific meanings of certain labels or annotations, they MAY +exclude these from irrelevant resources. + +This is intended to support integration with the kitchen-sync of Kubernetes extensions which rely on labels and annotations, +such as as prometheus scraping, object grouping/organization, etc. + +#### Arbitrary Customization + +GEP-1867 introduces a new `infrastructure` field, which allows customization of some common configurations (version, size, etc) +and allows a per-Gateway generic `parametersRef`. +This can be utilized for the remainder of customizations. + +### Resource Attachment + +Resources Generated in response to the `Gateway` are guaranteed to have two attributes: +* A `gateway.networking.k8s.io/metadata.name: ` label. +* A name `-`. + +The generated resources MUST be in the same namespaces as the `Gateway`. + +Implementations MAY set `ownerReferences` to the `Gateway` in most cases, as well, but this is not required +as some implementations may have different cleanup mechanisms. + + +These can be relied on to attach resources to. +While "Policy attachment" in the gateway-api would use attachment to the actual `Gateway` resource itself, +many existing things attach only to resources like `Deployment` or `Service`. + +An example using these: +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway +spec: + gatewayClassName: example + listeners: + - name: default + hostname: "example.com" + port: 80 + protocol: HTTP +--- +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: gateway +spec: + # Match the generated Deployment by reference + # Note: Do not use `kind: Gateway`. + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: gateway-example + minReplicas: 2 + maxReplicas: 5 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 50 +--- +apiVersion: policy/v1 +kind: PodDisruptionBudget +metadata: + name: gateway +spec: + minAvailable: 1 + selector: + # Match the generated Deployment by label + matchLabels: + gateway.networking.k8s.io/metadata.name: gateway +``` + +Note: there is [discussion](https://github.com/kubernetes-sigs/gateway-api/discussions/1355) around a way to attach a HPA to a Gateway directly. + +## API + +This GEP extends the `infrastructure` API introduced in [GEP-1867](https://gateway-api.sigs.k8s.io/geps/gep-1867). + +```go +type GatewayInfrastructure struct { + // Labels that should be applied to any resources created in response to this Gateway. + // + // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. + // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. + // + // Support: Implementation-specific + Labels map[string]string `json:"labels,omitempty"` + // Annotations that should be applied to any resources created in response to this Gateway. + // + // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. + // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. + // + // Support: Implementation-specific + Annotations map[string]string `json:"annotations,omitempty"` + // AttachTo is an optional, implementation-specific way to attach a Gateway to some existing infrastructure. + // For in-cluster deployments, implementations MAY allow attaching to an existing `Service` object. + // + // When this field is set, an implementation: + // * MUST continue to provide an address in `Status.Addresses` where the Gateway can be reached. This SHOULD be + // derived from the referenced `spec.infrastructure.attachTo` object, if applicable. + // * If the presence of an `attachTo` means new infrastructure will not be provisioned, an implementation MUST reject + // any other `infrastructure` fields that will not be respected as a result. + // * For in-cluster implementations referencing `Service` + // * MUST not create or modify compute resources (such as `Deployment` or `Service`) into the cluster; it is expected that a user will do these actions. + // + // Support: Implementation-specific + // + // +optional + AttachTo []LocalObjectReference `json:"attachTo,omitempty"` + ... +} + +type GatewayClassInfrastructure struct { + ... +} +``` From f9ddd75f3b01edc88c0391d80be4589ceb1176a1 Mon Sep 17 00:00:00 2001 From: John Howard Date: Fri, 25 Aug 2023 08:45:00 -0700 Subject: [PATCH 2/6] Address Nick's comments --- geps/gep-1762.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 54 insertions(+), 1 deletion(-) diff --git a/geps/gep-1762.md b/geps/gep-1762.md index 2c9846549a..f4e465325e 100644 --- a/geps/gep-1762.md +++ b/geps/gep-1762.md @@ -1,4 +1,4 @@ -# GEP-1757: In Cluster Gateway Deployments +# GEP-1762: In Cluster Gateway Deployments * Status: Provisional @@ -248,6 +248,8 @@ type GatewayInfrastructure struct { // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. // + // If Labels is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. + // // Support: Implementation-specific Labels map[string]string `json:"labels,omitempty"` // Annotations that should be applied to any resources created in response to this Gateway. @@ -255,6 +257,8 @@ type GatewayInfrastructure struct { // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. // + // If Annotations is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. + // // Support: Implementation-specific Annotations map[string]string `json:"annotations,omitempty"` // AttachTo is an optional, implementation-specific way to attach a Gateway to some existing infrastructure. @@ -272,10 +276,59 @@ type GatewayInfrastructure struct { // // +optional AttachTo []LocalObjectReference `json:"attachTo,omitempty"` + // ParametersRef is a reference to a resource that contains the configuration + // parameters corresponding to the Gateway. This is optional if the + // controller does not require any additional configuration. + // + // ParametersRef can reference a standard Kubernetes resource, i.e. ConfigMap, + // or an implementation-specific custom resource. The resource must be namespace-scoped + // and live in the same namespace. + // + // If ParametersRef is set on both a Gateway and the corresponding GatewayClass, the merging + // behavior between the two are implementation-specific. + // + // Support: Implementation-specific + // + // +optional + ParametersRef *LocalParametersReference `json:"parametersRef,omitempty"` ... } type GatewayClassInfrastructure struct { + // Labels that should be applied to any resources created in response to this GatewayClass. + // + // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. + // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. + // + // If Labels is set on the Gateway as well, the labels are merged with the Gateway's labels taking precedence. + // + // Support: Implementation-specific + Labels map[string]string `json:"labels,omitempty"` + // Annotations that should be applied to any resources created in response to this GatewayClass. + // + // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. + // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. + // + // If Annotations is set on the Gateway as well, the labels are merged with the Gateway's annotations taking precedence. + // + // Support: Implementation-specific + Annotations map[string]string `json:"annotations,omitempty"` ... } + +// LocalParametersReference identifies an API object containing controller-specific +// configuration resource within the cluster. +type LocalParametersReference struct { + // Group is the group of the referent. + Group Group `json:"group"` + + // Kind is kind of the referent. + Kind Kind `json:"kind"` + + // Name is the name of the referent. + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Name string `json:"name"` +} ``` From 5cd70cba5556ab0010ebf9698b6213eae9c64ab4 Mon Sep 17 00:00:00 2001 From: John Howard Date: Fri, 15 Sep 2023 10:20:18 -0700 Subject: [PATCH 3/6] Drop attachTo --- geps/gep-1762.md | 42 +----------------------------------------- 1 file changed, 1 insertion(+), 41 deletions(-) diff --git a/geps/gep-1762.md b/geps/gep-1762.md index f4e465325e..f6135fc87c 100644 --- a/geps/gep-1762.md +++ b/geps/gep-1762.md @@ -66,32 +66,7 @@ With this configuration, an implementation: ### Manual Deployments -When a manual deployment is used, a user must explicitly link their Gateway to existing infrastructure. This is done by specifying `spec.infrastructure.attachTo`. - -For the common case that the "existing infrastructure" is a `Service`: - -```yaml -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: my-gateway -spec: - infrastructure: - attachTo: - kind: Service - name: existing-service - gatewayClassName: example - listeners: - - name: default - port: 80 - protocol: HTTP -``` - -With this configuration, an implementation: -* MUST provide an address in `Status.Addresses` where the Gateway can be reached. - This MUST be derived from the referenced `spec.infrastructure.attachTo`. -* MUST not deploy any resources into the cluster; it is expected that a user will do these actions. -* MUST reject any other `infrastructure` fields (see [GEP-1867](https://gateway-api.sigs.k8s.io/geps/gep-1867/#api)) being configured; these will not be respected. +Managing "Manual Deployments" is currently out of scope for this GEP, but may be added in the future. ### Customizations @@ -261,21 +236,6 @@ type GatewayInfrastructure struct { // // Support: Implementation-specific Annotations map[string]string `json:"annotations,omitempty"` - // AttachTo is an optional, implementation-specific way to attach a Gateway to some existing infrastructure. - // For in-cluster deployments, implementations MAY allow attaching to an existing `Service` object. - // - // When this field is set, an implementation: - // * MUST continue to provide an address in `Status.Addresses` where the Gateway can be reached. This SHOULD be - // derived from the referenced `spec.infrastructure.attachTo` object, if applicable. - // * If the presence of an `attachTo` means new infrastructure will not be provisioned, an implementation MUST reject - // any other `infrastructure` fields that will not be respected as a result. - // * For in-cluster implementations referencing `Service` - // * MUST not create or modify compute resources (such as `Deployment` or `Service`) into the cluster; it is expected that a user will do these actions. - // - // Support: Implementation-specific - // - // +optional - AttachTo []LocalObjectReference `json:"attachTo,omitempty"` // ParametersRef is a reference to a resource that contains the configuration // parameters corresponding to the Gateway. This is optional if the // controller does not require any additional configuration. From 4c2ae889c6ce055173e59aad70a3f31e06ccabff Mon Sep 17 00:00:00 2001 From: John Howard Date: Fri, 15 Sep 2023 12:39:26 -0700 Subject: [PATCH 4/6] Address Rob's comments --- geps/gep-1762.md | 69 ++++++++++++++++++++++++++---------------------- 1 file changed, 37 insertions(+), 32 deletions(-) diff --git a/geps/gep-1762.md b/geps/gep-1762.md index f6135fc87c..3f232585e5 100644 --- a/geps/gep-1762.md +++ b/geps/gep-1762.md @@ -7,6 +7,7 @@ Gateway API provides a common abstraction over different implementations, whether they are implemented by cloud load balancers, in-cluster deployments, or other mechanisms. However, many in-cluster implementations have solved some of the same problems in different ways. Related discussions: + * [Support cluster-local Gateways](https://github.com/kubernetes-sigs/gateway-api/discussions/1247) * [Scaling Gateway Resources](https://github.com/kubernetes-sigs/gateway-api/discussions/1355) * [Manual deployments](https://github.com/kubernetes-sigs/gateway-api/issues/1687) @@ -32,12 +33,10 @@ This document uses a few terms throughout. To ensure consistency, they are defin This is *often*, but not necessarily, by deploying a `Deployment`/`DaemonSet` and `Service`. * Automated deployment: refers to an implementation that automatically deploys the data plane based on a `Gateway`. That is, the user simply creates a `Gateway` resource and the rest is handled behind the scenes by the implementation. -* Manual deployment: refers to an implementation that does NOT automatically deploy the data plane based on a `Gateway`. - This may require, for example, a user to manually create a `Deployment` and `Service` for the data plane, then link this to a `Gateway`. ## Design -This GEP does not introduce new API fields, but rather standardizes how implementations should behave when implementing the existing API. +This GEP both introduces new API fields, and standardizes how implementations should behave when implementing the existing API. ### Automated Deployments @@ -57,22 +56,21 @@ spec: ``` With this configuration, an implementation: + * MUST mark the Gateway as `Programmed` and provide an address in `Status.Addresses` where the Gateway can be reached on each configured port. -* MUST label all generated resources (Service, Deployment, etc) with `gateway.networking.k8s.io/metadata.name: my-gateway` (where `my-gateway` is the name of the Gateway resource). +* MUST label all generated resources (Service, Deployment, etc) with `gateway.networking.k8s.io/gateway-name: my-gateway` (where `my-gateway` is the name of the Gateway resource). * MUST provision generated resources in the same namespace as the Gateway if they are namespace scoped resources. * Cluster scoped resources are not recommended. -* MUST name all generated resources `my-gateway-example` (`-`). +* SHOULD name all generated resources `my-gateway-example` (`-`). This is not simply `NAME` to reduce the chance of conflicts with existing resources. - -### Manual Deployments - -Managing "Manual Deployments" is currently out of scope for this GEP, but may be added in the future. + Where required, this can also serve as the prefix for the object. ### Customizations With any in-cluster deployment, customization requirements will arise. Some common requirements would be: + * `Service.spec.type`, to control whether a service is a `ClusterIP` or `LoadBalancer`. * IP in the Service to assign to it. * Arbitrary labels and annotations on generated resources. @@ -81,6 +79,8 @@ Some common requirements would be: * Service `externalTrafficPolicy` * Affinity rules. +This GEP currently only aims to solve a subset of these concerns. Additional concerns may be addressed in future revisions or other GEPs. + #### Gateway Type This is handled by [GEP-1651](https://github.com/kubernetes-sigs/gateway-api/pull/1653), so won't be described here. @@ -138,8 +138,8 @@ Note this may mean an annotation intended for a `Service` may end up on a `Deplo This is typically not a concern; however, if an implementation is aware of specific meanings of certain labels or annotations, they MAY exclude these from irrelevant resources. -This is intended to support integration with the kitchen-sync of Kubernetes extensions which rely on labels and annotations, -such as as prometheus scraping, object grouping/organization, etc. +This is intended to clearly identify resources associatd with a specific application, environment, or Gateway. +Additionally, it can be used support integration with the kitchen-sync of Kubernetes extensions which rely on labels and annotations. #### Arbitrary Customization @@ -149,9 +149,10 @@ This can be utilized for the remainder of customizations. ### Resource Attachment -Resources Generated in response to the `Gateway` are guaranteed to have two attributes: -* A `gateway.networking.k8s.io/metadata.name: ` label. -* A name `-`. +Resources Generated in response to the `Gateway` will have two attributes: + +* A `gateway.networking.k8s.io/gateway-name: ` label. +* A name `-`. This format is not strictly required for implementations, but strongly recommended for consistency in attachment. The generated resources MUST be in the same namespaces as the `Gateway`. @@ -160,8 +161,8 @@ as some implementations may have different cleanup mechanisms. These can be relied on to attach resources to. -While "Policy attachment" in the gateway-api would use attachment to the actual `Gateway` resource itself, -many existing things attach only to resources like `Deployment` or `Service`. +While "Policy attachment" in Gateway API would use attachment to the actual `Gateway` resource itself, +many existing resources attach only to resources like `Deployment` or `Service`. An example using these: ```yaml @@ -223,18 +224,18 @@ type GatewayInfrastructure struct { // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. // - // If Labels is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. - // - // Support: Implementation-specific + // If Labels is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. + // + // Support: Extended Labels map[string]string `json:"labels,omitempty"` // Annotations that should be applied to any resources created in response to this Gateway. // // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. // - // If Annotations is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. - // - // Support: Implementation-specific + // If Annotations is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. + // + // Support: Extended Annotations map[string]string `json:"annotations,omitempty"` // ParametersRef is a reference to a resource that contains the configuration // parameters corresponding to the Gateway. This is optional if the @@ -242,11 +243,11 @@ type GatewayInfrastructure struct { // // ParametersRef can reference a standard Kubernetes resource, i.e. ConfigMap, // or an implementation-specific custom resource. The resource must be namespace-scoped - // and live in the same namespace. + // and live in the same namespace. + // + // If ParametersRef is set on both a Gateway and the corresponding GatewayClass, the merging + // behavior between the two are implementation-specific. // - // If ParametersRef is set on both a Gateway and the corresponding GatewayClass, the merging - // behavior between the two are implementation-specific. - // // Support: Implementation-specific // // +optional @@ -259,19 +260,19 @@ type GatewayClassInfrastructure struct { // // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. - // - // If Labels is set on the Gateway as well, the labels are merged with the Gateway's labels taking precedence. // - // Support: Implementation-specific + // If Labels is set on the Gateway as well, the labels are merged with the Gateway's labels taking precedence. + // + // Support: Extended Labels map[string]string `json:"labels,omitempty"` // Annotations that should be applied to any resources created in response to this GatewayClass. // // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. // - // If Annotations is set on the Gateway as well, the labels are merged with the Gateway's annotations taking precedence. - // - // Support: Implementation-specific + // If Annotations is set on the Gateway as well, the labels are merged with the Gateway's annotations taking precedence. + // + // Support: Extended Annotations map[string]string `json:"annotations,omitempty"` ... } @@ -292,3 +293,7 @@ type LocalParametersReference struct { Name string `json:"name"` } ``` + +## Future Work + +* Allow various policies, [such as HPA](https://github.com/kubernetes-sigs/gateway-api/discussions/1355), to attach directly to `Gateway` rather than just `Deployment`. From 1c01615fdb97631592940fc27d416d6c30ac80ad Mon Sep 17 00:00:00 2001 From: John Howard Date: Mon, 18 Sep 2023 07:57:53 -0700 Subject: [PATCH 5/6] Remove GC and clarify address --- geps/gep-1762.md | 70 ++++++------------------------------------------ 1 file changed, 8 insertions(+), 62 deletions(-) diff --git a/geps/gep-1762.md b/geps/gep-1762.md index 3f232585e5..6e6777272a 100644 --- a/geps/gep-1762.md +++ b/geps/gep-1762.md @@ -87,7 +87,10 @@ This is handled by [GEP-1651](https://github.com/kubernetes-sigs/gateway-api/pul #### Gateway IP -For setting a specific IP address in a generated `Service`, the `.Spec.Addresses` field can be used. +This section just clarifies and existing part of the spec, how to handle `.spec.addresses` for in-cluster implementations. +Like all other Gateway types, this should impact the address the `Gateway` is reachable at. + +For implementations using a `Service`, this means the `clusterIP` or `loadBalancerIP` (depending on the `Service` type). For example: @@ -109,10 +112,6 @@ spec: This would generate a `Service` with `clusterIP` or `loadBalancerIP`, depending on the Service type. -This follows the same behavior as out-of-cluster Gateway implementations. - -Note: this differs from the selection of "manual deployment", which doesn't use `IPAddress` type. -Instead, these Gateways attach to some existing infrastructure (such as Service) which provides an address. #### Labels and Annotations @@ -141,6 +140,8 @@ exclude these from irrelevant resources. This is intended to clearly identify resources associatd with a specific application, environment, or Gateway. Additionally, it can be used support integration with the kitchen-sync of Kubernetes extensions which rely on labels and annotations. +Validation will be added to prevent any usage with `gateway.networking.k8s.io/` prefix, to avoid conflicts with `gateway.networking.k8s.io/gateway-name` or other future additions. + #### Arbitrary Customization GEP-1867 introduces a new `infrastructure` field, which allows customization of some common configurations (version, size, etc) @@ -224,74 +225,19 @@ type GatewayInfrastructure struct { // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. // - // If Labels is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. - // // Support: Extended + // +kubebuilder:validation:MaxItems=16 Labels map[string]string `json:"labels,omitempty"` // Annotations that should be applied to any resources created in response to this Gateway. // // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. // - // If Annotations is set on the GatewayClass as well, the labels are merged with the Gateway's labels taking precedence. - // // Support: Extended + // +kubebuilder:validation:MaxItems=16 Annotations map[string]string `json:"annotations,omitempty"` - // ParametersRef is a reference to a resource that contains the configuration - // parameters corresponding to the Gateway. This is optional if the - // controller does not require any additional configuration. - // - // ParametersRef can reference a standard Kubernetes resource, i.e. ConfigMap, - // or an implementation-specific custom resource. The resource must be namespace-scoped - // and live in the same namespace. - // - // If ParametersRef is set on both a Gateway and the corresponding GatewayClass, the merging - // behavior between the two are implementation-specific. - // - // Support: Implementation-specific - // - // +optional - ParametersRef *LocalParametersReference `json:"parametersRef,omitempty"` ... } - -type GatewayClassInfrastructure struct { - // Labels that should be applied to any resources created in response to this GatewayClass. - // - // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. - // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. - // - // If Labels is set on the Gateway as well, the labels are merged with the Gateway's labels taking precedence. - // - // Support: Extended - Labels map[string]string `json:"labels,omitempty"` - // Annotations that should be applied to any resources created in response to this GatewayClass. - // - // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. - // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. - // - // If Annotations is set on the Gateway as well, the labels are merged with the Gateway's annotations taking precedence. - // - // Support: Extended - Annotations map[string]string `json:"annotations,omitempty"` - ... -} - -// LocalParametersReference identifies an API object containing controller-specific -// configuration resource within the cluster. -type LocalParametersReference struct { - // Group is the group of the referent. - Group Group `json:"group"` - - // Kind is kind of the referent. - Kind Kind `json:"kind"` - - // Name is the name of the referent. - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - Name string `json:"name"` -} ``` ## Future Work From c6a7dcbb9bd3966bbcdc21c9fb0a07b1e55d3ae8 Mon Sep 17 00:00:00 2001 From: John Howard Date: Mon, 18 Sep 2023 15:11:20 -0700 Subject: [PATCH 6/6] Address comments --- geps/gep-1762.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/geps/gep-1762.md b/geps/gep-1762.md index 6e6777272a..1a890cdb4b 100644 --- a/geps/gep-1762.md +++ b/geps/gep-1762.md @@ -17,7 +17,7 @@ Related discussions: ## Goals * Provide prescriptive guidance for how in-cluster implementations should behave. -* Provide requirements (tested by conformance) for how in-cluster implementations should behave. +* Provide requirements for how in-cluster implementations should behave. Note that some changes will be suggestions, while others will be requirements. @@ -137,7 +137,7 @@ Note this may mean an annotation intended for a `Service` may end up on a `Deplo This is typically not a concern; however, if an implementation is aware of specific meanings of certain labels or annotations, they MAY exclude these from irrelevant resources. -This is intended to clearly identify resources associatd with a specific application, environment, or Gateway. +This is intended to clearly identify resources associated with a specific application, environment, or Gateway. Additionally, it can be used support integration with the kitchen-sync of Kubernetes extensions which rely on labels and annotations. Validation will be added to prevent any usage with `gateway.networking.k8s.io/` prefix, to avoid conflicts with `gateway.networking.k8s.io/gateway-name` or other future additions. @@ -160,8 +160,7 @@ The generated resources MUST be in the same namespaces as the `Gateway`. Implementations MAY set `ownerReferences` to the `Gateway` in most cases, as well, but this is not required as some implementations may have different cleanup mechanisms. - -These can be relied on to attach resources to. +The `gateway.networking.k8s.io/gateway-name` label and standardize resource naming format can be relied on to attach resources to. While "Policy attachment" in Gateway API would use attachment to the actual `Gateway` resource itself, many existing resources attach only to resources like `Deployment` or `Service`. @@ -225,16 +224,20 @@ type GatewayInfrastructure struct { // For implementations creating other Kubernetes objects, this should be the `metadata.labels` field on resources. // For other implementations, this refers to any relevant (implementation specific) "labels" concepts. // + // An implementation may chose to add additional implementation-specific labels as they see fit. + // // Support: Extended - // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:MaxItems=8 Labels map[string]string `json:"labels,omitempty"` // Annotations that should be applied to any resources created in response to this Gateway. // // For implementations creating other Kubernetes objects, this should be the `metadata.annotations` field on resources. // For other implementations, this refers to any relevant (implementation specific) "annotations" concepts. // + // An implementation may chose to add additional implementation-specific annotations as they see fit. + // // Support: Extended - // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:MaxItems=8 Annotations map[string]string `json:"annotations,omitempty"` ... }