From 52abf7d02fe89ef5756d05ffa817c4cd7d3498af Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Thu, 2 Feb 2023 14:42:54 +0100 Subject: [PATCH 01/17] rfc: RateLimitPolicy v2 --- rfcs/0000-rlp-v2.md | 440 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 440 insertions(+) create mode 100644 rfcs/0000-rlp-v2.md diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md new file mode 100644 index 00000000..c943b90d --- /dev/null +++ b/rfcs/0000-rlp-v2.md @@ -0,0 +1,440 @@ +# RFC RateLimitPolicy v2 + +- Feature Name: `rlp-v2` +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- RFC PR: [Kuadrant/architecture#0000](https://github.com/Kuadrant/architecture/pull/0000) +- Issue tracking: [Kuadrant/architecture#0000](https://github.com/Kuadrant/architecture/issues/0000) + +## Summary +[summary]: #summary + +Proposal of new API for the Kuadrant's `RateLimitPolicy` (RLP) CRD, for improved UX. + +## Motivation +[motivation]: #motivation + +The [`RateLimitPolicy`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimitPolicy) API (v1beta1), particularly its [`RateLimit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimit) type used in `ratelimitpolicy.spec.rateLimits`, designed in part to fit the underlying implementation based on the Envoy [Rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) filter, has been proven to be (i) _unnecessarily complex_, as well as (ii) possibly _limiting for the extension of the API_ either beyond this specific implementation and/or for supporting use cases of rate limiting not contemplated in the original designs. + +Users of the `RateLimitPolicy` will immediately recognize elements of Envoy's Rate limit API in the definitions of the `RateLimit` type, with almost 1:1 correspondence between the [`Configuration`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Configuration) and [`Rule`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Rule) types and their counterparts in the Envoy configuration. Although compatibility between those continue to be desired, the leakage of such implementation details to the level of the API could otherwise be avoided to provide a better abstraction for the context in vogue, where activators ("matchers") and payload ("descriptors") are defined by users in a same document instead of in different configurations of different components. + +Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Limit) type – used as well in the `RateLimitPolicy`'s `RateLimit` – implies presently a logical relationship between its inner concepts – i.e. conditions and variables on one side, and limits themselves on the other – that otherwise could be shaped in a different manner, to provide (i) clearer understanding of the meaning of these concepts by the user as well as (ii) to avoid repetition. + +### Goals + +1. Decouple the API from the underlying implementation - i.e. provide a more generic abstraction +2. Simplify the API - i.e. DRY, straight to the point (for defining rate limits) +3. Consistency with Kuadrant's [AuthPolicy](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#AuthPolicy) API - i.e. same language, similar UX + +### Current WIP to consider + +1. Inheritance of the network matching rules from the upstream network object (https://github.com/Kuadrant/architecture/pull/4) +2. Implement `skip_if_absent` for the RequestHeaders action (https://github.com/Kuadrant/wasm-shim/issues/29) +3. Rate Limiting across clusters ([doc](https://docs.google.com/document/d/1pqCODRAkNUTLB6lJfRWcv3d2Hlj9WqsGySmjrP707Vk/edit#heading=h.nzpgr3pef6uy)) + +## Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +### Examples of RLPs based on the new API + +Given the following HTTPRoute: + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: HTTPRoute +metadata: + name: toystore +spec: + parentRefs: + - name: istio-ingressgateway + namespace: istio-system + hostnames: ["*.toystore.com"] + rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + backendRefs: + - name: toystore + port: 80 +``` + +These are examples of RLPs targeting the route: + +**Example 1.** Minimal example - unconditional, unqualified rate limiting + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - rates: + - limit: 5 + unit: minute +``` + +**Example 2.** All you can get - conditions, counter qualifiers, multiple rates, and variable increments + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: writers + when: + - selector: context.request.http.method + operator: eq + value: POST + rates: + - limit: 5 + duration: 1 + unit: minute + - limit: 100 + duration: 12 + unit: hour + counters: + - auth.identity.username + increment: + value: 1 # default + + - name: readers + when: + - selector: context.request.http.method + operator: eq + value: GET + rates: + - limit: 50 + duration: 1 + unit: minute + counters: + - auth.identity.username + increment: + value: 1 + + - name: read-expensive + when: + - selector: context.request.http.method + operator: eq + value: GET + - selector: context.request.http.path + operator: eq + value: /toys/expensive + rates: + - limit: 10 + duration: 1 + unit: minute + increment: + value: 2 # each hit artificially made twice as expensive without reseting the counters +``` + +### Comparison to current RateLimitPolicy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CurrentNewReason
1:1 relation between Limit (object) and the actual Rate limit (value)Rate becomes a detail of Limit where each limit may define one or more rates (1:N) +
    +
  • It allows to reuse when conditions and counters for groups of rate limits
    • +
+
Parsed conditionsStructured when condition fields composed of { selector, operator, value } + +
configurations field as a list of "variables assignments" - direct exposure of Envoy's RL "actions"No configurations field - descriptor actions composed from selectors used in the limit definitions only (when[].selector and counters[]) +
    +
  • Abstract the Envoy-specific concepts of "actions" and "descriptors"
    • +
  • No risk of mismatching descriptors keys between "actions" and actual usage in the limits
    • +
  • No (useless?) generic descriptors (e.g. "limited = 1")
    • +
  • Source value of the selectors defined from an implicit "context" data structure
    • +
+
Key-value descriptorsStructured descriptors from a "context" data structure + +
maxValuelimit
secondsduration and unit +
    +
  • Support for more units beyond seconds
    • +
  • duration: 1 by default
    • +
+
Fixed incrementincrement: {VALUE} +
    +
  • Flexibility to change the limits without reseting the counters
    • +
  • Placeholder for more complex RL systems in the future – e.g. dynamic-RL, %, 𝑓(), etc
    • +
  • Migration path from other RL solutions, e.g. 3scale (?)
    • +
+
+ +## Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +### Composing RL descriptor actions + +By completely dropping out the `configurations` field from the RLP, the `actions` to composing the RL _descriptor actions_ is now done based essentially on the selectors listed in the `when` conditions and `counters`. + +Each selector is a direct reference to a path within a _well-known data structure_ that holds the `context` (L4 and L7 data of the original request handled by the proxy), as well as occasionally the `auth` data (dynamic metadata exported by the external authorization filter and injected by the proxy into the rate-limit filter). + +#### The well-known data structure for user-defined RLP selectors and Kuadrant-generated RL descriptor actions + +The well-known data structure for building RL descriptor actions out of selectors in a RLP resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. + +To keep compatibility with the [Envoy Rate Limit protocol](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. + +```yaml +context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type + source: + address: … + service: … + … + destination: + address: … + service: … + … + request: + http: + host: … + path: … + method: … + headers: {…} + + auth: # Dynamic metadata exported by the external authorization service + + ratelimit: # Envoy's Rate Limit `RateLimitRequest` type + domain: … # generated by the Kuadrant controller + descriptors: {…} # descriptors configured by the user directly in the proxy (not generated by the Kuadrant controller, if allowed) + hitsAddend: … # only in case we want to allow users to refer to this value in a policy +``` + +#### Mechanics of generating RL descriptor actions + +From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure. While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, + +The user story: +> _Whenever the context is a HTTP request to the path (`context.request.http.path`) that is equal to `"/toys"`, I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ + +Materializes as the following RLP: + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: toys-path + when: + - selector: context.request.http.path + operator: eq + value: /toys + rates: + - limit: 50 + duration: 1 + unit: second + counters: + - auth.identity.username +``` + +The following selectors are to be interpreted: +- `context.request.http.path` +- `auth.identity.username` + +The RLP controller uses a map to translate each selector into its corresponding descriptor action. Roughly described: + +``` +context.source.address → source_cluster(...) (?) +context.source.service → source_cluster(...) (?) +context.destination... → destination_cluster(...) +context.destination... → destination_cluster(...) +context.request.http. → request_headers(header_name: ":") +context.request... → ... +auth. → metadata(key: "envoy.filters.http.ext_authz", path: ) +ratelimit.domain → +``` + +...to yield effectively: + +```yaml +rate_limits: + - actions: + - request_headers: + descriptor_key: "context.request.http.path" + header_name: ":path" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" +``` + +## Drawbacks +[drawbacks]: #drawbacks + + + +[TODO] + +## Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +### Possible variations for the selectors (conditions and counter qualifiers) + +The main drivers behind the proposed design for the selectors (conditions and counter qualifiers), based on (i) structured condition expressions composed of fields `selector`, `operator`, and `value`, and (ii) `when` conditions and `counters` separated in two distinct fields (variation "C" below), are: +1. consistency with the Authorino `AuthConfig` API, which also specifies [`when`](https://github.com/Kuadrant/authorino/blob/main/docs/features.md#common-feature-conditions-when) conditions expressed in `selector`, `operator`, and `value` fields; +2. explicit user intent, without subtle distinction of meaning based on presence of optional fields. + +Nonetheless here are a few alternative variations to consider: + + + + + + + + + + + + + + + + + + + + + +
Structured condition expressionsParsed condition expressions
Single field + A + 
+selectors:
+  - selector: context.request.http.method
+    operator: eq
+    value: GET
+  - selector: auth.identity.username
+ 		+ B + 
+selectors:
+  - context.request.http.method == "GET"
+  - auth.identity.username
+
Distinct fields + C ⭐️ + 
+when:
+  - selector: context.request.http.method
+    operator: eq
+    value: GET
+counters:
+  - auth.identity.username
+ 		+ D + 
+when:
+  - context.request.http.method == "GET"
+counters:
+  - auth.identity.username
+
+ +⭐️ Variation adopted for the examples and (so far) final design proposal. + +## Prior art +[prior-art]: #prior-art + + + +[TODO] + +## Unresolved questions +[unresolved-questions]: #unresolved-questions + +1. What does this mean with _defaults_ and _overrides_? +2. Do we need anything here that points to shared counters across clusters/Limitador instances or does that belong to different layer (`Limitador`, `Kuadrant` CRDs, MCTC)? +3. What condition `operator`s do we need (e.g. `eq`, `neq`, `exists`, `nexists`, `matches`)? + +## Future possibilities +[future-possibilities]: #future-possibilities + + + +[TODO] From 36e828f38a6dfa38cda823f22dcc729d70e94060 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Wed, 22 Feb 2023 17:35:34 +0100 Subject: [PATCH 02/17] add spec.limits.matches --- rfcs/0000-rlp-v2.md | 466 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 402 insertions(+), 64 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index c943b90d..0f5f7bb1 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -10,7 +10,7 @@ Proposal of new API for the Kuadrant's `RateLimitPolicy` (RLP) CRD, for improved UX. -## Motivation +### Motivation [motivation]: #motivation The [`RateLimitPolicy`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimitPolicy) API (v1beta1), particularly its [`RateLimit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimit) type used in `ratelimitpolicy.spec.rateLimits`, designed in part to fit the underlying implementation based on the Envoy [Rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) filter, has been proven to be (i) _unnecessarily complex_, as well as (ii) possibly _limiting for the extension of the API_ either beyond this specific implementation and/or for supporting use cases of rate limiting not contemplated in the original designs. @@ -21,28 +21,54 @@ Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/k ### Goals -1. Decouple the API from the underlying implementation - i.e. provide a more generic abstraction +1. Decouple the API from the underlying implementation - i.e. provide a more generic and more user-friendly abstraction 2. Simplify the API - i.e. DRY, straight to the point (for defining rate limits) 3. Consistency with Kuadrant's [AuthPolicy](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#AuthPolicy) API - i.e. same language, similar UX ### Current WIP to consider -1. Inheritance of the network matching rules from the upstream network object (https://github.com/Kuadrant/architecture/pull/4) +1. A single Policy scoped to HTTPRoutes and HTTPRouteRule (https://github.com/Kuadrant/architecture/pull/4) 2. Implement `skip_if_absent` for the RequestHeaders action (https://github.com/Kuadrant/wasm-shim/issues/29) 3. Rate Limiting across clusters ([doc](https://docs.google.com/document/d/1pqCODRAkNUTLB6lJfRWcv3d2Hlj9WqsGySmjrP707Vk/edit#heading=h.nzpgr3pef6uy)) +### Highlights + +- `spec.rateLimits` replaced by `spec.limits` +- `spec.rateLimits.limits` replaced by `spec.limits.rates` +- `spec.rateLimits.limits.maxValue` replaced by `spec.limits.rates.limit` +- `spec.rateLimits.limits.seconds` replaced by `spec.limits.rates.duration` + `spec.limits.rates.unit` +- `spec.rateLimits.limits.conditions` replaced by `spec.limits.when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) +- `spec.rateLimits.limits.variables` replaced by `spec.limits.counters`, based on _well-known selectors_ +- `spec.rateLimits.rules` replaced by `spec.limits.matches`, where each match is a perfect match to one of the `HTTPRouteMatch`es of the targeted `HTTPRoute` (`httproute.spec.rules.matches`) – complies with [architecture#4](https://github.com/Kuadrant/architecture/pull/4) +- `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.matches` +- Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted `HTTPRouteMatch` +- `spec.limits.increment` introduced + +For detailed differences between current and vew RLP API, see [Comparison to current RateLimitPolicy](#comparison-to-current-ratelimitpolicy). + ## Guide-level explanation [guide-level-explanation]: #guide-level-explanation ### Examples of RLPs based on the new API -Given the following HTTPRoute: +Given the following network resources: ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: Gateway +metadata: + name: istio-ingressgateway + namespace: istio-system +spec: + gatewayClassName: istio + listeners: + - hostname: "*.toystore.com" +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 kind: HTTPRoute metadata: name: toystore + namespace: toystore spec: parentRefs: - name: istio-ingressgateway @@ -63,44 +89,70 @@ spec: port: 80 ``` -These are examples of RLPs targeting the route: +The following are examples of RLPs targeting the route and the gateway. + +#### Targeting routes and route matching rules + +##### Example 1. Minimal example - network resource targeted entirely without filtering, unconditional and unqualified rate limiting -**Example 1.** Minimal example - unconditional, unqualified rate limiting +In this example, all traffic to `*.toystore.com` will be limited to 5 rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore + name: toystore-simple-infra-rl spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - rates: + - rates: # at least one rate limit required - limit: 5 - unit: minute + unit: second ``` -**Example 2.** All you can get - conditions, counter qualifiers, multiple rates, and variable increments +##### Example 2. Specific route matching rule targeted, conditions, counter qualifiers and multiple rates + +In this example, a distinct limit is associated to each individual matching rule of the targeted HTTPRoute, using the `matches` field for fine-grained filtering. In both cases, the rate limits are to be enforced per username and only in case the user is not an admin. + +For each matching rule in the RLP, the RLP controller will try to find a matching rule in the HTTPRoute that is an identical match to it and bind the two matching rules together; the first identical match prevails. In case there is no identical match in the HTTPRoute, the RLP is considered invalid. In case there is more than one matching rule specified in the RLP that is an identical match to the same matching rule in the HTTPRoute, the first matching rule on the list in the RLP is bound to its identical match in the HTTPRoute, thus "shadowing" any other rule on the list that is also an identical match. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore + name: toystore-per-endpoint-per-user spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: writers + - name: readers + matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + rates: + - limit: 50 + duration: 1 + unit: minute + counters: + - auth.identity.username when: - - selector: context.request.http.method - operator: eq - value: POST + - selector: auth.identity.group + operator: neq + value: admin + + - name: writers + matches: + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 5 duration: 1 @@ -110,39 +162,189 @@ spec: unit: hour counters: - auth.identity.username - increment: - value: 1 # default + when: + - selector: auth.identity.group + operator: neq + value: admin +``` - - name: readers +##### Example 3. Route filtering using `when` conditions + +`when` conditions are preferably to be used for special cases of conditional filtering based on values other than attributes of the route that could otherwise be specified using the `matches` field. However, in some cases such as where the condition is not related to the HTTPRoute (e.g. see [Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates) above; filterring based on hostname, etc) or a temporary measure while the target object misses the desired matching rule, the `when` conditions remains an option. + +In this example, a special limit with one rate limit of 150 rps is set for `GET` requests to the `/toys/special` path. Because the `toystore` HTTPRoute is unaltered, the special case is defined using the `when` conditions field. + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-special-toys +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: read-special + matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + rates: + - limit: 150 + unit: second when: - - selector: context.request.http.method + - selector: context.request.http.path operator: eq - value: GET + value: /toys/special +``` + +By using the route rule as-is, simply `GET /toys*` instead of more specific `GET /toys/special`, the rate limit filter (wasm filter) will continue invoking the rate limit service (Limitador) for all requests that match HTTP method equal to `GET` and path prefix equal to `/toys`, including requests to paths other than `/toys/special`, but the limit will only be enforced when the path fully matches `/toys/special`, as specified in the `when` condition. + +On one hand, this makes it easy to set limits when either the HTTPRoute cannot be modified to include a special matching rule or as a shortcut in cases where multiple HTTPRoutes would have to be touched, such when targeting a Gateway with special conditions based on attributes of the route. On the other hand however, users might miss information in the status in a scenario where the status of rate limiting is reported at the level of the route rule or rule matcher. Effectively, the route rule `GET /toys*` might be reported as rate limited to '150 rps' when that is actually only the case of requests to `GET /toys/special`. These special conditions of the limit definition need therefore to be included in the status. + +In the case of existing other limit definitions targeting the `GET /toys*` matching rule of the `toystore` HTTPRoute, because a merge strategy is expect to take into account the `matches` field as part of the qualification of the limit, there should be no problem of multiple simultaneous limits enforced to the same route rules, differenciated only by special conditions. Nevertheless, to avoid dealing of complex status reports including too many special conditions associated with a limit, users are instead encouraged to favor altering the HTTPRoutes for additional route rules that can be referred in the `matches` field preferably. + +##### Example 4. Route filtering by refining the HTTPRoute + +To achieve the same goal as state in [Example 3](#example-3-route-filtering-using-when-conditions) yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `matches` field instead of the `when` conditions field. + +New matching rule added to the HTTPRoute: + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: HTTPRoute +metadata: + name: toystore + namespace: toystore +spec: + parentRefs: + - name: istio-ingressgateway + namespace: istio-system + hostnames: ["*.toystore.com"] + rules: + - matches: + - path: # new matching rule added so it can be targeted by the RLP + type: Exact + value: "/toys/special" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + backendRefs: + - name: toystore + port: 80 +``` + +RateLimitPolicy: + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-special-toys +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: read-special + matches: + - path: + type: Exact + value: "/toys/special" + method: GET + rates: + - limit: 150 + unit: second +``` + +##### Example 5. One limit, two matches + +In this example, both route matching rules, `GET /toys*` and `POST /toys*`, are targeted by the same limit. This will cause the limit to be bound to the two HTTPRouteMatches, effectively applying 50rpm per username, regardless of the HTTP method `GET` or `POST`, at requests with path prefix equal to `/toys`. I.e. the rules are OR'ed, just like in the HTTPRoute itself. The same unique hash identifier is associated to both route rules. + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-per-endpoint-per-user +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 50 duration: 1 unit: minute counters: - auth.identity.username - increment: - value: 1 +``` - - name: read-expensive - when: - - selector: context.request.http.method - operator: eq - value: GET - - selector: context.request.http.path - operator: eq - value: /toys/expensive +##### Example 6. Dynamic rate with flexible increments + +By setting the increment of the limit, the rate can vary with no need to reset the counters. + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-dynamic +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: dynamic-rate rates: - - limit: 10 - duration: 1 - unit: minute + - limit: 100 + unit: second increment: value: 2 # each hit artificially made twice as expensive without reseting the counters ``` +#### Targeting Gateways + +Targeting a Gateway is a shortcut to targeting individually each HTTPRoute pointing to the gateway, without any filtering based on the `matches` field. + +> **Note:** it is hard to give any additional meaning and context to this without going into [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). + +##### Example 7. Targeting the Gateway + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-simple-infra-rl +spec: + targetRef: + group: gateway.networking.k8s.io + kind: Gateway + name: istio-ingressgateway + limits: + - rates: + - limit: 5 + unit: second +``` + ### Comparison to current RateLimitPolicy @@ -155,8 +357,8 @@ spec: - - + + - - + + - - + + - + - - - + + + + + + + + - - + + - - + + + + + + + + + + + + - + - + - + @@ -660,7 +715,7 @@ spec: ## Reference-level explanation [reference-level-explanation]: #reference-level-explanation -By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus artificial generic conditions used to match the specific route rule (when `matches` is specified). +By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus artificial generic conditions used to match the specific route rule (when the `rules` is non-empty or implicit). ### Well-known selectors @@ -668,7 +723,7 @@ Each selector is a direct reference to a path within a well-known data structure The well-known data structure for building RL descriptor actions resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. -To keep compatibility with the [Envoy Rate Limit protocol](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. +To keep compatibility with the [Envoy Rate Limit API](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. ```yaml context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type @@ -687,12 +742,12 @@ context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type method: … headers: {…} - auth: # Dynamic metadata exported by the external authorization service +auth: # Dynamic metadata exported by the external authorization service - ratelimit: # Envoy's Rate Limit `RateLimitRequest` type - domain: … # generated by the Kuadrant controller - descriptors: {…} # descriptors configured by the user directly in the proxy (not generated by the Kuadrant controller, if allowed) - hitsAddend: … # only in case we want to allow users to refer to this value in a policy +ratelimit: # Envoy's Rate Limit `RateLimitRequest` type + domain: … # generated by the Kuadrant controller + descriptors: {…} # descriptors configured by the user directly in the proxy (not generated by the Kuadrant controller, if allowed) + hitsAddend: … # only in case we want to allow users to refer to this value in a policy ``` ### Mechanics of generating RL descriptor actions @@ -700,9 +755,9 @@ context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure ([_well-known selectors_](#well-known-selectors)). While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, The user story: -> _Whenever the context is an HTTP request sent to `dolls.toystore.com` (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ +> _Whenever the context is an HTTP request sent to `dolls.toystore.com` hostname (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ -Materializes as the following RLP: +...materializes as the following RLP: ```yaml apiVersion: kuadrant.io/v2beta1 @@ -728,11 +783,11 @@ spec: - auth.identity.username ``` -The following selectors are to be interpreted: +The following selectors are to be interpreted by the RLP controller: - `context.request.http.host` - `auth.identity.username` -The RLP controller uses a map to translate each selector into its corresponding descriptor action. Roughly described: +The RLP controller uses a map to translate each selector into its corresponding descriptor action. (Roughly described:) ``` context.source.address → source_cluster(...) # TBC @@ -764,13 +819,13 @@ rate_limits: key: "username" ``` -### Artificial Limitador condition for `matches` +### Artificial Limitador condition for `rules` -For each limit, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when that one filterred matching rule is honoured to serve the request. This can be implemented with a 2-step procedure: -1. generate an unique identifier for the limit – e.g. by applying any sufficiently entropic hash function of choice to the `matches` block of the limit; -2. associate a `generic_key` type descriptor action to the each `HTTPRouteMatch` targeted by the limit. +For each limit with non-empty (or implicit) `rules` field, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when the filterred rules are honoured when serving the request. This can be implemented with a 2-step procedure: +1. generate an unique identifier for the limit – e.g. by applying any sufficiently entropic hash function of choice to the `rules` block of the limit; +2. associate a `generic_key` type descriptor action to the each `HTTPRouteRule` targeted by the limit. -For example, given the following RLP ([Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates)): +For example, given the following RLP: ```yaml apiVersion: kuadrant.io/v2beta1 @@ -783,12 +838,16 @@ spec: kind: HTTPRoute name: toystore limits: - - name: readers + - name: toys matches: - path: type: PathPrefix value: "/toys" method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 50 duration: 1 @@ -798,12 +857,11 @@ spec: operator: neq value: admin - - name: writers + - name: assets matches: - path: type: PathPrefix - value: "/toys" - method: POST + value: "/assets/" rates: - limit: 5 duration: 1 @@ -814,7 +872,7 @@ spec: value: admin ``` -Apart from the descriptor action (associated with both routes): +Apart from the following descriptor action associated with both routes: ```yaml - metadata: @@ -837,15 +895,15 @@ auth.identity.group != "admin" The following additional artificial descriptor actions will be generated: ```yaml -# associated with route GET /toys* +# associated with route rule GET|POST /toys* - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" # SHA256 hashing of [{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"}] + descriptor_value: "c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] -# associated with route POST /toys* +# associated with route rule /assets/* - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" # SHA256 hashing of [{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}] + descriptor_value: "643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] ``` ...and their corresponding Limitador conditions. @@ -854,27 +912,37 @@ In the end, the following Limitador configuration is yielded: ```yaml - conditions: - - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - context.request.http.route_matcher == "c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - auth.identity.group != "admin" max_value: 50 seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" + - context.request.http.route_matcher == "643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" - auth.identity.group != "admin" max_value: 5 seconds: 60 namespace: "*.toystore.com" ``` -The route matcher hash identifiers can be qualified with a plain identifier of the RLP itself (`namespace/name`) and limit where it is defined (`name` of the limit when available, index in the array of limits otherwise), thus making the identifier unique to scope of the entire cluster. E.g.: the two unique identifiers from above, prefixed with the unique limit qualifier, become respectively `toystore/toystore-non-admin-users/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05` and `toystore/toystore-non-admin-users/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d`. This has a consequence to the readability of the identifier, but also and more importantly it ensures uniqueness of the counters in Limitador. By qualifying (or salting) the identifiers, two limits or two RLPs that happen to target the same `HTTPRouteMatches` will not register the same counters in Limitador, but be treated as independent ones instead. +The limit-to-route rule matcher identifiers can be qualified with a plain identifier of the RLP itself (`namespace/name`) and the limit where it is defined (`name` of the limit when available, index in the array of limits otherwise), thus making the identifier unique to the scope of the entire cluster. + +E.g.: the two unique identifiers from above, prefixed with the unique limit qualifier, become respectively `toystore/toystore-non-admin-users/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f` and `toystore/toystore-non-admin-users/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66`. + +This has a consequence to the readability of the identifier, but also and more importantly it ensures uniqueness of the counters in Limitador. By qualifying (or salting) the identifiers, two limits or two RLPs that happen to target the same HTTPRouteRules will not register the same counters in Limitador, but be treated as independent ones instead. + +### Support in wasm shim and Envoy RL API + +This proposal tries to keep compatibility with the [Envoy API for rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) and does not introduce any new requirement that otherwise would require the use of [wasm shim](https://github.com/Kuadrant/wasm-shim) to be implemented. + +In the case of implementation of this proposal in the wasm shim, all types of matchers supported by the [HTTPRouteMatch](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRouteMatch) type of Gateway API must be also supported in the `rate_limit_policies.gateway_actions.rules` field of the [wasm plugin configuration](https://github.com/Kuadrant/kuadrant-operator/blob/faf41ff6b08df27946a663c34a5736476578dea5/pkg/rlptools/wasm_utils.go#L109). These include matchers based on path (prefix, exact), headers, query string parameters and method. ## Drawbacks [drawbacks]: #drawbacks -**Two types of conditions – `matches` and `when` conditions**
-Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and capable of expressing different sets of rules (HTTPRouteMatch-related rules vs. HTTP request-related and non HTTP request-related rules), an overlap between these two types and ways of representing conditions does exist. +**Two types of conditions – `rules` and `when` conditions**
+Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and capable of expressing different sets of rules (HTTPRouteRule-targeting rules vs. HTTP request-related and non HTTP request-related "soft" conditions), an overlap between these two types and ways of representing conditions does exist. **Prone to consistency issues**
Typos and updates to the HTTPRoute can easily cause a mismatch and invalidate a RLP. @@ -959,5 +1027,5 @@ Most implementations currently orbiting around Gateway API (e.g. Istio, Envoy Ga ## Future possibilities [future-possibilities]: #future-possibilities -- Port `matches` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). +- Port `rules` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). - Defaults and overrides, either along the lines of [architecture#4](https://github.com/Kuadrant/architecture/pull/4) or [architecture#10](https://github.com/Kuadrant/architecture/pull/10). From c9d306b567df80cc6620caaff58eb159766ce589 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Mon, 20 Mar 2023 16:32:35 +0100 Subject: [PATCH 14/17] Back to matching any subset of HTTPRouteMatches as the way to select the HTTPRouteRules to be bound to a limit + Limit's name becomes a first-class citizen --- rfcs/0000-rlp-v2.md | 1125 +++++++++++++++++++++++++++---------------- 1 file changed, 712 insertions(+), 413 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 1f9312b7..d0c64599 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -13,36 +13,37 @@ Proposal of new API for the Kuadrant's `RateLimitPolicy` (RLP) CRD, for improved ### Motivation [motivation]: #motivation -The [`RateLimitPolicy`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimitPolicy) API (v1beta1), particularly its [`RateLimit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimit) type used in `ratelimitpolicy.spec.rateLimits`, designed in part to fit the underlying implementation based on the Envoy [Rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) filter, has been proven to be (i) _unnecessarily complex_, as well as (ii) possibly _limiting for the extension of the API_ either beyond this specific implementation and/or for supporting use cases of rate limiting not contemplated in the original designs. +The [`RateLimitPolicy`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimitPolicy) API (v1beta1), particularly its [`RateLimit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimit) type used in `ratelimitpolicy.spec.rateLimits`, designed in part to fit the underlying implementation based on the Envoy [Rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) filter, has been proven to be complex, as well as somewhat limiting for the extension of the API for other platforms and/or for supporting use cases of not contemplated in the original design. -Users of the `RateLimitPolicy` will immediately recognize elements of Envoy's Rate limit API in the definitions of the `RateLimit` type, with almost 1:1 correspondence between the [`Configuration`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Configuration) and [`Rule`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Rule) types and their counterparts in the Envoy configuration. Although compatibility between those continue to be desired, the leakage of such implementation details to the level of the API could otherwise be avoided to provide a better abstraction for the context in vogue, where activators ("matchers") and payload ("descriptors") are defined by users in a same document instead of in different configurations of different components. +Users of the `RateLimitPolicy` will immediately recognize elements of Envoy's Rate limit API in the definitions of the `RateLimit` type, with almost 1:1 correspondence between the [`Configuration`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Configuration) type and its [counterpart in the Envoy configuration](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-msg-config-route-v3-ratelimit). Although compatibility between those continue to be desired, leaking such implementation details to the level of the API can be avoided to provide a better abstraction for activators ("matchers") and payload ("descriptors"), stated by users in a seamless way. -Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Limit) type – used as well in the `RateLimitPolicy`'s `RateLimit` – implies presently a logical relationship between its inner concepts – i.e. conditions and variables on one side, and limits themselves on the other – that otherwise could be shaped in a different manner, to provide (i) clearer understanding of the meaning of these concepts by the user as well as (ii) to avoid repetition. +Furthermore, the [`Limit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Limit) type – used as well in the RLP's `RateLimit` type – implies presently a logical relationship between its inner concepts – i.e. conditions and variables on one side, and limits themselves on the other – that otherwise could be shaped in a different manner, to provide clearer understanding of the meaning of these concepts by the user and avoid repetition. I.e., one limit definition contains multiple rate limits, and not the other way around. ### Goals 1. Decouple the API from the underlying implementation - i.e. provide a more generic and more user-friendly abstraction -2. Simplify the API - i.e. DRY, straight to the point (for defining rate limits) -3. Consistency with Kuadrant's [AuthPolicy](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#AuthPolicy) API - i.e. same language, similar UX +2. Prepare the API for upcoming changes in the Gateway API Policy Attachment specification +3. Improve consistency of the API with respect to Kuadrant's [AuthPolicy](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#AuthPolicy) CRD - i.e. same language, similar UX ### Current WIP to consider -1. A single Policy scoped to HTTPRoutes and HTTPRouteRule ([architecture#4](https://github.com/Kuadrant/architecture/pull/4)) -2. *No* merging of policies ([architecture#10](https://github.com/Kuadrant/architecture/pull/10)) -3. Implement `skip_if_absent` for the RequestHeaders action ([wasm-shim#29](https://github.com/Kuadrant/wasm-shim/issues/29)) -4. Rate Limiting across clusters ([doc](https://docs.google.com/document/d/1pqCODRAkNUTLB6lJfRWcv3d2Hlj9WqsGySmjrP707Vk/edit#heading=h.nzpgr3pef6uy)) +1. Policy attachment update ([kubernetes-sigs/gateway-api#1565](https://github.com/kubernetes-sigs/gateway-api/pull/1565)) +2. *No* merging of policies ([kuadrant/architecture#10](https://github.com/Kuadrant/architecture/pull/10)) +3. A single Policy scoped to HTTPRoutes and HTTPRouteRule ([kuadrant/architecture#4](https://github.com/Kuadrant/architecture/pull/4)) - future +4. Implement `skip_if_absent` for the RequestHeaders action ([kuadrant/wasm-shim#29](https://github.com/Kuadrant/wasm-shim/issues/29)) ### Highlights -- `spec.rateLimits` replaced with `spec.limits` -- `spec.rateLimits.limits` replaced with `spec.limits.rates` -- `spec.rateLimits.limits.maxValue` replaced with `spec.limits.rates.limit` -- `spec.rateLimits.limits.seconds` replaced with `spec.limits.rates.duration` + `spec.limits.rates.unit` -- `spec.rateLimits.limits.conditions` replaced with `spec.limits.when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) -- `spec.rateLimits.limits.variables` replaced with `spec.limits.counters`, based on _well-known selectors_ -- `spec.rateLimits.rules` replaced with `spec.limits.rules`, for "sub-targeting" HTTPRouteRules within an HTTPRoute -- `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.rules` -- Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted HTTPRouteRules +- `spec.rateLimits[]` replaced with `spec.limits{: }` +- `spec.rateLimits.limits` replaced with `spec.limits..rates` +- `spec.rateLimits.limits.maxValue` replaced with `spec.limits..rates.limit` +- `spec.rateLimits.limits.seconds` replaced with `spec.limits..rates.duration` + `spec.limits..rates.unit` +- `spec.rateLimits.limits.conditions` replaced with `spec.limits..when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) +- `spec.rateLimits.limits.variables` replaced with `spec.limits..counters`, based on _well-known selectors_ +- `spec.rateLimits.rules` replaced with `spec.limits..triggers`, for selecting (or "sub-targeting") HTTPRouteRules that trigger the limit +- new matcher `spec.limits..triggers.hostnames[]` +- `spec.rateLimits.configurations` removed – descriptor actions configuration (previously `spec.rateLimits.configurations.actions`) generated from `spec.limits..when.selector` ∪ `spec.limits..counters` and unique identifier of the limit (associated with `spec.limits..triggers`) +- Limitador conditions composed of "soft" `spec.limits..when` conditions + a "hard" condition that binds the limit to its trigger HTTPRouteRules For detailed differences between current and vew RLP API, see [Comparison to current RateLimitPolicy](#comparison-to-current-ratelimitpolicy). @@ -62,7 +63,8 @@ metadata: spec: gatewayClassName: istio listeners: - - hostname: "*.toystore.com" + - hostname: + - "*.acme.com" --- apiVersion: gateway.networking.k8s.io/v1alpha2 kind: HTTPRoute @@ -71,57 +73,60 @@ metadata: namespace: toystore spec: parentRefs: - - name: istio-ingressgateway - namespace: istio-system - hostnames: ["*.toystore.com"] + - name: istio-ingressgateway + namespace: istio-system + hostnames: + - "*.toystore.acme.com" rules: - - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST - backendRefs: - - name: toystore - port: 80 - - matches: - - path: - type: PathPrefix - value: "/assets/" - backendRefs: - - name: toystore - port: 80 - filters: - - type: ResponseHeaderModifier - responseHeaderModifier: - set: - - name: Cache-Control - value: max-age=31536000, immutable + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + backendRefs: + - name: toystore + port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: "max-age=31536000, immutable" ``` The following are examples of RLPs targeting the route and the gateway. Each example is independent from the other. #### Example 1. Minimal example - network resource targeted entirely without filtering, unconditional and unqualified rate limiting -In this example, all traffic to `*.toystore.com` will be limited to 5rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. +In this example, all traffic to `*.toystore.acme.com` will be limited to 5rps, regardless of any other attribute of the HTTP request (method, path, headers, etc), without any extra "soft" conditions (conditions non-related to the HTTP route), across all consumers of the API (unqualified rate limiting). ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-simple-infra-rl + name: toystore-infra-rl + namespace: toystore spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: + base: # user-defined name of the limit definition - future use for handling hierarchical policy attachment - rates: # at least one rate limit required - - limit: 5 - unit: second + - limit: 5 + unit: second ```
@@ -130,90 +135,87 @@ spec: ```yaml gateway_actions: - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] - configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] - - rules: - - paths: ["/assets/*"] - methods: ["*"] - hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] + - paths: ["/assets/*"] + hosts: ["*.toystore.acme.com"] configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-infra-rl/base" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" + - ratelimit.binding == "toystore/toystore-infra-rl/base" max_value: 5 seconds: 1 - namespace: "*.toystore.com" + namespace: TDB ```
-#### Example 2. Targeting specific route rules, plus conditions, counter qualifiers and multiple rates +#### Example 2. Targeting specific route rules, with counter qualifiers, multiple rates per limit definition and "soft" conditions + +In this example, a distinct limit will be associated ("bound") to each individual HTTPRouteRule of the targeted HTTPRoute, by using the `triggers` field for selecting (or "sub-targeting") the HTTPRouteRule. -In this example, a distinct limit will be associated to each individual rule of the targeted HTTPRoute, using the `rules` field for fine-grained filtering (or "sub-targeting"). -- `GET|POST /toys*` → 50rpm, enforced per username (counter qualifier) and only in case the user is not an admin (condition). +The following limit definitions will be bound to each HTTPRouteRule: +- `GET|POST /toys*` → 50rpm, enforced per username (counter qualifier) and only in case the user is not an admin ("soft" condition). - `/assets/*` → 5rpm / 100rp12h -Each rule in the RLP must perfectly match a rule in the HTTPRoute. In case there is no identical match amongst the HTTPRoute rules for a given rule defined in the RLP, the RLP is considered invalid. In case there is more than one rule specified in the RLP that is an identical match to the same rule in the HTTPRoute, the first matching rule on the list in the RLP is bound to its identical match in the HTTPRoute, thus "shadowing" any other rule on the list that is also an identical match. +Each set of trigger matches in the RLP will be matched to **all** HTTPRouteRules whose HTTPRouteMatches is a superset of the set of trigger matches in the RLP. For every HTTPRouteRule matched, the HTTPRouteRule will be bound to the corresponding limit definition that specifies that trigger. In case no HTTPRouteRule is found containing at least one HTTPRouteMatch that is identical to some set of matching rules of a particular limit definition, the limit definition is considered invalid and reported as such in the status of RLP. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: name: toystore-per-endpoint + namespace: toystore spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: toys - rules: - - matches: # matches the 1st rule in the HTTPRoute - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST + toys: rates: - - limit: 50 - duration: 1 - unit: minute + - limit: 50 + duration: 1 + unit: minute counters: - - auth.identity.username + - auth.identity.username + triggers: + - matches: # matches the 1st HTTPRouteRule (i.e. GET or POST to /toys*) + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST when: - - selector: auth.identity.group - operator: neq - value: admin - - - name: assets - rules: - - matches: # matches the 2nd rule in the HTTPRoute - - path: - type: PathPrefix - value: "/assets/" + - selector: auth.identity.group + operator: neq + value: admin + + assets: rates: - - limit: 5 - duration: 1 - unit: minute - - limit: 100 - duration: 12 - unit: hour + - limit: 5 + duration: 1 + unit: minute + - limit: 100 + duration: 12 + unit: hour + triggers: + - matches: # matches the 2nd HTTPRouteRule (i.e. /assets/*) + - path: + type: PathPrefix + value: "/assets/" ```
@@ -222,116 +224,211 @@ spec: ```yaml gateway_actions: - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] - - metadata: - descriptor_key: "auth.identity.group" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "group" - - metadata: - descriptor_key: "auth.identity.username" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "username" + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-per-endpoint/toys" + - metadata: + descriptor_key: "auth.identity.group" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "group" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" - rules: - - paths: ["/assets/*"] - methods: ["*"] - hosts: ["*.toystore.com"] + - paths: ["/assets/*"] + hosts: ["*.toystore.acme.com"] configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-per-endpoint/assets" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - - auth.identity.group != "admin" + - ratelimit.binding == "toystore/toystore-per-endpoint/toys" + - auth.identity.group != "admin" variables: - - auth.identity.username + - auth.identity.username max_value: 50 seconds: 60 - namespace: "*.toystore.com" + namespace: TBD - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" + - ratelimit.binding == "toystore/toystore-per-endpoint/assets" max_value: 5 seconds: 60 - namespace: "*.toystore.com" + namespace: TBD - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" + - ratelimit.binding == "toystore/toystore-per-endpoint/assets" max_value: 100 seconds: 43200 # 12 hours - namespace: "*.toystore.com" + namespace: TBD ```
-#### Example 3. Route filtering using `when` conditions +#### Example 3. Targeting a subset of an HTTPRouteRule - HTTPRouteMatch missing -This is a similar RLP to the one from [Example 2](#example-2-targeting-specific-route-rules-plus-conditions-counter-qualifiers-and-multiple-rates) regarding the use case of applying specific limits to portions of an HTTPRoute. Whereas in the previous example the route rules map perfectly for the limits to be applied, in this example we use `when` conditions to apply the limit to a subpath without having a corresponding HTTPRouteRule that could be referred using the `rules` field. +Consider a 150rps rate limit set on requests to `GET /toys/special`. Such specific application endpoint is covered by the first HTTPRouteRule in the HTTPRoute (as a subset of `GET` or `POST` to any path that starts with `/toys`). However, to avoid binding limits to HTTPRouteRules that are more permissive than the actual intended scope of the limit, the RateLimitPolicy controller requires trigger matches to find identical matching rules explicitly defined amongst the sets of HTTPRouteMatches of the HTTPRouteRules potentially targeted. -`when` conditions are to be used preferably for special cases of conditional filtering based on values other than attributes of the HTTP request that otherwise could be specified using the `rules` field of the RLP. However, in some cases such as where the conditions are not related to the HTTPRoute (e.g. filterring based on hostname, filterring based on metadata) or while the targeted object (HTTPRoute or Gateway) misses the desired rule and cannot be changed, the `when` conditions remain an option. +As a consequence, by simply defining a trigger match for `GET /toys/special` in the RLP, the `GET|POST /toys*` HTTPRouteRule will NOT be bound to the limit definition. In order to ensure the limit definition is properly bound to a routing rule that strictly covers the `GET /toys/special` application endpoint, first the user has to modify the spec of the HTTPRoute by adding an explicit HTTPRouteRule for this case: -In this example, a special limit with one rate limit of 150rps is set for `GET /toys/special`. Ideally, an additional HTTPRoute rule would be created and targeted using the `rules` field, with resulting benefits for optimization, status reporting and matching. But because the `toystore` HTTPRoute is unaltered, the special case is implemented using the `when` conditions field instead. +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: HTTPRoute +metadata: + name: toystore + namespace: toystore +spec: + parentRefs: + - name: istio-ingressgateway + namespace: istio-system + hostnames: + - "*.toystore.acme.com" + rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + backendRefs: + - name: toystore + port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: "max-age=31536000, immutable" + - matches: # new (more specific) HTTPRouteRule added + - path: + type: Exact + value: "/toys/special" + method: GET + backendRefs: + - name: toystore + port: 80 +``` + +After that, the RLP can target the new HTTPRouteRule strictly: ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: name: toystore-special-toys + namespace: toystore spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: read-special - rules: - - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: # must be included as well or it wouldn't match the http route rule - type: PathPrefix - value: "/toys" - method: POST + specialToys: rates: - - limit: 150 - unit: second - when: - - selector: context.request.http.method - operator: eq - value: GET - - selector: context.request.http.path - operator: eq - value: /toys/special + - limit: 150 + unit: second + triggers: + - matches: # matches the new HTTPRouteRule (i.e. GET /toys/special) + - path: + type: Exact + value: "/toys/special" + method: GET ``` -By not targeting a specific HTTPRoute rule added for `GET /toys/special`, the rate limit filter (wasm shim) will continue invoking the rate limit service (Limitador) for all requests that match the HTTP the path prefix `/toys` (methods `GET` and `POST`), as targeted HTTPRoute rule. This includes requests to paths other than `GET /toys/special` nevertheless. However, the limit will be ensured to be enforced in Limitador only when the request matches `GET /toys/special`. +
+ How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys/special"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + configurations: + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-special-toys/specialToys" + ``` + + ```yaml + limits: + - conditions: + - ratelimit.binding == "toystore/toystore-special-toys/specialToys" + max_value: 150 + seconds: 1 + namespace: TBD + ``` +
+ +**Usage of "soft" `when` conditions for filtering based on attributes of the HTTP request** -Technically, the `rules` field did not have to be included in the RLP, and the `when` would still ensure the limit applies only to requests to `GET /toys/special`. However, in this case the other HTTPRoute rule (`/assets/*`) would also be initally targeted by the limit, though never effectivelly triggered. +User are encoraged to never use "soft" `when` conditions to selective apply rate limit definitions based on attributes of the HTTP request that otherwise could be specified using the "hard" `triggers` field of the RLP instead. -On one hand, using `when` conditions for route filterring makes it easy to set limits when either the HTTPRoute cannot be modified to include the special rule or as a shortcut in cases where multiple HTTPRoutes would have to be touched (e.g., when targeting a Gateway with special conditions based on attributes of the HTTP request). On the other hand however, users might miss information in the status in a scenario where the status of rate limiting is reported at the level of the route rule. Effectively, the route rule for `GET|POST /toys*` in the above example might be reported as rate limited to '150rps' with additional details that this is in fact only when requests match `GET /toys/special`. +On one hand, using `when` conditions for route filtering makes it easy to define limits when the HTTPRoute cannot be modified to include the special rule. On the other hand, users might miss information in the status. Effectively, the HTTPRouteRule for `GET|POST /toys*` in the example might be reported as rate limited to 150rps with additional details that this is in fact only for requests to `GET /toys/special`. -In the case of existing other limit definitions targeting the `GET|POST /toys*` rule of the `toystore` HTTPRoute, because any merge strategy is expected to take into account the `rules` field as part of the qualification of the limit, all limits are guaranteed to be enforced, and only differenciated by the special "soft" the condition. Nevertheless, to avoid dealing with complex status reports including too many special conditions associated with a limit, **users are encouraged to favor altering the HTTPRoutes for additional route rules in all cases where the rules can be referred in the `rules` field of the RLP, whenever possible**. +Moreover, by not specifying a more strict HTTPRouteRule for `GET /toys/special`, the RLP controller might bind the limit definition to other rules that would cause the rate limit filter to invoke the rate limit service (Limitador) for cases other than strictly `GET /toys/special`. Even if the rate limits are ensured to apply in Limitador only for `GET /toys/special` requests (due to the presence of a "soft" `when` condition), the extra no-op hop to the rate limit service could be avoided. + +Example of usage of "soft" `when` conditions to selective apply rate limit based on attributes of the HTTP request: + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-special-toys + namespace: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + specialToys: + rates: + - limit: 150 + unit: second + triggers: + - matches: # matches the original HTTPRouteRule GET|POST /toys* + - path: + type: PathPrefix + value: "/toys" + method: GET + when: + - selector: context.request.http.method # cannot omit this selector or POST /toys/special would also be rate limited + operator: eq + value: GET + - selector: context.request.http.path + operator: eq + value: /toys/special +```
How is this RLP implemented under the hood? @@ -339,37 +436,43 @@ In the case of existing other limit definitions targeting the `GET|POST /toys*` ```yaml gateway_actions: - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-special-toys/read-special/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] - - request_headers: - descriptor_key: "context.request.http.path" - header_name: ":path" + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-special-toys/specialToys" + - request_headers: + descriptor_key: "context.request.http.method" + header_name: ":method" + - request_headers: + descriptor_key: "context.request.http.path" + header_name: ":path" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-special-toys/read-special/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - - context.request.http.path == "/toys/special" + - ratelimit.binding == "toystore/toystore-special-toys/specialToys" + - context.request.http.method == "GET" + - context.request.http.path == "/toys/special" max_value: 150 seconds: 1 - namespace: "*.toystore.com" + namespace: TBD ```
-#### Example 4. Route filtering by refining the HTTPRoute +#### Example 4. Targeting a subset of an HTTPRouteRule - HTTPRouteMatch found -To achieve the same goal as stated in [Example 3](#example-3-route-filtering-using-when-conditions), yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition details associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `rules` field instead of the `when` conditions field. +This example is similar to [Example 3](#example-3-targeting-a-subset-of-an-httprouterule---httproutematch-missing). Consider the use case of setting a 150rpm rate limit on requests to `GET /toys*`. -New rule added to the HTTPRoute: +The targeted application endpoint is covered by the first HTTPRouteRule in the HTTPRoute (as a subset of `GET` or `POST` to any path that starts with `/toys`). However, unlike in the previous example where, at first, no HTTPRouteRule included an explicit HTTPRouteMatch for `GET /toys/special`, in this example the HTTPRouteMatch for the targeted application endpoint `GET /toys*` does exist explicitly in one of the HTTPRouteRules, thus the RateLimitPolicy controller would find no problem to bind the limit definition to the HTTPRouteRule. That would nonetheless cause a unexpected behavior of the limit triggered not strictly for `GET /toys*`, but also for `POST /toys*`. + +To avoid extending the scope of the limit beyiond desired, with no extra "soft" conditions, again the user must modify the spec of the HTTPRoute, so an exclusive HTTPRouteRule exists for the `GET /toys*` application endpoint: ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 @@ -379,11 +482,120 @@ metadata: namespace: toystore spec: parentRefs: - - name: istio-ingressgateway - namespace: istio-system - hostnames: ["*.toystore.com"] + - name: istio-ingressgateway + namespace: istio-system + hostnames: + - "*.toystore.acme.com" rules: - - matches: + - matches: # first HTTPRouteRule split into two – one for GET /toys*, other for POST /toys* + - path: + type: PathPrefix + value: "/toys" + method: GET + backendRefs: + - name: toystore + port: 80 + - matches: + - path: + type: PathPrefix + value: "/toys" + method: POST + backendRefs: + - name: toystore + port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: "max-age=31536000, immutable" +``` + +The RLP can then target the new HTTPRouteRule strictly: + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toy-readers + namespace: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + toyReaders: + rates: + - limit: 150 + unit: second + triggers: + - matches: # matches the new more speficic HTTPRouteRule (i.e. GET /toys*) + - path: + type: PathPrefix + value: "/toys" + method: GET +``` + +
+ How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + configurations: + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toy-readers/toyReaders" + ``` + + ```yaml + limits: + - conditions: + - ratelimit.binding == "toystore/toy-readers/toyReaders" + max_value: 150 + seconds: 1 + namespace: TBD + ``` +
+ +#### Example 5. One limit triggered by multiple HTTPRouteRules + +In this example, both HTTPRouteRules, i.e. `GET|POST /toys*` and `/assets/*`, are targeted by the same limit of 50rpm per username. + +Because the HTTPRoute has no other rule, this is technically equivalent to targeting the entire HTTPRoute and therefore similar to [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting). However, if the HTTPRoute had other rules or got other rules added afterwards, this would ensure the limit applies only to the two original route rules. + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-per-user + namespace: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + toysOrAssetsPerUsername: + rates: + - limit: 50 + duration: 1 + unit: minute + counters: + - auth.identity.username + triggers: + - matches: - path: type: PathPrefix value: "/toys" @@ -392,55 +604,98 @@ spec: type: PathPrefix value: "/toys" method: POST - backendRefs: - - name: toystore - port: 80 - - matches: + - matches: - path: type: PathPrefix value: "/assets/" - backendRefs: - - name: toystore - port: 80 - filters: - - type: ResponseHeaderModifier - responseHeaderModifier: - set: - - name: Cache-Control - value: max-age=31536000, immutable - - matches: # new rule added so it can be targeted in the RLP - - path: - type: Exact - value: "/toys/special" - method: GET - backendRefs: - - name: toystore - port: 80 ``` -RateLimitPolicy: +
+ How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] + - paths: ["/assets/*"] + hosts: ["*.toystore.acme.com"] + configurations: + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-per-user/toysOrAssetsPerUsername" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + ``` + + ```yaml + limits: + - conditions: + - ratelimit.binding == "toystore/toystore-per-user/toysOrAssetsPerUsername" + variables: + - auth.identity.username + max_value: 50 + seconds: 60 + namespace: TBD + ``` +
+ +#### Example 6. Multiple limit definitions targeting the same HTTPRouteRule + +In case multiple limit definitions target a same HTTPRouteRule, only the first limit definition will be bound to the HTTPRouteRule, thus "shadowing" the other limit definitions for that one HTTPRouteRule. + +E.g., the following RLP will set 20rps on `GET|POST /toys*` and 100rps on `/assets/*`: ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-special-toys + name: toystore-per-endpoint + namespace: toystore spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: read-special - rules: - - matches: - - path: - type: Exact - value: "/toys/special" - method: GET + readToys: + rates: + - limit: 50 + unit: second + counters: + - auth.identity.username + triggers: + - matches: # matches the 1st HTTPRouteRule (i.e. GET or POST to /toys*) + - path: + type: PathPrefix + value: "/toys" + method: GET + + assets: rates: - - limit: 150 - unit: second + - limit: 100 + unit: second + triggers: + - matches: # matches the 1st HTTPRouteRule (i.e. GET or POST to /toys*) but it's shadowed by 'readToys' + - path: + type: PathPrefix + value: "/toys" + method: POST + - matches: # matches the 2nd HTTPRouteRule (i.e. /assets/*) + - path: + type: PathPrefix + value: "/assets/" ```
@@ -449,62 +704,78 @@ spec: ```yaml gateway_actions: - rules: - - paths: ["/toys/special"] - methods: ["GET"] - hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] + configurations: + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-per-endpoint/readToys" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + - rules: + - paths: ["/assets/*"] + hosts: ["*.toystore.acme.com"] configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-special-toys/0/8267df441e5cda729095a9ea78db3abb2420855f7152379b4e88c90b8a4f562e" # SHA256 hashing of [{"matches":{"path":{"type":"Exact","value":"/toys/special"},"method":"GET"}}] + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-per-endpoint/assets" ``` ```yaml limits: - - conditions: - - context.request.http.route_matcher == "toystore/toystore-special-toys/0/8267df441e5cda729095a9ea78db3abb2420855f7152379b4e88c90b8a4f562e" - max_value: 150 + - conditions: # actually applies to GET|POST /toys* + - ratelimit.binding == "toystore/toystore-per-endpoint/readToys" + variables: + - auth.identity.username + max_value: 50 seconds: 1 - namespace: "*.toystore.com" + namespace: TBD + - conditions: # does not apply to POST /toys* + - ratelimit.binding == "toystore/toystore-per-endpoint/assets" + max_value: 100 + seconds: 1 + namespace: TBD ```
-#### Example 5. One limit, many rules +#### Example 7. Limits triggered for specific hostnames -In this example, both HTTPRoute rules, i.e. `GET|POST /toys*` and `/assets/*`, are targeted by the same limit of 50rpm per username. - -Because the HTTPRoute has no other rule, this is technically equivalent to targeting the entire HTTPRoute and therefore similar to [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting). But if the HTTPRoute had other rules or got other rules added afterwards, this would ensure the limit applies only to the two original route rules. +In the previous examples, the limit definitions and therefore the counters were set indistinctly for all hostnames – i.e. no matter if the request is sent to `games.toystore.acme.com` or `dolls.toystore.acme.com`, the same counters are expected to be affected. In this example on the other hand, a 1000rpd rate limit is set for requests to `/assets/*` only when the hostname matches `games.toystore.acme.com`. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-per-user + name: toystore-per-hostname + namespace: toystore spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - rules: - - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST - - matches: - - path: - type: PathPrefix - value: "/assets/" + games: rates: - - limit: 50 - duration: 1 - unit: minute - counters: - - auth.identity.username + - limit: 1000 + unit: day + triggers: + - matches: + - path: + type: PathPrefix + value: "/assets/" + hostnames: + - games.toystore.acme.com ```
@@ -513,100 +784,79 @@ spec: ```yaml gateway_actions: - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] - configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] - - metadata: - descriptor_key: "auth.identity.username" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "username" - - rules: - - paths: ["/assets/*"] - methods: ["*"] - hosts: ["*.toystore.com"] + - paths: ["/assets/*"] + hosts: ["*.toystore.acme.com"] configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] - - metadata: - descriptor_key: "auth.identity.username" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "username" + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-per-hostname/games" + - request_headers: + descriptor_key: "context.request.http.host" + header_name: ":authority" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" - variables: - - auth.identity.username - max_value: 50 - seconds: 60 - namespace: "*.toystore.com" + - ratelimit.binding == "toystore/toystore-per-hostname/games" + - context.request.http.host == "games.toystore.acme.com" + max_value: 1000 + seconds: 86400 # 1 day + namespace: TBD ```
-#### Example 6. Targeting the Gateway +#### Example 8. Targeting the Gateway > **Note:** Additional meaning and context may be given to this use case in the future, when discussing [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). -Targeting a Gateway is a shortcut to targeting all individual HTTPRoutes referencing to the gateway as parent, without any filtering based on the `rules` field. - -This differs from [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting) because, by targeting the gateway rather than an individual HTTPRoute, the RLP applies automatically to all HTTPRoutes pointing to the gateway, including routes created before and after the creation of the RLP. Moreover, all those routes will share the same limit counters specified in the RLP. +Targeting a Gateway is a shortcut to targeting all individual HTTPRoutes referencing the gateway as parent. This differs from [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting) nonetheless because, by targeting the gateway rather than an individual HTTPRoute, the RLP applies automatically to all HTTPRoutes pointing to the gateway, including routes created before and after the creation of the RLP. Moreover, all those routes will share the same limit counters specified in the RLP. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-simple-infra-rl + name: gw-rl + namespace: istio-ingressgateway spec: targetRef: group: gateway.networking.k8s.io kind: Gateway name: istio-ingressgateway limits: + base: - rates: - - limit: 5 - unit: second + - limit: 5 + unit: second ```
How is this RLP implemented under the hood? - Because there is no matcher in the Gateway, this limit rather have to define a generic key descriptor that is valid for all routes – i.e. without the [artificial Limitador condition](#artificial-limitador-condition-for-rules) associated with the `rules` field, such as one that identifies the RLP only. - ```yaml gateway_actions: - - configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl" + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] + - paths: ["/assets/*"] + hosts: ["*.toystore.acme.com"] + configurations: + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "istio-system/gw-rl/base" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl" + - ratelimit.binding == "istio-system/gw-rl/base" max_value: 5 seconds: 1 - namespace: "*.toystore.com" + namespace: TDB ```
@@ -623,7 +873,7 @@ spec: - + - + - + - + - - + + - + - + - + @@ -715,13 +966,17 @@ spec: ## Reference-level explanation [reference-level-explanation]: #reference-level-explanation -By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus artificial generic conditions used to match the specific route rule (when the `rules` is non-empty or implicit). +By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus an artificial condition used to bind the HTTPRouteRules to the corresponding limits to trigger in Limitador. + +The descriptor actions composed from the selectors in the "soft" `when` conditions and counter qualifiers originate from the direct references these selectors make to paths within a well-known data structure that stores information about the context (HTTP request and ext-authz filter). These selectors in "soft" `when` conditions and counter qualifiers are thereby called _well-known selectors_. + +Other descriptor actions might be composed by the RLP controller to define additional RL conditions to bind HTTPRouteRules and corresponding limits. ### Well-known selectors -Each selector is a direct reference to a path within a well-known data structure that holds the `context` (L4 and L7 data of the original request handled by the proxy), as well as occasionally the `auth` data (dynamic metadata exported by the external authorization filter and injected by the proxy into the rate-limit filter). +Each selector used in a `when` condition or counter qualifier is a direct reference to a path within a well-known data structure that stores information about the `context` (L4 and L7 data of the original request handled by the proxy), as well as `auth` data (dynamic metadata occasionally exported by the external authorization filter and injected by the proxy into the rate-limit filter). -The well-known data structure for building RL descriptor actions resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. +The well-known data structure for building RL descriptor actions resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API (marshalled as JSON). Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of predominantly serving for HTTP applications. To keep compatibility with the [Envoy Rate Limit API](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. @@ -752,10 +1007,10 @@ ratelimit: # Envoy's Rate Limit `RateLimitRequest` type ### Mechanics of generating RL descriptor actions -From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure ([_well-known selectors_](#well-known-selectors)). While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, +From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure (see [_Well-known selectors_](#well-known-selectors)). While desiging a policy, the user intuitively pictures the well-known data structure and states each limit definition having in mind the possible values assumed by each of those paths in the data plane. For example, The user story: -> _Whenever the context is an HTTP request sent to `dolls.toystore.com` hostname (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ +> _Whenever the context is an HTTP request sent to `dolls.toystore.acme.com` hostname (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ ...materializes as the following RLP: @@ -770,17 +1025,17 @@ spec: kind: HTTPRoute name: toystore limits: - - name: toys-path + dolls: when: - - selector: context.request.http.host - operator: eq - value: dolls.toystore.com + - selector: context.request.http.host + operator: eq + value: dolls.toystore.com rates: - - limit: 50 - duration: 1 - unit: second + - limit: 50 + duration: 1 + unit: second counters: - - auth.identity.username + - auth.identity.username ``` The following selectors are to be interpreted by the RLP controller: @@ -804,26 +1059,26 @@ ratelimit.domain → ```yaml rate_limits: - - actions: - - request_headers: - descriptor_key: "context.request.http.host" - header_name: ":authority" - - metadata: - descriptor_key: "auth.identity.username" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "username" +- actions: + - request_headers: + descriptor_key: "context.request.http.host" + header_name: ":authority" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" ``` -### Artificial Limitador condition for `rules` +### Artificial Limitador condition for `triggers` -For each limit with non-empty (or implicit) `rules` field, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when the filterred rules are honoured when serving the request. This can be implemented with a 2-step procedure: -1. generate an unique identifier for the limit – e.g. by applying any sufficiently entropic hash function of choice to the `rules` block of the limit; -2. associate a `generic_key` type descriptor action to the each `HTTPRouteRule` targeted by the limit. +For each limit definition that explicitly or implicitly defines a `triggers` field, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when the filterred rules are honoured when serving the request. This can be implemented with a 2-step procedure: +1. generate an unique identifier of the limit - i.e. `//` +2. associate a `generic_key` type descriptor action with each `HTTPRouteRule` targeted by the limit – i.e. `{ descriptor_key: "ratelimit.binding", descriptor_value: }`. For example, given the following RLP: @@ -832,44 +1087,45 @@ apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: name: toystore-non-admin-users + namespace: toystore spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: toys + toys: matches: - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - - limit: 50 - duration: 1 - unit: minute + - limit: 50 + duration: 1 + unit: minute when: - - selector: auth.identity.group - operator: neq - value: admin + - selector: auth.identity.group + operator: neq + value: admin - - name: assets + assets: matches: - - path: - type: PathPrefix - value: "/assets/" + - path: + type: PathPrefix + value: "/assets/" rates: - - limit: 5 - duration: 1 - unit: minute + - limit: 5 + duration: 1 + unit: minute when: - - selector: auth.identity.group - operator: neq - value: admin + - selector: auth.identity.group + operator: neq + value: admin ``` Apart from the following descriptor action associated with both routes: @@ -880,10 +1136,10 @@ Apart from the following descriptor action associated with both routes: metadata_key: key: "envoy.filters.http.ext_authz" path: - - segment: - key: "identity" - - segment: - key: "group" + - segment: + key: "identity" + - segment: + key: "group" ``` ...and its corresponding Limitador condition: @@ -897,13 +1153,13 @@ The following additional artificial descriptor actions will be generated: ```yaml # associated with route rule GET|POST /toys* - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-non-admin-users/toys" # associated with route rule /assets/* - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-non-admin-users/assets" ``` ...and their corresponding Limitador conditions. @@ -912,26 +1168,20 @@ In the end, the following Limitador configuration is yielded: ```yaml - conditions: - - context.request.http.route_matcher == "c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - - auth.identity.group != "admin" + - ratelimit.binding == "toystore/toystore-non-admin-users/toys" + - auth.identity.group != "admin" max_value: 50 seconds: 60 - namespace: "*.toystore.com" + namespace: TBD - conditions: - - context.request.http.route_matcher == "643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" - - auth.identity.group != "admin" + - ratelimit.binding == "toystore/toystore-non-admin-users/assets" + - auth.identity.group != "admin" max_value: 5 seconds: 60 - namespace: "*.toystore.com" + namespace: TBD ``` -The limit-to-route rule matcher identifiers can be qualified with a plain identifier of the RLP itself (`namespace/name`) and the limit where it is defined (`name` of the limit when available, index in the array of limits otherwise), thus making the identifier unique to the scope of the entire cluster. - -E.g.: the two unique identifiers from above, prefixed with the unique limit qualifier, become respectively `toystore/toystore-non-admin-users/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f` and `toystore/toystore-non-admin-users/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66`. - -This has a consequence to the readability of the identifier, but also and more importantly it ensures uniqueness of the counters in Limitador. By qualifying (or salting) the identifiers, two limits or two RLPs that happen to target the same HTTPRouteRules will not register the same counters in Limitador, but be treated as independent ones instead. - ### Support in wasm shim and Envoy RL API This proposal tries to keep compatibility with the [Envoy API for rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) and does not introduce any new requirement that otherwise would require the use of [wasm shim](https://github.com/Kuadrant/wasm-shim) to be implemented. @@ -941,15 +1191,62 @@ In the case of implementation of this proposal in the wasm shim, all types of ma ## Drawbacks [drawbacks]: #drawbacks -**Two types of conditions – `rules` and `when` conditions**
-Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and capable of expressing different sets of rules (HTTPRouteRule-targeting rules vs. HTTP request-related and non HTTP request-related "soft" conditions), an overlap between these two types and ways of representing conditions does exist. +**HTTPRoute editing occasionally required**
+Need to duplicate rules that don't explicitly include a matcher wanted for the policy, so that matcher can be added as a special case for each of those rules. + +**Risk of over-targeting**
+Some HTTPRouteRules might need to be split into more specific ones so a limit definition is not bound to beyond intended (e.g. target `method: GET` when the route matches `method: POST|GET`). **Prone to consistency issues**
Typos and updates to the HTTPRoute can easily cause a mismatch and invalidate a RLP. +**Two types of conditions – "hard" `triggers` and "soft" `when` conditions**
+Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and meant for expressing different types of rules (HTTPRouteRule selectors for "hard" triggers vs. HTTP request-related and non HTTP request-related "soft" conditions), an overlap between these two types and ways of representing conditions does exist. + ## Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives +### Targeting full HTTPRouteRules + +Requiring users to specify full HTTPRouteRule matches in the RLP (as opposed to any subset of HTTPRoureMatches of targeted HTTPRouteRules – current proposal) contains some of the same [drawbacks](#drawbacks) of this proposal, such as HTTPRoute editing occasionally required and prone to consistency issues. If, on one hand, it eliminates the risk of over-targeting, on the other hand, it does it at the cost of requiring excessively verbose policies written by the users, to the point of sometimes expecting user to have to specify trigger matching rules that are significantly more than what's originally and strictly intended. + +E.g.: + +On a HTTPRoute that contains the following HTTPRouteRules (simplified representation): + +``` +{ header: x-canary=true } → backend-canary +{ * } → backend-rest +``` + +Where the user wants to define a RLP that targets `{ method: POST }`. First, the user needs to edit the HTTPRoute and duplicate the HTTPRouteRules: + +``` +{ header: x-canary=true, method: POST } → backend-canary +{ header: x-canary=true } → backend-canary +{ method: POST } → backend-rest +{ * } → backend-rest +``` + +Then, user needs to include the following trigger in the RLP so only full HTTPRouteRules are specified: + +``` +{ header: x-canary=true, method: POST } +{ method: POST } +``` + +The first matching rule of the trigger (i.e. `{ header: x-canary=true, method: POST }`) is beoynd the original user intent of targeting simply `{ method: POST }`. + +This issue can be even more concerning in the case of targeting gateways with multiple child HTTPRoutes. All the HTTPRoutes would have to be fixed and the HTTPRouteRules that cover for all the cases in all HTTPRoutes listed in the policy targeting the gateway. + +### Limit shadowing vs. All limit definitions apply + +The proposed [binding between limit definition and HTTPRouteRules that trigger the limit](#artificial-limitador-condition-for-triggers) was thought so only one limit definition can be bound to an HTTPRouteRule that triggers the limit. Once a limit is bound to the HTTPRouteRule, that limit "shadows" any other limit definition that tries to be bound to the same HTTPRouteRule. + +An alternative to such limit shadowing approach led by the first bound limit definition is allowing all limit definitions that targets (via trigger selectors) an HTTPRouteRule to be bound to that HTTPRouteRule. While the first approach ("limit shadowing") causes an artificial Limitador condition of the form `ratelimit.binding == "//"`, the alternative approach can be implemented by generating a descriptor of the following form instead: `// == "1"`. + +The downside of allowing multiple bindings to the same HTTPRouteRule is that all limits would apply in Limitador, thus making status report potently very hard. Moreover, the most restritive rate limit strategy implemented by Limitador might not be as obvious to users setting multiple limit definitions as it is when setting multiple rate limits within a single limit definition. + ### Possible variations for the selectors (conditions and counter qualifiers) The main drivers behind the proposed design for the selectors (conditions and counter qualifiers), based on (i) structured condition expressions composed of fields `selector`, `operator`, and `value`, and (ii) `when` conditions and `counters` separated in two distinct fields (variation "C" below), are: @@ -1020,12 +1317,14 @@ Most implementations currently orbiting around Gateway API (e.g. Istio, Envoy Ga ## Unresolved questions [unresolved-questions]: #unresolved-questions -1. What does this mean with _defaults_ and _overrides_? -2. Do we need anything here that points to shared counters across clusters/Limitador instances or does that belong to different layer (`Limitador`, `Kuadrant` CRDs, MCTC)? -3. What condition `operator`s do we need (e.g. `eq`, `neq`, `exists`, `nexists`, `matches`)? +1. In case a limit definition lists triggers such that some of the triggers can be bound to HTTPRouteRules and some cannot (see [Example 6](#example-6-multiple-limit-definitions-targeting-the-same-httprouterule)), do we bind the valid triggers and ignore the invalid ones or the limit definition is invalid altogether and bound to no HTTPRouteRule at all? +2. What should we fill domain/namespace with, if no longer with the hostname? This can be useful for multi-tenancy. +3. How do we support lists of hostnames in Limitador conditions (single counter)? Should we open an issue for a new `in` operator? +4. What "soft" condition `operator`s do we need to support (e.g. `eq`, `neq`, `exists`, `nexists`, `matches`)? +5. Do we need special field to define shared counters across clusters/Limitador instances or that's to be solved at another layer (`Limitador`, `Kuadrant` CRDs, MCTC)? ## Future possibilities [future-possibilities]: #future-possibilities -- Port `rules` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). +- Port `triggers` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). - Defaults and overrides, either along the lines of [architecture#4](https://github.com/Kuadrant/architecture/pull/4) or [architecture#10](https://github.com/Kuadrant/architecture/pull/10). From e09b7daa467d683625dbde95b79e005522224640 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 21 Mar 2023 17:54:20 +0100 Subject: [PATCH 15/17] Soft conditions do not accept expressions related to attributes of the HTTP request --- rfcs/0000-rlp-v2.md | 156 ++++++++++++++++++++++---------------------- 1 file changed, 78 insertions(+), 78 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index d0c64599..02995a32 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -389,83 +389,6 @@ spec: ``` -**Usage of "soft" `when` conditions for filtering based on attributes of the HTTP request** - -User are encoraged to never use "soft" `when` conditions to selective apply rate limit definitions based on attributes of the HTTP request that otherwise could be specified using the "hard" `triggers` field of the RLP instead. - -On one hand, using `when` conditions for route filtering makes it easy to define limits when the HTTPRoute cannot be modified to include the special rule. On the other hand, users might miss information in the status. Effectively, the HTTPRouteRule for `GET|POST /toys*` in the example might be reported as rate limited to 150rps with additional details that this is in fact only for requests to `GET /toys/special`. - -Moreover, by not specifying a more strict HTTPRouteRule for `GET /toys/special`, the RLP controller might bind the limit definition to other rules that would cause the rate limit filter to invoke the rate limit service (Limitador) for cases other than strictly `GET /toys/special`. Even if the rate limits are ensured to apply in Limitador only for `GET /toys/special` requests (due to the presence of a "soft" `when` condition), the extra no-op hop to the rate limit service could be avoided. - -Example of usage of "soft" `when` conditions to selective apply rate limit based on attributes of the HTTP request: - -```yaml -apiVersion: kuadrant.io/v2beta1 -kind: RateLimitPolicy -metadata: - name: toystore-special-toys - namespace: toystore -spec: - targetRef: - group: gateway.networking.k8s.io - kind: HTTPRoute - name: toystore - limits: - specialToys: - rates: - - limit: 150 - unit: second - triggers: - - matches: # matches the original HTTPRouteRule GET|POST /toys* - - path: - type: PathPrefix - value: "/toys" - method: GET - when: - - selector: context.request.http.method # cannot omit this selector or POST /toys/special would also be rate limited - operator: eq - value: GET - - selector: context.request.http.path - operator: eq - value: /toys/special -``` - -
- How is this RLP implemented under the hood? - - ```yaml - gateway_actions: - - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.acme.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.acme.com"] - configurations: - - generic_key: - descriptor_key: "ratelimit.binding" - descriptor_value: "toystore/toystore-special-toys/specialToys" - - request_headers: - descriptor_key: "context.request.http.method" - header_name: ":method" - - request_headers: - descriptor_key: "context.request.http.path" - header_name: ":path" - ``` - - ```yaml - limits: - - conditions: - - ratelimit.binding == "toystore/toystore-special-toys/specialToys" - - context.request.http.method == "GET" - - context.request.http.path == "/toys/special" - max_value: 150 - seconds: 1 - namespace: TBD - ``` -
- #### Example 4. Targeting a subset of an HTTPRouteRule - HTTPRouteMatch found This example is similar to [Example 3](#example-3-targeting-a-subset-of-an-httprouterule---httproutematch-missing). Consider the use case of setting a 150rpm rate limit on requests to `GET /toys*`. @@ -1201,7 +1124,7 @@ Some HTTPRouteRules might need to be split into more specific ones so a limit de Typos and updates to the HTTPRoute can easily cause a mismatch and invalidate a RLP. **Two types of conditions – "hard" `triggers` and "soft" `when` conditions**
-Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and meant for expressing different types of rules (HTTPRouteRule selectors for "hard" triggers vs. HTTP request-related and non HTTP request-related "soft" conditions), an overlap between these two types and ways of representing conditions does exist. +Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and meant for expressing different types of rules (HTTPRouteRule selectors for "hard" triggers vs. "soft" conditions based on attributes not related to the HTTP request), users might still perceive these as two ways of expressing conditions and find difficult to understand at first that "soft" conditions do not accept expressions related to attributes of the HTTP request. ## Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives @@ -1247,6 +1170,83 @@ An alternative to such limit shadowing approach led by the first bound limit def The downside of allowing multiple bindings to the same HTTPRouteRule is that all limits would apply in Limitador, thus making status report potently very hard. Moreover, the most restritive rate limit strategy implemented by Limitador might not be as obvious to users setting multiple limit definitions as it is when setting multiple rate limits within a single limit definition. +### Writing "soft" `when` conditions based on attributes of the HTTP request + +As a first steps, users will not be able to write "soft" `when` conditions to selective apply rate limit definitions based on attributes of the HTTP request that otherwise could be specified using the "hard" `triggers` field of the RLP instead. + +On one hand, using `when` conditions for route filtering would make it easy to define limits when the HTTPRoute cannot be modified to include the special rule. On the other hand, users would miss information in the status. An HTTPRouteRule for `GET|POST /toys*`, for example, that is targeted with an additional "soft" `when` condition that specifies that the method must be equal to `GET` and the path exactly equal to `/toys/special` (see [Example 3](#example-3-targeting-a-subset-of-an-httprouterule---httproutematch-missing)) would be reported as rate limited with extra details that this is in fact only for `GET /toys/special`. For small deployments, this might be considered acceptable; however it would easily explode to unmanageable number of cases for deployments with only a few limit definitions and HTTPRouteRules. + +Moreover, by not specifying a more strict HTTPRouteRule for `GET /toys/special`, the RLP controller would bind the limit definition to other rules that would cause the rate limit filter to invoke the rate limit service (Limitador) for cases other than strictly `GET /toys/special`. Even if the rate limits would still be ensured to apply in Limitador only for `GET /toys/special` (due to the presence of a hypothetical "soft" `when` condition), an extra no-op hop to the rate limit service would happen. This is avoided with the current imposed limitation. + +Example of "soft" `when` conditions for rate limit based on attributes of the HTTP request (NOT SUPPORTED): + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-special-toys + namespace: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + specialToys: + rates: + - limit: 150 + unit: second + triggers: + - matches: # matches the original HTTPRouteRule GET|POST /toys* + - path: + type: PathPrefix + value: "/toys" + method: GET + when: + - selector: context.request.http.method # cannot omit this selector or POST /toys/special would also be rate limited + operator: eq + value: GET + - selector: context.request.http.path + operator: eq + value: /toys/special +``` + +
+ How is this RLP would be implemented under the hood if supported? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.acme.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.acme.com"] + configurations: + - generic_key: + descriptor_key: "ratelimit.binding" + descriptor_value: "toystore/toystore-special-toys/specialToys" + - request_headers: + descriptor_key: "context.request.http.method" + header_name: ":method" + - request_headers: + descriptor_key: "context.request.http.path" + header_name: ":path" + ``` + + ```yaml + limits: + - conditions: + - ratelimit.binding == "toystore/toystore-special-toys/specialToys" + - context.request.http.method == "GET" + - context.request.http.path == "/toys/special" + max_value: 150 + seconds: 1 + namespace: TBD + ``` +
+ ### Possible variations for the selectors (conditions and counter qualifiers) The main drivers behind the proposed design for the selectors (conditions and counter qualifiers), based on (i) structured condition expressions composed of fields `selector`, `operator`, and `value`, and (ii) `when` conditions and `counters` separated in two distinct fields (variation "C" below), are: From 84380711993edfdc78fa7651949a6680f96ceee1 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 21 Mar 2023 18:05:58 +0100 Subject: [PATCH 16/17] Update the example of the Mechanics of generating RL descriptor actions for one that does not involve HTTP attributes used in 'soft' conditions --- rfcs/0000-rlp-v2.md | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 02995a32..9742987d 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -933,7 +933,7 @@ ratelimit: # Envoy's Rate Limit `RateLimitRequest` type From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure (see [_Well-known selectors_](#well-known-selectors)). While desiging a policy, the user intuitively pictures the well-known data structure and states each limit definition having in mind the possible values assumed by each of those paths in the data plane. For example, The user story: -> _Whenever the context is an HTTP request sent to `dolls.toystore.acme.com` hostname (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ +> _Each distinct user (`auth.identity.username`) can send no more than 1rps to the same HTTP path (`context.request.http.path`)._ ...materializes as the following RLP: @@ -949,21 +949,17 @@ spec: name: toystore limits: dolls: - when: - - selector: context.request.http.host - operator: eq - value: dolls.toystore.com rates: - - limit: 50 - duration: 1 + - limit: 1 unit: second counters: - auth.identity.username + - context.request.http.path ``` The following selectors are to be interpreted by the RLP controller: -- `context.request.http.host` - `auth.identity.username` +- `context.request.http.path` The RLP controller uses a map to translate each selector into its corresponding descriptor action. (Roughly described:) @@ -983,9 +979,6 @@ ratelimit.domain → ```yaml rate_limits: - actions: - - request_headers: - descriptor_key: "context.request.http.host" - header_name: ":authority" - metadata: descriptor_key: "auth.identity.username" metadata_key: @@ -995,6 +988,9 @@ rate_limits: key: "identity" - segment: key: "username" + - request_headers: + descriptor_key: "context.request.http.path" + header_name: ":path" ``` ### Artificial Limitador condition for `triggers` From 33dec29a4b38f87e32811ab593ebd9980734bffe Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 21 Mar 2023 17:59:10 +0100 Subject: [PATCH 17/17] Fix example 1 so it's clear the HTTPRouteMatch does not have to entirely stated within the trigger matches, but only a portion of it will do (if that portion is enough to scope the limit) --- rfcs/0000-rlp-v2.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 9742987d..58c5f591 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -164,7 +164,7 @@ spec: In this example, a distinct limit will be associated ("bound") to each individual HTTPRouteRule of the targeted HTTPRoute, by using the `triggers` field for selecting (or "sub-targeting") the HTTPRouteRule. The following limit definitions will be bound to each HTTPRouteRule: -- `GET|POST /toys*` → 50rpm, enforced per username (counter qualifier) and only in case the user is not an admin ("soft" condition). +- `/toys*` → 50rpm, enforced per username (counter qualifier) and only in case the user is not an admin ("soft" condition). - `/assets/*` → 5rpm / 100rp12h Each set of trigger matches in the RLP will be matched to **all** HTTPRouteRules whose HTTPRouteMatches is a superset of the set of trigger matches in the RLP. For every HTTPRouteRule matched, the HTTPRouteRule will be bound to the corresponding limit definition that specifies that trigger. In case no HTTPRouteRule is found containing at least one HTTPRouteMatch that is identical to some set of matching rules of a particular limit definition, the limit definition is considered invalid and reported as such in the status of RLP. @@ -193,11 +193,6 @@ spec: - path: type: PathPrefix value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST when: - selector: auth.identity.group operator: neq
1:1 relation between Limit (object) and the actual Rate limit (value)Rate becomes a detail of Limit where each limit may define one or more rates (1:N)1:1 relation between Limit (the object) and the actual Rate limit (the value) (spec.rateLimits.limits)Rate limit becomes a detail of Limit where each limit may define one or more rates (1:N) (spec.limits.rates)
  • It allows to reuse when conditions and counters for groups of rate limits
    • @@ -164,8 +366,8 @@ spec:
Parsed conditionsStructured when condition fields composed of { selector, operator, value }Parsed spec.rateLimits.limits.conditions field, directly exposing the Limitador's APIStructured spec.limits.when condition field composed of 3 well-defined properties: selector, operator and value
  • Feels more K8s-native
    • @@ -175,20 +377,20 @@ spec:
configurations field as a list of "variables assignments" - direct exposure of Envoy's RL "actions"No configurations field - descriptor actions composed from selectors used in the limit definitions only (when[].selector and counters[])spec.rateLimits.configurations as a list of "variables assignments" and direct exposure of Envoy's RL descriptor actions APIDescriptor actions composed implicitly from selectors used in the limit definitions (spec.limits.when.selector and spec.limits.counters) plus a fixed identifier of the route rule (spec.limits.matches)
  • Abstract the Envoy-specific concepts of "actions" and "descriptors"
  • No risk of mismatching descriptors keys between "actions" and actual usage in the limits
    • -
  • No (useless?) generic descriptors (e.g. "limited = 1")
    • +
  • No user-defined generic descriptors (e.g. "limited = 1")
  • Source value of the selectors defined from an implicit "context" data structure
Key-value descriptorsStructured descriptors from a "context" data structureStructured descriptors referring to a contextual well-known data structure
maxValuelimitLimitador conditions independent from the route rulesArtificial Limitador condition injected for each route match +
    +
  • Ensure the limit is enforced only for corresponding filtered HTTPRouteMatch
    • +
+
spec.rateLimits.rules ⊆ httproute.spec.rulesspec.limits.matches ∊ httproute.spec.rules.matches +
    +
  • Perfect match to HTTPRoute matching rules
    • +
  • Simpler to solve for defaults and overrides
    • +
+
secondsduration and unitspec.rateLimits.limits.secondsspec.limits.rates.duration and spec.limits.rates.unit
  • Support for more units beyond seconds
    • @@ -211,8 +427,26 @@ spec:
Fixed incrementincrement: {VALUE}spec.rateLimits.limits.variablesspec.limits.counters +
    +
  • Less generic naming
    • +
+
spec.rateLimits.limits.maxValuespec.limits.rates.limit +
    +
  • More generic naming
    • +
+
Fixed implicit increment of 1spec.limits.increment: {}
  • Flexibility to change the limits without reseting the counters
    • @@ -227,15 +461,13 @@ spec: ## Reference-level explanation [reference-level-explanation]: #reference-level-explanation -### Composing RL descriptor actions - -By completely dropping out the `configurations` field from the RLP, the `actions` to composing the RL _descriptor actions_ is now done based essentially on the selectors listed in the `when` conditions and `counters`. +By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus artificial generic conditions used to match the specific route rule (when `matches` is specified). -Each selector is a direct reference to a path within a _well-known data structure_ that holds the `context` (L4 and L7 data of the original request handled by the proxy), as well as occasionally the `auth` data (dynamic metadata exported by the external authorization filter and injected by the proxy into the rate-limit filter). +### Well-known selectors -#### The well-known data structure for user-defined RLP selectors and Kuadrant-generated RL descriptor actions +Each selector is a direct reference to a path within a well-known data structure that holds the `context` (L4 and L7 data of the original request handled by the proxy), as well as occasionally the `auth` data (dynamic metadata exported by the external authorization filter and injected by the proxy into the rate-limit filter). -The well-known data structure for building RL descriptor actions out of selectors in a RLP resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. +The well-known data structure for building RL descriptor actions resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. To keep compatibility with the [Envoy Rate Limit protocol](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. @@ -264,12 +496,12 @@ context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type hitsAddend: … # only in case we want to allow users to refer to this value in a policy ``` -#### Mechanics of generating RL descriptor actions +### Mechanics of generating RL descriptor actions -From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure. While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, +From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure ([_well-known selectors_](#well-known-selectors)). While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, The user story: -> _Whenever the context is a HTTP request to the path (`context.request.http.path`) that is equal to `"/toys"`, I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ +> _Whenever the context is an HTTP request sent to `dolls.toystore.com` (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ Materializes as the following RLP: @@ -286,9 +518,9 @@ spec: limits: - name: toys-path when: - - selector: context.request.http.path + - selector: context.request.http.host operator: eq - value: /toys + value: dolls.toystore.com rates: - limit: 50 duration: 1 @@ -298,14 +530,14 @@ spec: ``` The following selectors are to be interpreted: -- `context.request.http.path` +- `context.request.http.host` - `auth.identity.username` The RLP controller uses a map to translate each selector into its corresponding descriptor action. Roughly described: ``` -context.source.address → source_cluster(...) (?) -context.source.service → source_cluster(...) (?) +context.source.address → source_cluster(...) # TBC +context.source.service → source_cluster(...) # TBC context.destination... → destination_cluster(...) context.destination... → destination_cluster(...) context.request.http. → request_headers(header_name: ":") @@ -320,8 +552,8 @@ ratelimit.domain → rate_limits: - actions: - request_headers: - descriptor_key: "context.request.http.path" - header_name: ":path" + descriptor_key: "context.request.http.host" + header_name: ":host" - metadata: descriptor_key: "auth.identity.username" metadata_key: @@ -333,6 +565,112 @@ rate_limits: key: "username" ``` +### Artificial Limitador condition for `matches` + +For each limit, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when that one filterred matching rule is honoured to serve the request. This can be implemented with a 2-step procedure: +1. generate an unique identifier for the limit – e.g. by applying any sufficiently entropic hash function of choice to the `matches` block of the limit; +2. associate a `generic_key` type descriptor action to the each `HTTPRouteMatch` targeted by the limit. + +For example, given the following RLP ([Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates)): + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore-non-admin-users +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: readers + matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + rates: + - limit: 50 + duration: 1 + unit: minute + when: + - selector: auth.identity.group + operator: neq + value: admin + + - name: writers + matches: + - path: + type: PathPrefix + value: "/toys" + method: POST + rates: + - limit: 5 + duration: 1 + unit: minute + when: + - selector: auth.identity.group + operator: neq + value: admin +``` + +Apart from the descriptor action (associated with both routes): + +```yaml +- metadata: + descriptor_key: "auth.identity.group" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "group" +``` + +...and its corresponding Limitador condition: + +``` +auth.identity.group != "admin" +``` + +The following additional artificial descriptor actions will be generated: + +```yaml +# associated with route GET /toys* +- generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" # SHA256 hashing of [{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"}] + +# associated with route POST /toys* +- generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" # SHA256 hashing of [{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}] +``` + +...and their corresponding Limitador conditions. + +In the end, the following Limitador configuration is yielded: + +```yaml +- conditions: + - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - auth.identity.group != "admin" + max_value: 50 + seconds: 60 + namespace: "*.toystore.com" + +- conditions: + - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" + - auth.identity.group != "admin" + max_value: 5 + seconds: 60 + namespace: "*.toystore.com" +``` + +To improve readability, the unique hash identifiers can be suffixed with a plain identifier of the RLP itself (namespace and name) and the limit (limit name when available, limit index otherwise). E.g.: the two unique identifiers from above could become respectively `2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-non-admin-users/readers` and `4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-non-admin-users/writers`. + ## Drawbacks [drawbacks]: #drawbacks From b28df35991595fe4ea9b7b582b958dbf87288f1d Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 12:05:53 +0100 Subject: [PATCH 03/17] Added drawbacks --- rfcs/0000-rlp-v2.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 0f5f7bb1..8ec121ef 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -674,9 +674,11 @@ To improve readability, the unique hash identifiers can be suffixed with a plain ## Drawbacks [drawbacks]: #drawbacks - +**Two types of conditions – `matches` and `when` conditions**
    +Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and capable of expressing different sets of rules (HTTPRouteMatch-related rules vs. HTTP request-related and non HTTP request-related rules), an overlap between these two types and ways of representing conditions does exist. -[TODO] +**Prone to consistency issues**
    +Typos and updates to the HTTPRoute can easily cause a mismatch and invalidate a RLP. ## Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From 6d821578460a86ad5f81f38e06f0d3f4ac02d2b5 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 18:07:31 +0100 Subject: [PATCH 04/17] Added implementation details for each example --- rfcs/0000-rlp-v2.md | 267 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 267 insertions(+) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 8ec121ef..fa733b2b 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -113,6 +113,34 @@ spec: unit: second ``` +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + max_value: 5 + seconds: 1 + namespace: "*.toystore.com" + ``` +
    + ##### Example 2. Specific route matching rule targeted, conditions, counter qualifiers and multiple rates In this example, a distinct limit is associated to each individual matching rule of the targeted HTTPRoute, using the `matches` field for fine-grained filtering. In both cases, the rate limits are to be enforced per username and only in case the user is not an admin. @@ -168,6 +196,94 @@ spec: value: admin ``` +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-per-endpoint-per-user/readers" + - metadata: + descriptor_key: "auth.identity.group" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "group" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + - rules: + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-per-endpoint-per-user/writers" + - metadata: + descriptor_key: "auth.identity.group" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "group" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-per-endpoint-per-user/readers" + - auth.identity.group != "admin" + variables: + - auth.identity.username + max_value: 50 + seconds: 60 + namespace: "*.toystore.com" + - conditions: + - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-per-endpoint-per-user/writers" + - auth.identity.group != "admin" + variables: + - auth.identity.username + max_value: 5 + seconds: 60 + namespace: "*.toystore.com" + - conditions: + - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-per-endpoint-per-user/writers" + - auth.identity.group != "admin" + variables: + - auth.identity.username + max_value: 100 + seconds: 43200 # 12 hours + namespace: "*.toystore.com" + ``` +
    + ##### Example 3. Route filtering using `when` conditions `when` conditions are preferably to be used for special cases of conditional filtering based on values other than attributes of the route that could otherwise be specified using the `matches` field. However, in some cases such as where the condition is not related to the HTTPRoute (e.g. see [Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates) above; filterring based on hostname, etc) or a temporary measure while the target object misses the desired matching rule, the `when` conditions remains an option. @@ -206,6 +322,35 @@ On one hand, this makes it easy to set limits when either the HTTPRoute cannot b In the case of existing other limit definitions targeting the `GET /toys*` matching rule of the `toystore` HTTPRoute, because a merge strategy is expect to take into account the `matches` field as part of the qualification of the limit, there should be no problem of multiple simultaneous limits enforced to the same route rules, differenciated only by special conditions. Nevertheless, to avoid dealing of complex status reports including too many special conditions associated with a limit, users are instead encouraged to favor altering the HTTPRoutes for additional route rules that can be referred in the `matches` field preferably. +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-special-toys/read-special" + - request_headers: + descriptor_key: "context.request.http.path" + header_name: ":path" + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-special-toys/read-special" + - context.request.http.path == "/toys/special" + max_value: 150 + seconds: 1 + namespace: "*.toystore.com" + ``` +
    + ##### Example 4. Route filtering by refining the HTTPRoute To achieve the same goal as state in [Example 3](#example-3-route-filtering-using-when-conditions) yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `matches` field instead of the `when` conditions field. @@ -266,6 +411,31 @@ spec: unit: second ``` +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys/special"] + methods: ["GET"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c--toystore/toystore-special-toys/0" + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c--toystore/toystore-special-toys/0" + max_value: 150 + seconds: 1 + namespace: "*.toystore.com" + ``` +
    + ##### Example 5. One limit, two matches In this example, both route matching rules, `GET /toys*` and `POST /toys*`, are targeted by the same limit. This will cause the limit to be bound to the two HTTPRouteMatches, effectively applying 50rpm per username, regardless of the HTTP method `GET` or `POST`, at requests with path prefix equal to `/toys`. I.e. the rules are OR'ed, just like in the HTTPRoute itself. The same unique hash identifier is associated to both route rules. @@ -298,6 +468,45 @@ spec: - auth.identity.username ``` +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-per-endpoint-per-user/0" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-per-endpoint-per-user/0" + variables: + - auth.identity.username + max_value: 50 + seconds: 60 + namespace: "*.toystore.com" + ``` +
    + ##### Example 6. Dynamic rate with flexible increments By setting the increment of the limit, the rate can vary with no need to reset the counters. @@ -321,6 +530,36 @@ spec: value: 2 # each hit artificially made twice as expensive without reseting the counters ``` +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-dynamic/0" + - hits_addend: 2 # Not implemented - see https://github.com/envoyproxy/envoy/issues/12969 + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-dynamic/0" + max_value: 5 + seconds: 1 + namespace: "*.toystore.com" + increment: # Not implemented + ``` +
    + #### Targeting Gateways Targeting a Gateway is a shortcut to targeting individually each HTTPRoute pointing to the gateway, without any filtering based on the `matches` field. @@ -345,6 +584,34 @@ spec: unit: second ``` +
    + How is this RLP implemented under the hood? + + ```yaml + gateway_actions: + - rules: + - paths: ["/toys*"] + methods: ["GET"] + hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + ``` + + ```yaml + limits: + - conditions: + - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + max_value: 5 + seconds: 1 + namespace: "*.toystore.com" + ``` +
    + ### Comparison to current RateLimitPolicy From d45ae53866b8b438067bfc89f95e478e38ae7b7e Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 18:11:21 +0100 Subject: [PATCH 05/17] Example section headers moved one level up --- rfcs/0000-rlp-v2.md | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index fa733b2b..a5c900f1 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -91,9 +91,7 @@ spec: The following are examples of RLPs targeting the route and the gateway. -#### Targeting routes and route matching rules - -##### Example 1. Minimal example - network resource targeted entirely without filtering, unconditional and unqualified rate limiting +#### Example 1. Minimal example - network resource targeted entirely without filtering, unconditional and unqualified rate limiting In this example, all traffic to `*.toystore.com` will be limited to 5 rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. @@ -141,7 +139,7 @@ spec: ``` -##### Example 2. Specific route matching rule targeted, conditions, counter qualifiers and multiple rates +#### Example 2. Specific route matching rule targeted, conditions, counter qualifiers and multiple rates In this example, a distinct limit is associated to each individual matching rule of the targeted HTTPRoute, using the `matches` field for fine-grained filtering. In both cases, the rate limits are to be enforced per username and only in case the user is not an admin. @@ -284,7 +282,7 @@ spec: ``` -##### Example 3. Route filtering using `when` conditions +#### Example 3. Route filtering using `when` conditions `when` conditions are preferably to be used for special cases of conditional filtering based on values other than attributes of the route that could otherwise be specified using the `matches` field. However, in some cases such as where the condition is not related to the HTTPRoute (e.g. see [Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates) above; filterring based on hostname, etc) or a temporary measure while the target object misses the desired matching rule, the `when` conditions remains an option. @@ -351,7 +349,7 @@ In the case of existing other limit definitions targeting the `GET /toys*` match ``` -##### Example 4. Route filtering by refining the HTTPRoute +#### Example 4. Route filtering by refining the HTTPRoute To achieve the same goal as state in [Example 3](#example-3-route-filtering-using-when-conditions) yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `matches` field instead of the `when` conditions field. @@ -436,7 +434,7 @@ spec: ``` -##### Example 5. One limit, two matches +#### Example 5. One limit, two matches In this example, both route matching rules, `GET /toys*` and `POST /toys*`, are targeted by the same limit. This will cause the limit to be bound to the two HTTPRouteMatches, effectively applying 50rpm per username, regardless of the HTTP method `GET` or `POST`, at requests with path prefix equal to `/toys`. I.e. the rules are OR'ed, just like in the HTTPRoute itself. The same unique hash identifier is associated to both route rules. @@ -507,7 +505,7 @@ spec: ``` -##### Example 6. Dynamic rate with flexible increments +#### Example 6. Dynamic rate with flexible increments By setting the increment of the limit, the rate can vary with no need to reset the counters. @@ -560,14 +558,12 @@ spec: ``` -#### Targeting Gateways +#### Example 7. Targeting the Gateway Targeting a Gateway is a shortcut to targeting individually each HTTPRoute pointing to the gateway, without any filtering based on the `matches` field. > **Note:** it is hard to give any additional meaning and context to this without going into [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). -##### Example 7. Targeting the Gateway - ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy From de3cd2ebf82d02e2361cbbe561a4637ab4b66810 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 18:16:52 +0100 Subject: [PATCH 06/17] Future possibilities --- rfcs/0000-rlp-v2.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index a5c900f1..c18503ee 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -1034,10 +1034,5 @@ Note that while precedent set by other projects is some motivation, it does not ## Future possibilities [future-possibilities]: #future-possibilities - - -[TODO] +- Port `matches` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). +- Defaults and overrides, either along the lines of [architecture#4](https://github.com/Kuadrant/architecture/pull/4) or [architecture#10](https://github.com/Kuadrant/architecture/pull/10). From 6b08d20f0aa01ac20cbb77ce02697c32de010726 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 18:22:54 +0100 Subject: [PATCH 07/17] Prior art --- rfcs/0000-rlp-v2.md | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index c18503ee..f5677d58 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -1011,18 +1011,7 @@ counters: ## Prior art [prior-art]: #prior-art - - -[TODO] +Most implementations currently orbiting around Gateway API (e.g. Istio, Envoy Gateway, etc) for added RL functionality seem to have been leaning more to the direct route extension pattern instead of Policy Attachment. That might be an option particularly suitable for gateway implementations (gateway providers) and for those aiming to avoid dealing with defaults and overrides. ## Unresolved questions [unresolved-questions]: #unresolved-questions From d4767a682308ecc533961f3910aa2ad17c78c92a Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 18:24:15 +0100 Subject: [PATCH 08/17] Added date and issue refs --- rfcs/0000-rlp-v2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index f5677d58..42b91710 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -1,9 +1,9 @@ # RFC RateLimitPolicy v2 - Feature Name: `rlp-v2` -- Start Date: (fill me in with today's date, YYYY-MM-DD) -- RFC PR: [Kuadrant/architecture#0000](https://github.com/Kuadrant/architecture/pull/0000) -- Issue tracking: [Kuadrant/architecture#0000](https://github.com/Kuadrant/architecture/issues/0000) +- Start Date: 2023-02-02 +- RFC PR: [Kuadrant/architecture#8](https://github.com/Kuadrant/architecture/pull/8) +- Issue tracking: N/A ## Summary [summary]: #summary From 09cf8e64415b8448a6978f599bae867e6f905beb Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Tue, 28 Feb 2023 18:27:00 +0100 Subject: [PATCH 09/17] Added pr #10 to the watchlist --- rfcs/0000-rlp-v2.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 42b91710..cdb6438f 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -27,9 +27,10 @@ Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/k ### Current WIP to consider -1. A single Policy scoped to HTTPRoutes and HTTPRouteRule (https://github.com/Kuadrant/architecture/pull/4) -2. Implement `skip_if_absent` for the RequestHeaders action (https://github.com/Kuadrant/wasm-shim/issues/29) -3. Rate Limiting across clusters ([doc](https://docs.google.com/document/d/1pqCODRAkNUTLB6lJfRWcv3d2Hlj9WqsGySmjrP707Vk/edit#heading=h.nzpgr3pef6uy)) +1. A single Policy scoped to HTTPRoutes and HTTPRouteRule ([architecture#4](https://github.com/Kuadrant/architecture/pull/4)) +2. *No* merging of policies ([architecture#10](https://github.com/Kuadrant/architecture/pull/10)) +3. Implement `skip_if_absent` for the RequestHeaders action ([wasm-shim#29](https://github.com/Kuadrant/wasm-shim/issues/29)) +4. Rate Limiting across clusters ([doc](https://docs.google.com/document/d/1pqCODRAkNUTLB6lJfRWcv3d2Hlj9WqsGySmjrP707Vk/edit#heading=h.nzpgr3pef6uy)) ### Highlights From cbd965b9891bbb9dbce829960047283f4f7e758e Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Wed, 1 Mar 2023 11:28:48 +0100 Subject: [PATCH 10/17] fix: header name :host -> :authority --- rfcs/0000-rlp-v2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index cdb6438f..11e93849 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -817,7 +817,7 @@ rate_limits: - actions: - request_headers: descriptor_key: "context.request.http.host" - header_name: ":host" + header_name: ":authority" - metadata: descriptor_key: "auth.identity.username" metadata_key: From 508f719e5b9e8e41a56c0ef292a28f852cbad7bf Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Fri, 3 Mar 2023 10:20:43 +0100 Subject: [PATCH 11/17] remove the concept of increment from the rfc - it deserves its own proposal to be discussed separately --- rfcs/0000-rlp-v2.md | 67 +-------------------------------------------- 1 file changed, 1 insertion(+), 66 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index 11e93849..a4aa4a24 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -43,7 +43,6 @@ Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/k - `spec.rateLimits.rules` replaced by `spec.limits.matches`, where each match is a perfect match to one of the `HTTPRouteMatch`es of the targeted `HTTPRoute` (`httproute.spec.rules.matches`) – complies with [architecture#4](https://github.com/Kuadrant/architecture/pull/4) - `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.matches` - Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted `HTTPRouteMatch` -- `spec.limits.increment` introduced For detailed differences between current and vew RLP API, see [Comparison to current RateLimitPolicy](#comparison-to-current-ratelimitpolicy). @@ -506,60 +505,7 @@ spec: ``` -#### Example 6. Dynamic rate with flexible increments - -By setting the increment of the limit, the rate can vary with no need to reset the counters. - -```yaml -apiVersion: kuadrant.io/v2beta1 -kind: RateLimitPolicy -metadata: - name: toystore-dynamic -spec: - targetRef: - group: gateway.networking.k8s.io - kind: HTTPRoute - name: toystore - limits: - - name: dynamic-rate - rates: - - limit: 100 - unit: second - increment: - value: 2 # each hit artificially made twice as expensive without reseting the counters -``` - -
    - How is this RLP implemented under the hood? - - ```yaml - gateway_actions: - - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] - configurations: - - generic_key: - descriptor_key: "context.request.http.route_matcher" - descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-dynamic/0" - - hits_addend: 2 # Not implemented - see https://github.com/envoyproxy/envoy/issues/12969 - ``` - - ```yaml - limits: - - conditions: - - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-dynamic/0" - max_value: 5 - seconds: 1 - namespace: "*.toystore.com" - increment: # Not implemented - ``` -
    - -#### Example 7. Targeting the Gateway +#### Example 6. Targeting the Gateway Targeting a Gateway is a shortcut to targeting individually each HTTPRoute pointing to the gateway, without any filtering based on the `matches` field. @@ -708,17 +654,6 @@ spec: -     - - - -
    Fixed implicit increment of 1spec.limits.increment: {} -
      -
    • Flexibility to change the limits without reseting the counters
      • -
    • Placeholder for more complex RL systems in the future – e.g. dynamic-RL, %, 𝑓(), etc
      • -
    • Migration path from other RL solutions, e.g. 3scale (?)
      • -
    -
    From 3092396e6514a69ebb68cc484a3d735b30eeba95 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Fri, 3 Mar 2023 10:46:14 +0100 Subject: [PATCH 12/17] fix unique hash route matcher identifiers --- rfcs/0000-rlp-v2.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index a4aa4a24..aba47bc9 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -126,13 +126,13 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + descriptor_value: "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" max_value: 5 seconds: 1 namespace: "*.toystore.com" @@ -206,7 +206,7 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-per-endpoint-per-user/readers" + descriptor_value: "toystore/toystore-per-endpoint-per-user/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" - metadata: descriptor_key: "auth.identity.group" metadata_key: @@ -232,7 +232,7 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-per-endpoint-per-user/writers" + descriptor_value: "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - metadata: descriptor_key: "auth.identity.group" metadata_key: @@ -256,7 +256,7 @@ spec: ```yaml limits: - conditions: - - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-per-endpoint-per-user/readers" + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" - auth.identity.group != "admin" variables: - auth.identity.username @@ -264,7 +264,7 @@ spec: seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-per-endpoint-per-user/writers" + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - auth.identity.group != "admin" variables: - auth.identity.username @@ -272,7 +272,7 @@ spec: seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-per-endpoint-per-user/writers" + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - auth.identity.group != "admin" variables: - auth.identity.username @@ -332,7 +332,7 @@ In the case of existing other limit definitions targeting the `GET /toys*` match configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-special-toys/read-special" + descriptor_value: "toystore/toystore-special-toys/read-special/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" - request_headers: descriptor_key: "context.request.http.path" header_name: ":path" @@ -341,7 +341,7 @@ In the case of existing other limit definitions targeting the `GET /toys*` match ```yaml limits: - conditions: - - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-special-toys/read-special" + - context.request.http.route_matcher == "toystore/toystore-special-toys/read-special/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" - context.request.http.path == "/toys/special" max_value: 150 seconds: 1 @@ -421,13 +421,13 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c--toystore/toystore-special-toys/0" + descriptor_value: "toystore/toystore-special-toys/0/d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c--toystore/toystore-special-toys/0" + - context.request.http.route_matcher == "toystore/toystore-special-toys/0/d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c" max_value: 150 seconds: 1 namespace: "*.toystore.com" @@ -481,7 +481,7 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-per-endpoint-per-user/0" + descriptor_value: "toystore/toystore-per-endpoint-per-user/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" - metadata: descriptor_key: "auth.identity.username" metadata_key: @@ -496,7 +496,7 @@ spec: ```yaml limits: - conditions: - - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-per-endpoint-per-user/0" + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" variables: - auth.identity.username max_value: 50 @@ -542,13 +542,13 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + descriptor_value: "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07--toystore/toystore-simple-infra-rl/0" + - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" max_value: 5 seconds: 1 namespace: "*.toystore.com" @@ -868,7 +868,7 @@ In the end, the following Limitador configuration is yielded: namespace: "*.toystore.com" ``` -To improve readability, the unique hash identifiers can be suffixed with a plain identifier of the RLP itself (namespace and name) and the limit (limit name when available, limit index otherwise). E.g.: the two unique identifiers from above could become respectively `2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05--toystore/toystore-non-admin-users/readers` and `4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d--toystore/toystore-non-admin-users/writers`. +The route matcher hash identifiers can be qualified with a plain identifier of the RLP itself (`namespace/name`) and limit where it is defined (`name` of the limit when available, index in the array of limits otherwise), thus making the identifier unique to scope of the entire cluster. E.g.: the two unique identifiers from above, prefixed with the unique limit qualifier, become respectively `toystore/toystore-non-admin-users/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05` and `toystore/toystore-non-admin-users/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d`. This has a consequence to the readability of the identifier, but also and more importantly it ensures uniqueness of the counters in Limitador. By qualifying (or salting) the identifiers, two limits or two RLPs that happen to target the same `HTTPRouteMatches` will not register the same counters in Limitador, but be treated as independent ones instead. ## Drawbacks [drawbacks]: #drawbacks From 874bd7d9bc532fede27bb09e7d0daf877359eef6 Mon Sep 17 00:00:00 2001 From: Guilherme Cassolato Date: Mon, 6 Mar 2023 19:53:16 +0100 Subject: [PATCH 13/17] Atomic targeting of HTTPRouteRules --- rfcs/0000-rlp-v2.md | 374 ++++++++++++++++++++++++++------------------ 1 file changed, 221 insertions(+), 153 deletions(-) diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index aba47bc9..1f9312b7 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -34,15 +34,15 @@ Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/k ### Highlights -- `spec.rateLimits` replaced by `spec.limits` -- `spec.rateLimits.limits` replaced by `spec.limits.rates` -- `spec.rateLimits.limits.maxValue` replaced by `spec.limits.rates.limit` -- `spec.rateLimits.limits.seconds` replaced by `spec.limits.rates.duration` + `spec.limits.rates.unit` -- `spec.rateLimits.limits.conditions` replaced by `spec.limits.when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) -- `spec.rateLimits.limits.variables` replaced by `spec.limits.counters`, based on _well-known selectors_ -- `spec.rateLimits.rules` replaced by `spec.limits.matches`, where each match is a perfect match to one of the `HTTPRouteMatch`es of the targeted `HTTPRoute` (`httproute.spec.rules.matches`) – complies with [architecture#4](https://github.com/Kuadrant/architecture/pull/4) -- `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.matches` -- Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted `HTTPRouteMatch` +- `spec.rateLimits` replaced with `spec.limits` +- `spec.rateLimits.limits` replaced with `spec.limits.rates` +- `spec.rateLimits.limits.maxValue` replaced with `spec.limits.rates.limit` +- `spec.rateLimits.limits.seconds` replaced with `spec.limits.rates.duration` + `spec.limits.rates.unit` +- `spec.rateLimits.limits.conditions` replaced with `spec.limits.when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) +- `spec.rateLimits.limits.variables` replaced with `spec.limits.counters`, based on _well-known selectors_ +- `spec.rateLimits.rules` replaced with `spec.limits.rules`, for "sub-targeting" HTTPRouteRules within an HTTPRoute +- `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.rules` +- Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted HTTPRouteRules For detailed differences between current and vew RLP API, see [Comparison to current RateLimitPolicy](#comparison-to-current-ratelimitpolicy). @@ -87,13 +87,26 @@ spec: backendRefs: - name: toystore port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: max-age=31536000, immutable ``` -The following are examples of RLPs targeting the route and the gateway. +The following are examples of RLPs targeting the route and the gateway. Each example is independent from the other. #### Example 1. Minimal example - network resource targeted entirely without filtering, unconditional and unqualified rate limiting -In this example, all traffic to `*.toystore.com` will be limited to 5 rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. +In this example, all traffic to `*.toystore.com` will be limited to 5rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. ```yaml apiVersion: kuadrant.io/v2beta1 @@ -126,42 +139,57 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + descriptor_value: "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + - rules: + - paths: ["/assets/*"] + methods: ["*"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" max_value: 5 seconds: 1 namespace: "*.toystore.com" ``` -#### Example 2. Specific route matching rule targeted, conditions, counter qualifiers and multiple rates +#### Example 2. Targeting specific route rules, plus conditions, counter qualifiers and multiple rates -In this example, a distinct limit is associated to each individual matching rule of the targeted HTTPRoute, using the `matches` field for fine-grained filtering. In both cases, the rate limits are to be enforced per username and only in case the user is not an admin. +In this example, a distinct limit will be associated to each individual rule of the targeted HTTPRoute, using the `rules` field for fine-grained filtering (or "sub-targeting"). +- `GET|POST /toys*` → 50rpm, enforced per username (counter qualifier) and only in case the user is not an admin (condition). +- `/assets/*` → 5rpm / 100rp12h -For each matching rule in the RLP, the RLP controller will try to find a matching rule in the HTTPRoute that is an identical match to it and bind the two matching rules together; the first identical match prevails. In case there is no identical match in the HTTPRoute, the RLP is considered invalid. In case there is more than one matching rule specified in the RLP that is an identical match to the same matching rule in the HTTPRoute, the first matching rule on the list in the RLP is bound to its identical match in the HTTPRoute, thus "shadowing" any other rule on the list that is also an identical match. +Each rule in the RLP must perfectly match a rule in the HTTPRoute. In case there is no identical match amongst the HTTPRoute rules for a given rule defined in the RLP, the RLP is considered invalid. In case there is more than one rule specified in the RLP that is an identical match to the same rule in the HTTPRoute, the first matching rule on the list in the RLP is bound to its identical match in the HTTPRoute, thus "shadowing" any other rule on the list that is also an identical match. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-per-endpoint-per-user + name: toystore-per-endpoint spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: readers - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET + - name: toys + rules: + - matches: # matches the 1st rule in the HTTPRoute + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 50 duration: 1 @@ -173,12 +201,12 @@ spec: operator: neq value: admin - - name: writers - matches: - - path: - type: PathPrefix - value: "/toys" - method: POST + - name: assets + rules: + - matches: # matches the 2nd rule in the HTTPRoute + - path: + type: PathPrefix + value: "/assets/" rates: - limit: 5 duration: 1 @@ -186,12 +214,6 @@ spec: - limit: 100 duration: 12 unit: hour - counters: - - auth.identity.username - when: - - selector: auth.identity.group - operator: neq - value: admin ```
    @@ -203,10 +225,13 @@ spec: - paths: ["/toys*"] methods: ["GET"] hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint-per-user/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + descriptor_value: "toystore/toystore-per-endpoint/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] - metadata: descriptor_key: "auth.identity.group" metadata_key: @@ -226,37 +251,19 @@ spec: - segment: key: "username" - rules: - - paths: ["/toys*"] - methods: ["POST"] + - paths: ["/assets/*"] + methods: ["*"] hosts: ["*.toystore.com"] configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - - metadata: - descriptor_key: "auth.identity.group" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "group" - - metadata: - descriptor_key: "auth.identity.username" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "username" + descriptor_value: "toystore/toystore-per-endpoint/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - auth.identity.group != "admin" variables: - auth.identity.username @@ -264,18 +271,12 @@ spec: seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - - auth.identity.group != "admin" - variables: - - auth.identity.username + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" max_value: 5 seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - - auth.identity.group != "admin" - variables: - - auth.identity.username + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" max_value: 100 seconds: 43200 # 12 hours namespace: "*.toystore.com" @@ -284,9 +285,11 @@ spec: #### Example 3. Route filtering using `when` conditions -`when` conditions are preferably to be used for special cases of conditional filtering based on values other than attributes of the route that could otherwise be specified using the `matches` field. However, in some cases such as where the condition is not related to the HTTPRoute (e.g. see [Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates) above; filterring based on hostname, etc) or a temporary measure while the target object misses the desired matching rule, the `when` conditions remains an option. +This is a similar RLP to the one from [Example 2](#example-2-targeting-specific-route-rules-plus-conditions-counter-qualifiers-and-multiple-rates) regarding the use case of applying specific limits to portions of an HTTPRoute. Whereas in the previous example the route rules map perfectly for the limits to be applied, in this example we use `when` conditions to apply the limit to a subpath without having a corresponding HTTPRouteRule that could be referred using the `rules` field. -In this example, a special limit with one rate limit of 150 rps is set for `GET` requests to the `/toys/special` path. Because the `toystore` HTTPRoute is unaltered, the special case is defined using the `when` conditions field. +`when` conditions are to be used preferably for special cases of conditional filtering based on values other than attributes of the HTTP request that otherwise could be specified using the `rules` field of the RLP. However, in some cases such as where the conditions are not related to the HTTPRoute (e.g. filterring based on hostname, filterring based on metadata) or while the targeted object (HTTPRoute or Gateway) misses the desired rule and cannot be changed, the `when` conditions remain an option. + +In this example, a special limit with one rate limit of 150rps is set for `GET /toys/special`. Ideally, an additional HTTPRoute rule would be created and targeted using the `rules` field, with resulting benefits for optimization, status reporting and matching. But because the `toystore` HTTPRoute is unaltered, the special case is implemented using the `when` conditions field instead. ```yaml apiVersion: kuadrant.io/v2beta1 @@ -300,25 +303,35 @@ spec: name: toystore limits: - name: read-special - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET + rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: # must be included as well or it wouldn't match the http route rule + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 150 unit: second when: + - selector: context.request.http.method + operator: eq + value: GET - selector: context.request.http.path operator: eq value: /toys/special ``` -By using the route rule as-is, simply `GET /toys*` instead of more specific `GET /toys/special`, the rate limit filter (wasm filter) will continue invoking the rate limit service (Limitador) for all requests that match HTTP method equal to `GET` and path prefix equal to `/toys`, including requests to paths other than `/toys/special`, but the limit will only be enforced when the path fully matches `/toys/special`, as specified in the `when` condition. +By not targeting a specific HTTPRoute rule added for `GET /toys/special`, the rate limit filter (wasm shim) will continue invoking the rate limit service (Limitador) for all requests that match the HTTP the path prefix `/toys` (methods `GET` and `POST`), as targeted HTTPRoute rule. This includes requests to paths other than `GET /toys/special` nevertheless. However, the limit will be ensured to be enforced in Limitador only when the request matches `GET /toys/special`. + +Technically, the `rules` field did not have to be included in the RLP, and the `when` would still ensure the limit applies only to requests to `GET /toys/special`. However, in this case the other HTTPRoute rule (`/assets/*`) would also be initally targeted by the limit, though never effectivelly triggered. -On one hand, this makes it easy to set limits when either the HTTPRoute cannot be modified to include a special matching rule or as a shortcut in cases where multiple HTTPRoutes would have to be touched, such when targeting a Gateway with special conditions based on attributes of the route. On the other hand however, users might miss information in the status in a scenario where the status of rate limiting is reported at the level of the route rule or rule matcher. Effectively, the route rule `GET /toys*` might be reported as rate limited to '150 rps' when that is actually only the case of requests to `GET /toys/special`. These special conditions of the limit definition need therefore to be included in the status. +On one hand, using `when` conditions for route filterring makes it easy to set limits when either the HTTPRoute cannot be modified to include the special rule or as a shortcut in cases where multiple HTTPRoutes would have to be touched (e.g., when targeting a Gateway with special conditions based on attributes of the HTTP request). On the other hand however, users might miss information in the status in a scenario where the status of rate limiting is reported at the level of the route rule. Effectively, the route rule for `GET|POST /toys*` in the above example might be reported as rate limited to '150rps' with additional details that this is in fact only when requests match `GET /toys/special`. -In the case of existing other limit definitions targeting the `GET /toys*` matching rule of the `toystore` HTTPRoute, because a merge strategy is expect to take into account the `matches` field as part of the qualification of the limit, there should be no problem of multiple simultaneous limits enforced to the same route rules, differenciated only by special conditions. Nevertheless, to avoid dealing of complex status reports including too many special conditions associated with a limit, users are instead encouraged to favor altering the HTTPRoutes for additional route rules that can be referred in the `matches` field preferably. +In the case of existing other limit definitions targeting the `GET|POST /toys*` rule of the `toystore` HTTPRoute, because any merge strategy is expected to take into account the `rules` field as part of the qualification of the limit, all limits are guaranteed to be enforced, and only differenciated by the special "soft" the condition. Nevertheless, to avoid dealing with complex status reports including too many special conditions associated with a limit, **users are encouraged to favor altering the HTTPRoutes for additional route rules in all cases where the rules can be referred in the `rules` field of the RLP, whenever possible**.
    How is this RLP implemented under the hood? @@ -329,10 +342,13 @@ In the case of existing other limit definitions targeting the `GET /toys*` match - paths: ["/toys*"] methods: ["GET"] hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-special-toys/read-special/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + descriptor_value: "toystore/toystore-special-toys/read-special/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] - request_headers: descriptor_key: "context.request.http.path" header_name: ":path" @@ -341,7 +357,7 @@ In the case of existing other limit definitions targeting the `GET /toys*` match ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-special-toys/read-special/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - context.request.http.route_matcher == "toystore/toystore-special-toys/read-special/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - context.request.http.path == "/toys/special" max_value: 150 seconds: 1 @@ -351,9 +367,9 @@ In the case of existing other limit definitions targeting the `GET /toys*` match #### Example 4. Route filtering by refining the HTTPRoute -To achieve the same goal as state in [Example 3](#example-3-route-filtering-using-when-conditions) yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `matches` field instead of the `when` conditions field. +To achieve the same goal as stated in [Example 3](#example-3-route-filtering-using-when-conditions), yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition details associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `rules` field instead of the `when` conditions field. -New matching rule added to the HTTPRoute: +New rule added to the HTTPRoute: ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 @@ -368,10 +384,6 @@ spec: hostnames: ["*.toystore.com"] rules: - matches: - - path: # new matching rule added so it can be targeted by the RLP - type: Exact - value: "/toys/special" - method: GET - path: type: PathPrefix value: "/toys" @@ -383,6 +395,27 @@ spec: backendRefs: - name: toystore port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: max-age=31536000, immutable + - matches: # new rule added so it can be targeted in the RLP + - path: + type: Exact + value: "/toys/special" + method: GET + backendRefs: + - name: toystore + port: 80 ``` RateLimitPolicy: @@ -399,11 +432,12 @@ spec: name: toystore limits: - name: read-special - matches: - - path: - type: Exact - value: "/toys/special" - method: GET + rules: + - matches: + - path: + type: Exact + value: "/toys/special" + method: GET rates: - limit: 150 unit: second @@ -421,43 +455,50 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-special-toys/0/d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c" + descriptor_value: "toystore/toystore-special-toys/0/8267df441e5cda729095a9ea78db3abb2420855f7152379b4e88c90b8a4f562e" # SHA256 hashing of [{"matches":{"path":{"type":"Exact","value":"/toys/special"},"method":"GET"}}] ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-special-toys/0/d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c" + - context.request.http.route_matcher == "toystore/toystore-special-toys/0/8267df441e5cda729095a9ea78db3abb2420855f7152379b4e88c90b8a4f562e" max_value: 150 seconds: 1 namespace: "*.toystore.com" ```
    -#### Example 5. One limit, two matches +#### Example 5. One limit, many rules + +In this example, both HTTPRoute rules, i.e. `GET|POST /toys*` and `/assets/*`, are targeted by the same limit of 50rpm per username. -In this example, both route matching rules, `GET /toys*` and `POST /toys*`, are targeted by the same limit. This will cause the limit to be bound to the two HTTPRouteMatches, effectively applying 50rpm per username, regardless of the HTTP method `GET` or `POST`, at requests with path prefix equal to `/toys`. I.e. the rules are OR'ed, just like in the HTTPRoute itself. The same unique hash identifier is associated to both route rules. +Because the HTTPRoute has no other rule, this is technically equivalent to targeting the entire HTTPRoute and therefore similar to [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting). But if the HTTPRoute had other rules or got other rules added afterwards, this would ensure the limit applies only to the two original route rules. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-per-endpoint-per-user + name: toystore-per-user spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST + - rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + - matches: + - path: + type: PathPrefix + value: "/assets/" rates: - limit: 50 duration: 1 @@ -481,7 +522,24 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint-per-user/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + descriptor_value: "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + - rules: + - paths: ["/assets/*"] + methods: ["*"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] - metadata: descriptor_key: "auth.identity.username" metadata_key: @@ -496,7 +554,7 @@ spec: ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + - context.request.http.route_matcher == "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" variables: - auth.identity.username max_value: 50 @@ -507,9 +565,11 @@ spec: #### Example 6. Targeting the Gateway -Targeting a Gateway is a shortcut to targeting individually each HTTPRoute pointing to the gateway, without any filtering based on the `matches` field. +> **Note:** Additional meaning and context may be given to this use case in the future, when discussing [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). -> **Note:** it is hard to give any additional meaning and context to this without going into [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). +Targeting a Gateway is a shortcut to targeting all individual HTTPRoutes referencing to the gateway as parent, without any filtering based on the `rules` field. + +This differs from [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting) because, by targeting the gateway rather than an individual HTTPRoute, the RLP applies automatically to all HTTPRoutes pointing to the gateway, including routes created before and after the creation of the RLP. Moreover, all those routes will share the same limit counters specified in the RLP. ```yaml apiVersion: kuadrant.io/v2beta1 @@ -530,25 +590,20 @@ spec:
    How is this RLP implemented under the hood? + Because there is no matcher in the Gateway, this limit rather have to define a generic key descriptor that is valid for all routes – i.e. without the [artificial Limitador condition](#artificial-limitador-condition-for-rules) associated with the `rules` field, such as one that identifies the RLP only. + ```yaml gateway_actions: - - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] - configurations: + - configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + descriptor_value: "toystore/toystore-simple-infra-rl" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl" max_value: 5 seconds: 1 namespace: "*.toystore.com" @@ -588,7 +643,7 @@ spec:
spec.rateLimits.configurations as a list of "variables assignments" and direct exposure of Envoy's RL descriptor actions APIDescriptor actions composed implicitly from selectors used in the limit definitions (spec.limits.when.selector and spec.limits.counters) plus a fixed identifier of the route rule (spec.limits.matches)Descriptor actions composed implicitly from selectors used in the limit definitions (spec.limits.when.selector and spec.limits.counters) plus a fixed identifier of the route rules (spec.limits.rules)
  • Abstract the Envoy-specific concepts of "actions" and "descriptors"
    • @@ -609,19 +664,19 @@ spec:
Limitador conditions independent from the route rulesArtificial Limitador condition injected for each route matchArtificial Limitador condition injected for each route rule
    -
  • Ensure the limit is enforced only for corresponding filtered HTTPRouteMatch
    • +
  • Ensure the limit is enforced only for corresponding filtered HTTPRouteRule
spec.rateLimits.rules ⊆ httproute.spec.rulesspec.limits.matches ∊ httproute.spec.rules.matchesspec.limits.rules.matches == httproute.spec.rules.matches
    -
  • Perfect match to HTTPRoute matching rules
    • +
  • Perfect match to HTTPRoute rules
  • Simpler to solve for defaults and overrides
1:1 relation between Limit (the object) and the actual Rate limit (the value) (spec.rateLimits.limits)Rate limit becomes a detail of Limit where each limit may define one or more rates (1:N) (spec.limits.rates)Rate limit becomes a detail of Limit where each limit may define one or more rates (1:N) (spec.limits.<limit-name>.rates)
  • It allows to reuse when conditions and counters for groups of rate limits
    • @@ -632,7 +882,7 @@ spec:
Parsed spec.rateLimits.limits.conditions field, directly exposing the Limitador's APIStructured spec.limits.when condition field composed of 3 well-defined properties: selector, operator and valueStructured spec.limits.<limit-name>.when condition field composed of 3 well-defined properties: selector, operator and value
  • Feels more K8s-native
    • @@ -643,7 +893,7 @@ spec:
spec.rateLimits.configurations as a list of "variables assignments" and direct exposure of Envoy's RL descriptor actions APIDescriptor actions composed implicitly from selectors used in the limit definitions (spec.limits.when.selector and spec.limits.counters) plus a fixed identifier of the route rules (spec.limits.rules)Descriptor actions composed from selectors used in the limit definitions (spec.limits.<limit-name>.when.selector and spec.limits.<limit-name>.counters) plus a fixed identifier of the route rules (spec.limits.<limit-name>.triggers)
  • Abstract the Envoy-specific concepts of "actions" and "descriptors"
    • @@ -664,26 +914,27 @@ spec:
Limitador conditions independent from the route rulesArtificial Limitador condition injected for each route ruleArtificial Limitador condition injected to bind routes and corresponding limits
    -
  • Ensure the limit is enforced only for corresponding filtered HTTPRouteRule
    • +
  • Ensure the limit is enforced only for corresponding selected HTTPRouteRules
spec.rateLimits.rules ⊆ httproute.spec.rulesspec.limits.rules.matches == httproute.spec.rules.matchestranslate(spec.rateLimits.rules) ⊂ httproute.spec.rulesspec.limits.<limit-name>.triggers.matches ⊆ httproute.spec.rules.matches
    -
  • Perfect match to HTTPRoute rules
    • -
  • Simpler to solve for defaults and overrides
    • +
  • HTTPRouteRule selector (via HTTPRouteMatch subset)
    • +
  • Gateway API language
    • +
  • Preparation for inherited policies and defaults & overrides
spec.rateLimits.limits.secondsspec.limits.rates.duration and spec.limits.rates.unitspec.limits.<limit-name>.rates.duration and spec.limits.<limit-name>.rates.unit
  • Support for more units beyond seconds
    • @@ -693,19 +944,19 @@ spec:
spec.rateLimits.limits.variablesspec.limits.countersspec.limits.<limit-name>.counters
    -
  • Less generic naming
    • +
  • Improved (more specific) naming
spec.rateLimits.limits.maxValuespec.limits.rates.limitspec.limits.<limit-name>.rates.limit
    -
  • More generic naming
    • +
  • Improved (more generic) naming