diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md new file mode 100644 index 00000000..cf5cfeae --- /dev/null +++ b/rfcs/0000-rlp-v2.md @@ -0,0 +1,417 @@ +# RFC RateLimitPolicy v2 + +- Feature Name: `rlp-v2` +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- RFC PR: [Kuadrant/architecture#0000](https://github.com/Kuadrant/architecture/pull/0000) +- Issue tracking: [Kuadrant/architecture#0000](https://github.com/Kuadrant/architecture/issues/0000) + +## Summary +[summary]: #summary + +Proposal of new API for the Kuadrant's `RateLimitPolicy` (RLP) CRD, for improved UX. + +## Motivation +[motivation]: #motivation + +The [`RateLimitPolicy`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimitPolicy) API (v1beta1), particularly its [`RateLimit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#RateLimit) type used in `ratelimitpolicy.spec.rateLimits`, designed in part to fit the underlying implementation based on the Envoy [Rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) filter, has been proven to be (i) _unnecessarily complex_, as well as (ii) possibly _limiting for the extension of the API_ either beyond this specific implementation and/or for supporting use cases of rate limiting not contemplated in the original designs. + +Users of the `RateLimitPolicy` will immediatelly recognize elements of Envoy's Rate limit API in the definitions of the `RateLimit` type, with almost 1:1 correspondence between the [`Configuration`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Configuration) and [`Rule`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Rule) types and their counterparts in the Envoy configuration. Although compatibility between those continue to be desired, the leakage of such implementation details to the level of the API could otherwise be avoided to provide a better abstraction for the context in vogue, where activators ("matchers") and payload ("descriptors") are defined by users in a same document instead of in different configurations of different components. + +Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#Limit) type – used as well in the `RateLimitPolicy`'s `RateLimit` – implies presently a logical relationship between its inner concepts – i.e. conditions and variables on one side, and limits themselves on the other – that otherwise could be shaped in a different manner, to provide (i) clearer understanding of the meaning of these concepts by the user as well as (ii) to avoid repetition. + +### Goals + +1. Decouple the API from the underlying implementation - i.e. provide a more generic abstraction +2. Simplify the API - i.e. DRY, straight to the point (for defining rate limits) +3. Consistency with Kuadrant's [AuthPolicy](https://pkg.go.dev/github.com/kuadrant/kuadrant-operator@v0.2.1/api/v1beta1#AuthPolicy) API - i.e. same language, similar UX + +### Current WIP to consider + +1. Inheritance of the network matching rules from the upstream network object (https://github.com/Kuadrant/architecture/pull/4) +2. Implement `skip_if_absent` for the RequestHeaders action (https://github.com/Kuadrant/wasm-shim/issues/29) +3. Rate Limiting across clusters ([doc](https://docs.google.com/document/d/1pqCODRAkNUTLB6lJfRWcv3d2Hlj9WqsGySmjrP707Vk/edit#heading=h.nzpgr3pef6uy)) + +## Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Given the following HTTPRoute: + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: HTTPRoute +metadata: + name: toystore +spec: + parentRefs: + - name: istio-ingressgateway + namespace: istio-system + hostnames: ["*.toystore.com"] + rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + backendRefs: + - name: toystore + port: 80 +``` + +These are examples of RLPs targeting the route: + +**Example 1.** Minimal example - unconditional, unqualified rate limiting + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - rates: + - limit: 5 + unit: minute +``` + +**Example 2.** All you can get - conditions, counter qualifiers, multiple rates, and variable increments + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: writers + conditions: + - selector: context.request.http.method + operator: eq + value: POST + rates: + - limit: 5 + duration: 1 + unit: minute + - limit: 100 + duration: 12 + unit: hour + counters: + - auth.identity.username + increment: + value: 1 # default + + - name: readers + conditions: + - selector: context.request.http.method + operator: eq + value: GET + rates: + - limit: 50 + duration: 1 + unit: minute + counters: + - auth.identity.username + increment: + value: 1 + + - name: read-expensive + conditions: + - selector: context.request.http.method + operator: eq + value: GET + - selector: context.request.http.path + operator: eq + value: /toys/expensive + rates: + - limit: 10 + duration: 1 + unit: minute + increment: + value: 2 # each hit artificially made twice as expensive without reseting the counters +``` + +### Comparison to current RateLimitPolicy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CurrentNewReason
1:1 relation between Limit (object) and the actual Rate limit (value)Rate becomes a detail of Limit where each limit may define one or more rates (1:N) +
    +
  • It allows to reuse the conditions and counters for groups of rate limits
    • +
+
Parsed conditionsStructured condition fields composed of { selector, operator, value } + +
Configurations as a list of variables assignments ("actions" for building "descriptors")Descriptor actions composed from the selectors (conditions[].selector and counters[]) +
    +
  • Abstract Envoy-specific concepts of "actions" and "descriptors"
    • +
  • Only build descriptors effectively used in the policy - no room for useless invariant descriptors (e.g. "limited = 1")
    • +
  • Source value of the selectors defined from an implicit "context" data structure
    • +
+
Key-value descriptorsStructured descriptors from a "context" data structure + +
maxValuelimit
secondsduration and unit +
    +
  • Support for more units beyond seconds
    • +
  • duration: 1 by default
    • +
+
Fixed incrementincrement: {VALUE} +
    +
  • Flexibility to change the limits without reseting the counters
    • +
  • Placeholder for more complex RL systems in the future – e.g. dynamic-RL, %, 𝑓(), etc
    • +
  • Migration path from other RL solutions, e.g. 3scale (?)
    • +
+
+ +## Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +### Composing RL descriptor actions + +By completely dropping out the `configurations` field from the RLP, the `actions` to composing the RL _descriptor actions_ is now done based essentially on the selectors listed in the `conditions` and `counters`. + +Each selector is a direct reference to a path within a _well-known data structure_ that holds the `context` (L4 and L7 data of the original request handled by the proxy), as well as occasionally the `auth` data (dynamic metadata exported by the external authorization filter and injected by the proxy into the rate-limit filter). + +#### The well-known data structure for user-defined RLP selectors and Kuadrant-generated RL descriptor actions + +The well-known data structure for building RL descriptor actions out of selectors in a RLP resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. + +To keep compatibility with the [Envoy Rate Limit protocol](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. + +```yaml +context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type + source: + address: … + service: … + … + destination: + address: … + service: … + … + request: + http: + host: … + path: … + method: … + headers: {…} + + auth: # Dynamic metadata exported by the external authorization service + + ratelimit: # Envoy's Rate Limit `RateLimitRequest` type + domain: … # generated by the Kuadrant controller + descriptors: {…} # descriptors configured by the user directly in the proxy (not generated by the Kuadrant controller, if allowed) + hitsAddend: … # only in case we want to allow users to refer to this value in a policy +``` + +#### Mechanics of generating RL descriptor actions + +From the perspective of a user who writes a RLP, the selectors used in the `conditions` and `counters` are paths to the well-known data structure. While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, + +> _Whenever the context is the HTTP request to the path (`context.request.http.path`) that is equal to `"/toys"`, I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ + +Resulting in the following RLP: + +```yaml +apiVersion: kuadrant.io/v2beta1 +kind: RateLimitPolicy +metadata: + name: toystore +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute + name: toystore + limits: + - name: toys-path + conditions: + - selector: context.request.http.path + operator: eq + value: toys + rates: + - limit: 50 + duration: 1 + unit: second + counters: + - auth.identity.username +``` + +...and the following selectors: +- `context.request.http.path` +- `auth.identity.username` + +The RLP controller, on the other hand, uses a map to translate each of these selectors into its corresponding descriptor actions. + +```yaml +rate_limits: + - actions: + - request_headers: + descriptor_key: "auth.identity.username" + header_name: ":path" + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" +``` + +## Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +## Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +### Possible variations for the selectors (conditions and counter qualifiers) + +The main drivers behind the proposed design for the selectors (conditions and counter qualifiers), based on (i) structured condition expressions and (ii) `conditions` and `counters` separated in distinct fields (variation "C" below), are: +1. consistency with the Authorino `AuthConfig` API, which also specifies conditions expressed in `selector`, `operator`, and `value` fields; +2. explicit user intent, without subtle distinction of meaning based on presence of optional fields. + +Nonetheless here are a few alternative variations to consider: + + + + + + + + + + + + + + + + + + + + + +
Structured condition expressionsParsed condition expressions
Single field + A + 
+selectors:
+  - selector: context.request.http.method
+    operator: eq
+    value: GET
+  - selector: auth.identity.username
+ 		+ B + 
+selectors:
+  - context.request.http.method == "GET"
+  - auth.identity.username
+
Distinct fields + C ⭐️ + 
+conditions:
+  - selector: context.request.http.method
+    operator: eq
+    value: GET
+counters:
+  - auth.identity.username
+ 		+ D + 
+conditions:
+  - context.request.http.method == "GET"
+counters:
+  - auth.identity.username
+
+ +⭐️ Variation adopted for the examples and (so far) final design proposal. + +## Prior art +[prior-art]: #prior-art + +Discuss prior art, both the good and the bad, in relation to this proposal. +A few examples of what this can include are: + +- Does another project have a similar feature? +- What can be learned from it? What's good? What's less optimal? +- Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background. + +This section is intended to encourage you as an author to think about the lessons from other tentatives - successful or not, provide readers of your RFC with a fuller picture. + +Note that while precedent set by other projects is some motivation, it does not on its own motivate an RFC. + +## Unresolved questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to resolve through the RFC process before this gets merged? +- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +## Future possibilities +[future-possibilities]: #future-possibilities + +Think about what the natural extension and evolution of your proposal would be and how it would affect the platform and project as a whole. Try to use this section as a tool to further consider all possible interactions with the project and its components in your proposal. Also consider how this all fits into the roadmap for the project and of the relevant sub-team. + +This is also a good place to "dump ideas", if they are out of scope for the RFC you are writing but otherwise related. + +Note that having something written down in the future-possibilities section is not a reason to accept the current or a future RFC; such notes should be in the section on motivation or rationale in this or subsequent RFCs. The section merely provides additional information.