Inconsistent application of NGINX directives in internal location blocks

[kate-osborn]

Describe the bug

Some NGINX directives are not applied or enforced when configured in an internal location. This occurs when redirecting or rewriting a request from an external location to an internal location.

When a request is rewritten or redirected from an external location to an internal location, we expect the directives set in the internal location to overwrite any directives set in the external location.

However, when NGINX rewrites or redirects a request, some NGINX directives set in this internal location block are not applied or enforced by NGINX.

So far, we've identified this behavior with the following directives:

client_max_body_size: if the value is greater than the value set in the external location or the default value, it is not enforced. This is due to the phases of NGINX; when NGINX handles the request in the external location, it will check the body size of the request against the client_max_body_size value set in the external location before rewriting it to the internal location. If client_max_body_size is not set, it will use the default value.

otel_trace: value is ignored if set in an internal location.

otel_trace_context: value is ignored if set in an internal location.

keepalive_requests: It is not applied if the value is greater than the value set in the external location.

It’s likely that other NGINX directives have inconsistent behavior when applied in an internal location.

Currently, this issue directly impacts the NGINX Gateway Fabric policies ClientSettingsPolicy and ObservabilityPolicy. However, this issue can potentially affect future policies that support attaching to Route types.

Related to #2079 #2105

Proposed Solution

Accept only one policy per external location
Only accept a policy if the Route it targets is the only Route that matches the hostname, port, and path combination. If other Routes overlap, the policy will be rejected.

This allows us to apply policy configuration to the external location instead of the internal locations. We would limit the policies we accept rather than limiting which Routes we accept.

This is possible because, with the policy restriction, a policy cannot be applied to a Route that shares an external location with another Route.

However, even with this solution, there are some cases where we need to apply configuration to the internal location. For example, the otel_span_name and otel_span_attr directives must be set in the internal locations for them to be set correctly.