feat: allow filtering incoming request headers

[bytemain]

I want to deploy go-httpbin to Alibaba Cloud Function Compute (A Serverless Runtime), But in FC, the Server will add some custom header on the request headers:

for example:

So I add this logic to process unused headers:

	opts := []httpbin.OptionFunc{
		httpbin.WithMaxBodySize(cfg.MaxBodySize),
		httpbin.WithMaxDuration(cfg.MaxDuration),
		httpbin.WithObserver(httpbin.StdLogObserver(logger)),
		httpbin.WithExcludeHeaders("x-ignore-*,x-info-this-key"),
	}

[codecov]

Codecov Report

Merging #139 (1bfbdb8) into main (3761c4a) will decrease coverage by 0.06%.
The diff coverage is 100.00%.

❗ Current head 1bfbdb8 differs from pull request most recent head f891b76. Consider uploading reports for the commit f891b76 to get more accurate results

@@            Coverage Diff             @@
##             main     #139      +/-   ##
==========================================
- Coverage   98.23%   98.17%   -0.06%     
==========================================
  Files           8        8              
  Lines        1527     1534       +7     
==========================================
+ Hits         1500     1506       +6     
- Misses         19       20       +1     
  Partials        8        8              

Files Changed	Coverage Δ
httpbin/httpbin.go	100.00% <ø> (ø)
httpbin/handlers.go	99.45% <100.00%> (ø)
httpbin/helpers.go	96.08% <100.00%> (-0.32%)	⬇️
httpbin/options.go	100.00% <100.00%> (ø)

[mccutchen]

Ah, interesting idea. Given that using this new Interceptor will require wiring up your own go-httpbin server rather than just running a prebuilt binary or docker image, I should point out that the same thing could be accomplished via plain old stlib HTTP handler middleware without requiring any changes to go-httpbin. (I'd be happy to show a specific example, if it would help!)

It seems like the real value would be exposing this as configuration, to make it easy for anyone to deploy a "stock" go-httpbin image and still drop platform-specific headers. I can imagine being able to do something like this (including wildcard prefix matches) being generally useful:

-exclude-headers="Foo,Bar,X-Fc-*"
EXCLUDE_HEADERS="Foo,Bar,X-Fc-*"

Do you have any interest in adapting this work to that end? (If so, I'd consider simplifying the implementation to just deal with headers. The notion of flexible "Interceptors" is nice, but feels like a bit more than we need here.)

[bytemain]

Yes, It seem that expose this feature as a configuration is more reasonable.

But in my opinion, I think the Interceptor mechanism should be kept, it make the whole httpbin more configurable. It is actually a hook system, bring some hooks so that we can configure by ourself.

In the first version PR, I just added a very simple interceptor implementation(I am lazy, XD), I think the Observer is the same as Interceptor, So can I still use the "interceptor"?

[mccutchen]

What other interceptors do you imagine being useful here? Maybe it's a lack of imagination on my part, but I do not think we'd want to support anything other than modifying incoming request headers, and I can't think of any reason to do anything aside from dropping them.

To do that, a fancy system of interceptor hooks is not required. It can be implemented as a fairly simple middleware.

(I'd put the Observer in a different category, simply because golang's stdlib makes it difficult/awkward to instrument outgoing HTTP responses. There, the Observer abstraction makes it possible for go-httpbin to surface details about the generated response in a way that is easy for 3rd parties to build on. No such abstraction is necessary for filtering out incoming request headers.)

[bytemain]

@mccutchen I have simplified the implementation and add configuration ability~

[mccutchen]

Nice, thanks! I've got some feedback below, please let me know if it makes sense to you. In particular, I think we can make the regex matching more efficient.

[bytemain]

Thanks for your advice, I have done for them.

And now the headerProcessor will only be used if user configured exclude-headers configuration.

[mccutchen]

This is looking great, thanks so much for iterating on it with me! We're very close, I just have a couple of final requests below.

Plus, there are a few very fiddly test cases that assert the CLI's help output, which are failing due to the new CLI argument here:
https://github.com/mccutchen/go-httpbin/actions/runs/5664402459/job/15362579323?pr=139

The easiest way to fix those failures is to copy the got: value directly into the test case. (I know this is very annoying, I'm sorry!)

[mccutchen]

👍 LGTM, thanks a lot for the contribution @bytemain!

[mccutchen]

Released in v2.11.0. Thanks again!

[bytemain]

maybe couldn't, the array of values is the resut serialized by the JsonMarshal, So if we want change the behaviour, we need extra config to change the json marshal result.

---

By the way, if we use the intercept mechanism, the developer can change this behavior:

• [compat] original httpbin combines mulitple header values into comma-separated string #15

[mccutchen]

… they could also do that in a simple middleware, by combining the header values into a comma-separated string before they reach go-httpbin.

That middleware would not be much more complex than a custom interceptor.