allow sending 1xx responses

[dunglas]

Currently, it's not possible to send informational responses such as 103 Early Hints or 102 Processing.

This patch allows calling WriteHeader() multiple times in order to send informational responses before the final one.
It follows the patch for HTTP/1 (golang/go#42597) and HTTP/2 (golang/net#96).

In conformance with RFC 8297, if the status code is 103 the current content of the header map is also sent. Its content is not removed after the call to WriteHeader() because the headers must also be included in the final response.

The Chrome and Fastly teams are starting a large-scale experiment to measure the real-life impact of the 103 status code.
Using Early Hints is proposed as a (partial) alternative to Server Push, which are going to be removed from Chrome: https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYLvmQUBY/m/21anpFhxAQAJ

Being able to send this status code from servers implemented using Go would help to see if implementing it in browsers is worth it.

[codecov]

Codecov Report

Merging #3047 (927fae1) into master (bfad411) will decrease coverage by 0.12%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3047      +/-   ##
==========================================
- Coverage   86.17%   86.05%   -0.12%     
==========================================
  Files         132      133       +1     
  Lines        9352     9441      +89     
==========================================
+ Hits         8059     8124      +65     
- Misses        936      955      +19     
- Partials      357      362       +5     

Impacted Files	Coverage Δ
http3/response_writer.go	79.17% <100.00%> (+1.39%)	⬆️
session.go	76.62% <0.00%> (-1.13%)	⬇️
internal/handshake/crypto_setup.go	66.19% <0.00%> (-0.64%)	⬇️
internal/congestion/cubic_sender.go	86.61% <0.00%> (-0.19%)	⬇️
internal/ackhandler/received_packet_history.go	96.67% <0.00%> (-0.16%)	⬇️
internal/ackhandler/sent_packet_handler.go	77.86% <0.00%> (-0.14%)	⬇️
config.go	100.00% <0.00%> (ø)
internal/congestion/cubic.go	100.00% <0.00%> (ø)
internal/congestion/pacer.go	100.00% <0.00%> (ø)
mtu_discoverer.go	100.00% <0.00%> (ø)
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bfad411...927fae1. Read the comment docs.

[marten-seemann]

Thank you for the PR!
Does this logic apply to all 1xx codes, or just to 103?

[marten-seemann]

Can you please rebase this PR? I fixed the CI configuration, so the tests should pass now.

[dunglas]

@marten-seemann this logic can be theoretically applied to all 1xx status codes.

[marten-seemann]

The headers are written to a buffered stream. Do we need to flush it for 1xx status codes?

[dunglas]

Yes, headers must be sent with the 103 response (it's what allow the client to preload needed resources early), and, by the spec, again in the final response. (rebased btw)

[marten-seemann]

We're writing headers to the bufferedStream. That makes sense for non-103 responses, as we'll flush out the header and the data in one go. For 103, you want the headers to be sent out immediately though (before WriteHeader returns), so you'd have to call Flush(), right?

[marten-seemann]

@dunglas Any update on this?

[dunglas]

@marten-seemann Sorry for the late reply, I was on vacation.

I tried to add a call to Flush(), but it breaks the tests: calling Write() after the Flush() returns an io.ErrShortWrite error and I've no idea why (see the last commit). Do you have a clue of how to fix this?

Also, shouldn't Flush() always been called, even for 200 status codes? This would allow clients to receive resource hints as early as possible.

[marten-seemann]

I tried to add a call to Flush(), but it breaks the tests: calling Write() after the Flush() returns an io.ErrShortWrite error and I've no idea why (see the last commit). Do you have a clue of how to fix this?

That looks like an artifact of the test setup. Please apply the following change:

--- a/http3/response_writer_test.go
+++ b/http3/response_writer_test.go
@@ -24,7 +24,7 @@ var _ = Describe("Response Writer", func() {
        BeforeEach(func() {
                strBuf = &bytes.Buffer{}
                str := mockquic.NewMockStream(mockCtrl)
-               str.EXPECT().Write(gomock.Any()).Do(strBuf.Write).AnyTimes()
+               str.EXPECT().Write(gomock.Any()).DoAndReturn(strBuf.Write).AnyTimes()
                rw = newResponseWriter(str, utils.DefaultLogger)
        })

Also, shouldn't Flush() always been called, even for 200 status codes? This would allow clients to receive resource hints as early as possible.

I don't know. The reason to not do is that if you're just writing a response, you'd want the HEADERS frame to go in the same QUIC packet as the DATA frames. How do resource hints work?

[dunglas]

That looks like an artifact of the test setup. Please apply the following change:

This does the trick, thanks!

How do resource hints work?

Actually the 103 status code is just an optimization for resources and preload hints, which are already supported by all modern browsers. They are special Link HTTP headers allowing to hint the browser to do some network related actions as early as possible (resolving a domain name, doing a TLS handshake, downloading a resource...).
It's probably not necessary to call Flush() automatically for the reasons you described, but maybe should it be documented to call Flush() manually if the user want to leverage these hints.

[marten-seemann]

It's probably not necessary to call Flush() automatically for the reasons you described, but maybe should it be documented to call Flush() manually if the user want to leverage these hints.

Thank you for your explanation, that makes sense. Would you mind sending us another PR to update the documentation?