settingsMaxTotalHeaderLength not working as expected

[tirumaraiselvan]

Raised a question here and K. A. Buhr helped in confirming a bug.

Cross-posting:

Warp has a settingsMaxTotalHeaderLength field which by default is 50*1024 : https://hackage.haskell.org/package/warp-3.3.10/docs/src/Network.Wai.Handler.Warp.Settings.html#defaultSettings

I suppose this means 50KB? But, when I try to send a header with ~33KB, server throws bad request:

curl -v -w '%{size_request} %{size_upload}' -H @temp.log localhost:8080/v1/version

Result:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /v1/version HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
> myheader2: <big header snipped>
> 
* HTTP 1.0, assume close after body
< HTTP/1.0 400 Bad Request
< Date: Wed, 22 Jul 2020 13:15:19 GMT
< Server: Warp/3.3.10
< Content-Type: text/plain; charset=utf-8
< 
* Closing connection 0
Bad Request33098 0

(note that the request size is 33098)

Same thing works with 32.5KB header file.

[Vlix]

A "quick" look around makes me think it's the len + bsLen in Network.Wai.Handler.Warp.Request.push?
Isn't the bsLen already all the previous bs prepended to the current one? So wouldn't it result in the length doing "fibonacci"-like jumps, where it adds the new chunk length on top of the new total length?

-- Every new chunk is of length 8 in the following examples
new = 8
{start}              -> (THStatus 0 _ id) 
id S.null      + new -> (THStatus 8 _ (S.append 8))
S.append 8 new + new -> (THStatus 24 _ (S.append 8 . S.append 8))
etc.

// Sorry for the bad example, but I think it conveys the underlying operations

I'm also not completely sure what end actually represents, so maybe the length calculations using end might also be off? (or not?)

[kazu-yamamoto]

Relating to #781.

[tirumaraiselvan]

Thank you @Vlix for #838 . Closing this now!