HTTP API - Content-Type should include charset

[richardschneider]

When returning text/... and application/json the API should include the charset in the HTTP header.

Content-Type: application/json; charset=utf-8

See http://www.w3.org/International/O-HTTP-charset for more derails.

[richardschneider]

@whyrusleeping I found this issue in the HTTP API but it may also exist in gateway.

[jbenet]

@richardschneider was this resolved?

[richardschneider]

No it was not resolved

GET http://localhost:5001/api/v0/version HTTP/1.1
User-Agent: Fiddler
Host: localhost:5001

responds with

HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked

1c
{
   "Version": "0.3.8-dev"
 }
0

[rht]

For gateway, this is already fixed in @AtnNn's #2008.
Though not yet for api.

[ingokeck]

That is actually a big issue. As the header is not set, the http/1.1 standard defines it to be the default ISO-8859-1, while JSON is utf8. So standard compliant clients will treat the response as ISO-8859-1 and may convert it to utf8 to decode the JSON content and then fail. Which now seems to happen to the python api?

[RichardLitt]

There are also cases where the content-type should be different: x-gzip, x-tar, etc. ipfs-inactive/http-api-spec#46 #2427 #2376

[richardschneider]

Any progress? go-ipfs/0.4.7 still doesn't include the charset

HTTP/1.1 200 OK
Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Content-Type: application/json
Server: go-ipfs/0.4.7
Trailer: X-Stream-Error
Vary: Origin
Date: Sun, 12 Nov 2017 23:30:41 GMT
Transfer-Encoding: chunked

[richardschneider]

Same error with go-ipfs/0.4.11

HTTP/1.1 200 OK
Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Content-Type: application/json
Server: go-ipfs/0.4.11
Trailer: X-Stream-Error
Vary: Origin
Date: Mon, 13 Nov 2017 00:26:33 GMT
Content-Length: 270

{"Arguments":{"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD":"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD"},"Objects":{"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD":{"Hash":"Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD","Size":11,"Type":"File","Links":null}}}

[da2x]

This is a more generic issue than just having to do with content encoding.

E.g. text/csv; header means a comma-separated value file where the first line contains column headers.