Non-ascii characters like ™, Ö, ... in room name break admin list room api with UnicodeEncodeError: 'ascii' codec can't encode character

[csett86]

Description

If a room name contains non-ascii characters like "™", Ä, Ö, Ü, ... the admin list room api (GET /_synapse/admin/v1/rooms) fails with 500 internal server error when listing the rooms.

Steps to reproduce

• Have non-ascii character like "™" in one room name
• Query GET /_synapse/admin/v1/rooms?limit=5000 (high limit to ensure that the room in question is in the list)
• Receive 500 internal server error and the following trace in the homeserver.log:

2020-08-30 14:53:57,431 - synapse.http.server - 84 - ERROR - GET-307980 - Failed handle request via 'ListRoomRestServlet': <XForwardedForRequest at 0x7f6c18d8ec50 method='GET' uri='/_synapse/admin/v1/rooms' clientproto='HTTP/1.1' site=32901>
Traceback (most recent call last):
  File "/home/pacs/REDACTED/users/synod/synapse/env37/lib/python3.7/site-packages/twisted/internet/defer.py", line 1436, in _inlineCallbacks
    result = current_context.run(g.send, result)
StopIteration: ([{'room_id': '!REDACTED:matrix.org', 'name': '', 'canonical_alias': None, 'joined_members': 3, 'joined_local_members': 1, 'version': '5', 'creator': '', 'encryption': None, 'federatable': True, 'public': False, 'join_rules': 'invite', 'guest_access': 'can_join', 'history_visibility': 'shared', 'state_events': 10}, ...], 1876)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pacs/REDACTED/users/synod/synapse/env37/lib/python3.7/site-packages/twisted/internet/defer.py", line 1436, in _inlineCallbacks
    result = current_context.run(g.send, result)
StopIteration: ([{'room_id': '!REDACTED:matrix.org', 'name': '', 'canonical_alias': None, 'joined_members': 3, 'joined_local_members': 1, 'version': '5', 'creator': '', 'encryption': None, 'federatable': True, 'public': False, 'join_rules': 'invite', 'guest_access': 'can_join', 'history_visibility': 'shared', 'state_events': 10}, ...], 1876)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pacs/REDACTED/users/synod/synapse/env37/lib/python3.7/site-packages/synapse/http/server.py", line 233, in _async_render_wrapper
    self._send_response(request, code, response)
  File "/home/pacs/REDACTED/users/synod/synapse/env37/lib/python3.7/site-packages/synapse/http/server.py", line 295, in _send_response
    canonical_json=self.canonical_json,
  File "/home/pacs/REDACTED/users/synod/synapse/env37/lib/python3.7/site-packages/synapse/http/server.py", line 536, in respond_with_json
    json_bytes = encode_pretty_printed_json(json_object) + b"\n"
  File "/home/pacs/REDACTED/users/synod/synapse/env37/lib/python3.7/site-packages/canonicaljson.py", line 96, in encode_pretty_printed_json
    return _pretty_encoder.encode(json_object).encode("ascii")
UnicodeEncodeError: 'ascii' codec can't encode character '\u2122' in position 14266: ordinal not in range(128)

Version information

• Homeserver: synod.im

If not matrix.org:

• Version: 1.19.1

• Install method: pip

• Platform: Debian 10

Also on:

• Homeserver: settgast.org

If not matrix.org:

• Version: 1.19.1

• Install method: package manager

• Platform: Ubuntu 18.04

[csett86]

Duplicate of #8188, same workaround (use eg. the Synapse user-agent with curl) worked.