Recurring too large event from matrix.kiwifarms.net

[anoadragon453]

Description

I've been getting the following in my logs for the past few hours repeatedly:

2020-01-25 16:43:56,039 - synapse.http.matrixfederationclient - 164 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net- {GET-O-120740} [matrix.kiwifarms.net] Completed: 200 OK
2020-01-25 16:43:56,113 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$15799662694FdIUz:calamari.space- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,114 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996628079659VsCoS:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,115 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996627879658ekygD:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,116 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996622079636ipUVL:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,117 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996627579656cgFdh:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,117 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,118 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996628179660apjeR:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,121 - synapse.handlers.federation - 2249 - INFO - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net-$157996628479661YFIAM:matrix.kiwifarms.net-$157996626879651nsxTJ:matrix.kiwifarms.net- Skipping auth_event fetch for outlier
2020-01-25 16:43:56,121 - synapse.handlers.federation - 411 - WARNING - PUT-804886-$157996629879666qwvGV:matrix.kiwifarms.net-$157996628879662OVzrF:matrix.kiwifarms.net- [!PvQGNjiCNulFZsMPud:matrix.org $157996628879662OVzrF:matrix.kiwifarms.net] Error attempting to resolve state at missing prev_events
...
Traceback (most recent call last):
  File "/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py", line 372, in on_receive_pdu
    origin, room_id, p, include_event_in_state=True
  File "/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py", line 587, in _get_state_for_room
    destination, room_id, desired_events
  File "/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py", line 640, in _get_events_from_store_or_dest
    destination=destination, room_id=room_id, events=missing_events
  File "/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py", line 1106, in _get_events_and_persist
    destination, event_infos,
  File "/home/synapse/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py", line 749, in send
    raise result.value
twisted.internet.defer.FirstError: FirstError[#6, [Failure instance: Traceback: <class 'synapse.api.errors.EventSizeError'>: 413: event too large
/home/synapse/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py:1418:_inlineCallbacks
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py:1948:_prep_event
/home/synapse/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py:1613:unwindGenerator
/home/synapse/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py:1529:_cancellableInlineCallbacks
--- <exception caught here> ---
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py:1793:prep
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py:1948:_prep_event
/home/synapse/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py:1418:_inlineCallbacks
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/handlers/federation.py:2140:do_auth
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/event_auth.py:49:check
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/event_auth.py:183:_check_size_limits
/home/synapse/synapse/env/lib/python3.7/site-packages/synapse/event_auth.py:170:too_big
]]

It seems to be affecting the performance of my server a bit.

Synapse v1.9.0

[richvdh]

I suspect this and #6774 are related

[anoadragon453]

This is due to a fairly large event getting accepted by one homeserver, but not by my own, suggesting there are code paths somewhere that are allowing large events in when they shouldn't be, and others that are rejecting it properly.