I got state-reset out of Matrix HQ (a v5 room)

[richvdh]

for the record: this happened just after I set the join_rules to invite to deal with a spammer.

[richvdh]

There was a lot of stuff like this:

2020-01-23 16:45:22,842 - synapse.handlers.federation - 411 - WARNING - PUT-83440-$ua1yC3-vS7EfGlN4YTqte9zgYLJJcI1EizFu5qVrHFU-$0ZW9s-cUionpKLjARQBNk_cT8HQ-c1hOXrFEBS8pSDw- [!OGEhHVWSdvArJzumhm:matrix.org $0ZW9s-cUionpKLjARQBNk_cT8HQ-c1hOXrFEBS8pSDw] Error attempting to resolve state at missing prev_events
....
twisted.internet.defer.FirstError: FirstError[#9, [Failure instance: Traceback: <class 'synapse.api.errors.EventSizeError'>: 413: event too large
...
2020-01-23 16:45:23,062 - synapse.handlers.federation - 549 - WARNING - PUT-83440-$ua1yC3-vS7EfGlN4YTqte9zgYLJJcI1EizFu5qVrHFU-$0ZW9s-cUionpKLjARQBNk_cT8HQ-c1hOXrFEBS8pSDw- [!OGEhHVWSdvArJzumhm:matrix.org $ua1yC3-vS7EfGlN4YTqte9zgYLJJcI1EizFu5qVrHFU] Received prev_event $0ZW9s-cUionpKLjARQBNk_cT8HQ-c1hOXrFEBS8pSDw failed history check.

that stopped at about 16:50, then:

2020-01-23 17:01:18,521 - synapse.storage.persist_events - 384 - INFO - persist_events-2091- Calculating state delta for room !OGEhHVWSdvArJzumhm:matrix.org
2020-01-23 17:01:18,525 - synapse.state - 497 - INFO - persist_events-2091- Resolving state for !OGEhHVWSdvArJzumhm:matrix.org with 5 groups
2020-01-23 17:01:18,525 - synapse.state - 520 - INFO - persist_events-2091- Resolving conflicted state for '!OGEhHVWSdvArJzumhm:matrix.org'
2020-01-23 17:01:18,542 - synapse.handlers.user_directory - 198 - INFO - user_directory.notify_new_event-27631- Server left room: '!OGEhHVWSdvArJzumhm:matrix.org'
...
2020-01-23 17:01:18,687 - synapse.handlers.federation - 235 - INFO - PUT-89163-$zs-pwRImfCVszkhS2whAWWdH1g4su0U8ZK6itYlrMTg- [!OGEhHVWSdvArJzumhm:matrix.org $zs-pwRImfCVszkhS2whAWWdH1g4su0U8ZK6itYlrMTg] Ignoring PDU from matrix.org as we're not in the room

[richvdh]

my forward extremities (which were presumably what it did state-res across) are:

synapse=# select * from event_forward_extremities where room_id='!OGEhHVWSdvArJzumhm:matrix.org';
                   event_id                   |            room_id             
----------------------------------------------+--------------------------------
 $B2ZSvSFmLyYaMlcNgjVzYRqyCTxAhgoBhRZV-Vl8tmU | !OGEhHVWSdvArJzumhm:matrix.org
 $A2TZUE8-tmfQ561vP-xdFWYwWSDKmnbKjW20IqvtXyA | !OGEhHVWSdvArJzumhm:matrix.org
 $JjAZZ5YVloumUZRmreiELfocHFXijEiwWG1Tqd_IpEo | !OGEhHVWSdvArJzumhm:matrix.org
 $HHCLdbXELiJjdc9X6X-qHWr11myqOcyvckge6nqf678 | !OGEhHVWSdvArJzumhm:matrix.org
 $Ml5HJlbcNJnxpYv2vk_cVHlP48pKTWFi-JhWFJYtxSY | !OGEhHVWSdvArJzumhm:matrix.org
(5 rows)

[richvdh]

And indeed a state res across those events has me not in the room:

>>> sh=hs.get_state_handler()
>>> events=['$B2ZSvSFmLyYaMlcNgjVzYRqyCTxAhgoBhRZV-Vl8tmU', '$A2TZUE8-tmfQ561vP-xdFWYwWSDKmnbKjW20IqvtXyA', '$JjAZZ5YVloumUZRmreiELfocHFXijEiwWG1Tqd_IpEo','$HHCLdbXELiJjdc9X6X-qHWr11myqOcyvckge6nqf678', '$Ml5HJlbcNJnxpYv2vk_cVHlP48pKTWFi-JhWFJYtxSY']
>>> sh.resolve_state_groups_for_events('!OGEhHVWSdvArJzumhm:matrix.org', events)
<Deferred at 0x7f6894f6efd0 current result: <synapse.state._StateCacheEntry object at 0x7f687db89408>>
>>> r=_.result
>>> ('m.room.member','@richvdh:sw1v.org') in r.state
False

[richvdh]

>>> d = hs.get_storage().state.get_state_for_events(events)
>>> r=d.result
>>> [ r[events[i]].get(('m.room.member','@richvdh:sw1v.org'),None) for  i in range(0,5)]
[<FrozenEventV3 event_id='$EMZDXTYZf8jFbejWeRmnpCpl8FjuQgyFSR_ogrqeqq4', type='m.room.member', state_key='@richvdh:sw1v.org'>, <FrozenEventV3 event_id='$EMZDXTYZf8jFbejWeRmnpCpl8FjuQgyFSR_ogrqeqq4', type='m.room.member', state_key='@richvdh:sw1v.org'>, None, <FrozenEventV3 event_id='$EMZDXTYZf8jFbejWeRmnpCpl8FjuQgyFSR_ogrqeqq4', type='m.room.member', state_key='@richvdh:sw1v.org'>, <FrozenEventV3 event_id='$EMZDXTYZf8jFbejWeRmnpCpl8FjuQgyFSR_ogrqeqq4', type='m.room.member', state_key='@richvdh:sw1v.org'>]

so apparently I also wasn't in the room at $JjAZZ5YVloumUZRmreiELfocHFXijEiwWG1Tqd_IpEo either.

(a) it's not entirely obvious to me why the state res alg would conclude that this means I'm not in the room at all
(b) perhaps I'll have to chase this up the DAG

[richvdh]

The answer to (a) appears to be that this is an expected consequence of resolving against a state group where a user is not present (ie, $JjAZZ5YVloumUZRmreiELfocHFXijEiwWG1Tqd_IpEo) against a state group where the join_rules are invite.

so (b), chasing it up the DAG: the first (non-outlier) event where I am not a member is $SlXyD5lYuTDpEZDIC18qdiH7WiqSZ_iUKWDf_k57VLw. This has a tiny state with only a couple of members.

The situation is the same on matrix.org however, and matrix.org didn't think I'd left the room.

The difference appears to be that matrix.org never state-resolved one of the events with the tiny state sets against a part of the dag with the join_rules being invite - or rather, if it did, it didn't have any permanent effects. The problem is that sw1v.org did that state-res for the current state (which is a sort of pseudo-state which isn't part of the DAG), realised I wasn't in it, and then never accepted any more events.

I guess it would be good to know why $SlXyD5lYuTDpEZDIC18qdiH7WiqSZ_iUKWDf_k57VLw had such a tiny state; I suppose it is to do with the errors shown above.

[richvdh]

I think this is probably a dup of #7742