Part of #13700

…e is invalid

Related to
 - #13622
    - #13635
 - #13676

Follow-up to #13815
which tracks event signature failures.

This PR aims to stop us from trying to
`_get_state_ids_after_missing_prev_event` after
we already know that the prev_event will fail
from a previous attempt

To explain an exact scenario around `/messages` -> backfill, we call `/backfill` and first check the signatures of the 100 events. We see bad signature for `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` and `$zuOn2Rd2vsC7SUia3Hp3r6JSkSFKcc5j3QTTqW_0jDw` (both member events). Then we process the 98 events remaining that have valid signatures but one of the events references `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` as a `prev_event`. So we have to do the whole `_get_state_ids_after_missing_prev_event` rigmarole which pulls in those same events which fail again because the signatures are still invalid.

 - `backfill`
    - `outgoing-federation-request` `/backfill`
    - `_check_sigs_and_hash_and_fetch`
       - `_check_sigs_and_hash_and_fetch_one` for each event received over backfill
          - ❗ `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` fails with `Signature on retrieved event was invalid.`: `unable to verify signature for sender domain xxx: 401: Failed to find any key to satisfy: _FetchKeyRequest(...)`
          - ❗ `$zuOn2Rd2vsC7SUia3Hp3r6JSkSFKcc5j3QTTqW_0jDw` fails with `Signature on retrieved event was invalid.`: `unable to verify signature for sender domain xxx: 401: Failed to find any key to satisfy: _FetchKeyRequest(...)`
   - `_process_pulled_events`
      - `_process_pulled_event` for each validated event
         - ❗ Event `$Q0iMdqtz3IJYfZQU2Xk2WjB5NDF8Gg8cFSYYyKQgKJ0` references `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` as a `prev_event` which is missing so we try to get it
            - `_get_state_ids_after_missing_prev_event`
               - `outgoing-federation-request` `/state_ids`
               - ❗ `get_pdu` for `$luA4l7QHhf_jadH3mI-AyFqho0U2Q-IXXUbGSMq6h6M` which fails the signature check again
               - ❗ `get_pdu` for `$zuOn2Rd2vsC7SUia3Hp3r6JSkSFKcc5j3QTTqW_0jDw` which fails the signature check

With this PR, we no longer call the burdensome `_get_state_ids_after_missing_prev_event`
because the signature failure will count as an attempt before we try to run this.

…ature-fails

See #13815 (comment)

See #13815 (comment)

So wes can avoid things like `CancelledError` which is a valid error
but probably should not count as an error

…ature-fails

…nto madlittlemods/13622-13700-stop-getting-state-ids-after-missing-prev-event-if-we-tried-to-get-this-event-recently

Conflicts:
	synapse/storage/databases/main/event_federation.py

…ature-fails

…ature-fails

…nto madlittlemods/13622-13700-stop-getting-state-ids-after-missing-prev-event-if-we-tried-to-get-this-event-recently

Conflicts:
	synapse/storage/databases/main/event_federation.py

…ate-ids-after-missing-prev-event-if-we-tried-to-get-this-event-recently

See #13816 (comment)

See #13816 (comment)

See #13816 (comment)

See #13816 (comment)

See #13816 (comment)

…ate-ids-after-missing-prev-event-if-we-tried-to-get-this-event-recently

Also can pass multiple event_ids,
see #13816 (comment)

…andle FederationPullAttemptBackoffError

See #13816 (comment)

…ults for all given events anyway

See #13816 (comment)

See #13816 (comment)

Co-authored-by: Erik Johnston <erik@matrix.org>