Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Reduce dispute coordinator load #5785

Merged
merged 51 commits into from
Aug 16, 2022
Merged

Reduce dispute coordinator load #5785

merged 51 commits into from
Aug 16, 2022

Commits on Jun 16, 2022

  1. Don't import backing statements directly

    into the dispute coordinator. This also gets rid of a redundant
    signature check. Both should have some impact on backing performance.
    In general this PR should make us scale better in the number of parachains.
    
    Reasoning (aka why this is fine):
    
    For the signature check: As mentioned, it is a redundant check. The
    signature has already been checked at this point. This is even made
    obvious by the used types. The smart constructor is not perfect as
    discussed [here](https://github.com/paritytech/polkadot/issues/3455),
    but is still a reasonable security.
    
    For not importing to the dispute-coordinator: This should be good as the
    dispute coordinator does scrape backing votes from chain. This suffices
    in practice as a super majority of validators must have seen a backing
    fork in order for a candidate to get included and only included
    candidates pose a threat to our system. The import from chain is
    preferable over direct import of backing votes for two reasons:
    
    1. The import is batched, greatly improving import performance. All
       backing votes for a candidate are imported with a single import.
       And indeed we were able to see in metrics that importing votes
       from chain is fast.
    2. We do less work in general as not every candidate for which
       statements are gossiped might actually make it on a chain. The
       dispute coordinator as with the current implementation would still
       import and keep those votes around for six sessions.
    
    While redundancy is good for reliability in the event of bugs, this also
    comes at a non negligible cost. The dispute-coordinator right now is the
    subsystem with the highest load, despite the fact that it should not be
    doing much during mormal operation and it is only getting worse
    with more parachains as the load is a direct function of the number of statements.
    
    We'll see on Versi how much of a performance improvement this PR
    eskimor committed Jun 16, 2022
    Configuration menu
    Copy the full SHA
    a5ff4ca View commit details
    Browse the repository at this point in the history
  2. Get rid of dead code.

    eskimor committed Jun 16, 2022
    Configuration menu
    Copy the full SHA
    0155ab9 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2022

  1. Dont send approval vote

    eskimor committed Jun 18, 2022
    Configuration menu
    Copy the full SHA
    550aa6b View commit details
    Browse the repository at this point in the history
  2. Make it pass CI

    eskimor committed Jun 18, 2022
    Configuration menu
    Copy the full SHA
    c2d2b1d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7f43db1 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2022

  1. Configuration menu
    Copy the full SHA
    f97e43b View commit details
    Browse the repository at this point in the history
  2. Explicit signature check.

    eskimor committed Jul 14, 2022
    Configuration menu
    Copy the full SHA
    c49214d View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2022

  1. Configuration menu
    Copy the full SHA
    7ee2e14 View commit details
    Browse the repository at this point in the history
  2. Send out approval votes in dispute-distribution.

    Use BTreeMap for ordered dispute votes.
    eskimor committed Jul 17, 2022
    Configuration menu
    Copy the full SHA
    6a99120 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5ae234e View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2022

  1. Fix approval voting tests.

    eskimor committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    7f950a9 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2022

  1. Don't send out dispute message on import + test

    + Some cleanup.
    eskimor committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    986e01d View commit details
    Browse the repository at this point in the history
  2. Guide changes.

    Note that the introduced complexity is actually redundant.
    eskimor committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    32cabb4 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2022

  1. WIP: guide changes.

    eskimor committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    19ecc78 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. Finish guide changes about dispute-coordinator

    conceputally. Requires more proof read still.
    
    Also removed obsolete implementation details, where the code is better
    suited as the source of truth.
    eskimor committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    19833e6 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2022

  1. Configuration menu
    Copy the full SHA
    32e67a8 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2022

  1. Configuration menu
    Copy the full SHA
    98d4389 View commit details
    Browse the repository at this point in the history
  2. Implement logic for retrieving approval-votes

    into approval-voting and approval-distribution subsystems.
    eskimor committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    e5fb770 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2022

  1. Update roadmap/implementers-guide/src/node/disputes/dispute-coordinat…

    …or.md
    
    Co-authored-by: asynchronous rob <rphmeier@gmail.com>
    eskimor and rphmeier authored Aug 5, 2022
    Configuration menu
    Copy the full SHA
    24275ba View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2022

  1. Review feedback.

    In particular: Add note about disputes of non included candidates.
    eskimor committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    dd0ac1d View commit details
    Browse the repository at this point in the history
  2. Incorporate Review Remarks

    eskimor committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    912daf3 View commit details
    Browse the repository at this point in the history
  3. Merge remote-tracking branch 'origin/rk-fast-dispute-coordinator' int…

    …o rk-fast-dispute-coordinator
    eskimor committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    39e913a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    11ab0e3 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2022

  1. Tidy up import logic a bit.

    Logical vote import is now separated, making the code more readable and
    maintainable.
    
    Also: Accept import if there is at least one invalid signer that has not
    exceeded its spam slots, instead of requiring all of them to not exceed
    their limits. This is more correct and a preparation for vote batching.
    eskimor committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    a890970 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2022

  1. Configuration menu
    Copy the full SHA
    8395724 View commit details
    Browse the repository at this point in the history
  2. Fix tests and bugs.

    eskimor committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    31e0769 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2011f35 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    6624bc6 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e3be4c2 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2022

  1. Configuration menu
    Copy the full SHA
    24537b6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    793666b View commit details
    Browse the repository at this point in the history
  3. Reasoning about boundedness.

    eskimor committed Aug 11, 2022
    Configuration menu
    Copy the full SHA
    943b064 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4287137 View commit details
    Browse the repository at this point in the history
  5. Remove redundant import.

    eskimor committed Aug 11, 2022
    Configuration menu
    Copy the full SHA
    8ae6797 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2022

  1. Review remarks.

    eskimor committed Aug 12, 2022
    Configuration menu
    Copy the full SHA
    c126b76 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b11eadf View commit details
    Browse the repository at this point in the history
  3. More review remarks.

    eskimor committed Aug 12, 2022
    Configuration menu
    Copy the full SHA
    b356acc View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f8cea8a View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    d46b03a View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2022

  1. Configuration menu
    Copy the full SHA
    1cdf49c View commit details
    Browse the repository at this point in the history
  2. More trace log

    eskimor committed Aug 13, 2022
    Configuration menu
    Copy the full SHA
    73669dc View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2022

  1. Break cycle.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    4ac86f4 View commit details
    Browse the repository at this point in the history
  2. Add some tracing.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    e6754bb View commit details
    Browse the repository at this point in the history
  3. Cleanup allowed messages.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    3ec17cc View commit details
    Browse the repository at this point in the history
  4. fmt

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    2c7fa5c View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    2b44d94 View commit details
    Browse the repository at this point in the history
  6. Better error.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    de0e85f View commit details
    Browse the repository at this point in the history
  7. Break cycle in all places.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    23f1ee0 View commit details
    Browse the repository at this point in the history
  8. Clarified comment some more.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    0ed7a1a View commit details
    Browse the repository at this point in the history
  9. Typo.

    eskimor committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    4466d30 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    a002cb6 View commit details
    Browse the repository at this point in the history