feat: Better progress reporting for stage checkpoints #2982
Conversation
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
gakonst
requested review from
rkrasiuk,
rakita,
joshieDo,
onbjerg and
shekhirin
as code owners
gakonst
changed the title
gakonst
changed the title
Codecov Report
@@ Coverage Diff @@
## main #2982 +/- ##
==========================================
+ Coverage 71.04% 71.17% +0.13%
==========================================
Files 519 519
Lines 67318 68077 +759
==========================================
+ Hits 47824 48457 +633
- Misses 19494 19620 +126
Flags with carried forward coverage won't be shown. Click here to find out more.
|
| } | ||
| } | ||
|
|
||
| // The function proceeds as follows: |
There was a problem hiding this comment.
nit: rewrite as doc comment so it shows up in tooltips etc
| } | ||
| } | ||
|
|
||
| // The function proceeds as follows: |
There was a problem hiding this comment.
@shekhirin pls do in followup
onbjerg
added
C-enhancement
crates/storage/db/src/tables/mod.rs
Outdated
There was a problem hiding this comment.
Should we/can we remove the SyncStageProgress table, since the mid-block checkpointing info is now stored in SyncStage?
There was a problem hiding this comment.
not really, the Merkle stage still uses it for the execute checkpoints. We couldn't merge it into the SyncStage straight away because it would force all the structs depending on StageCheckpoint to be Clone. It's not a breaking DB change to do it later because adding a new enum variant to StageUnitCheckpoint is backward-compatible.
Main benefit is that progress reporting is now normalized for all stages by the cost unit for that stage. This allows better node CLI UX, % completion metrics etc. Next step would be better performance by normalizing the load on db commits.
We introduced two new metrics for sync:
reth_sync_entities_processedandreth_sync_entities_total. By dividing them, we can get a percentage progress for each stage.Headers–processedis the number of downloaded headers,totalis the number of total headers since genesis with respect to the known remote tip.Bodies– reported as a block number, no change.SenderRecovery–processedis the number of entries in theTxSenderstable,totalis the number of entries in theTransactionstable.TotalDifficulty– reported as a block number, no change.(Account|Storage)Hashing–processedis the number of entries in theHashed(Account|Storage)table,totalis the number of entries in thePlain(Account|Storage)Statetable.Index(Account|Storage)History–processedis the number of walked changesets,totalis the number of total changesets in the(Account|Storage)ChangeSettable.Merkle–processedis the number of walked hashed accounts/storages,totalis the number of entries in theHashedAccountandHashedStoragetables.Execution–processedis the amount of gas executed,totalis the total amount of gas from headers.TransactionLookup–processedis the number of entries in theTxHashNumbertable,totalis the number of entries in theTransactionstable.Finish– N/AWe also need to be careful with the situation when we do an initial sync using
Pipeline, then keep up with the network usingBlockchainTree, then do aPipelinesync again, because we don't go through stages during theBlockchainTreephase, hence no updates to checkpoint progresses is done beyond block numbers.Currently, the solution is for stages to be able to detect the validity of checkpoint according to block range with which it was last ran. If it's invalid, we recalculate the whole checkpoint progress or the missing part, comparing the block ranges.