Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/state: fix eta calculation on pruning #22386

Merged
merged 1 commit into from
Feb 26, 2021

Conversation

holiman
Copy link
Contributor

@holiman holiman commented Feb 26, 2021

root@mon02:/datadrive# docker run -v /datadrive:/datadir -it ethereum/client-go  --datadir=/datadir/geth  snapshot prune-state                                                                  ...  
INFO [02-26|12:40:18.730] Iterating state snapshot                 accounts=0 slots=0 elapsed="118.987µs"
INFO [02-26|12:40:26.730] Iterating state snapshot                 accounts=126708 slots=186080 elapsed=8.000s      eta=2h5m46.954s

INFO [02-26|14:00:38.239] Iterated snapshot                        accounts=119817571 slots=414421589 elapsed=1h20m19.509s
INFO [02-26|14:00:59.508] Writing state bloom to disk              name=/datadir/geth/geth/statebloom.0xef2542a6cd3a64ca7188de8d682a3fad85c9e9e732df20a4d99da177263a7fa4.bf.gz
INFO [02-26|14:03:32.428] State bloom filter committed             name=/datadir/geth/geth/statebloom.0xef2542a6cd3a64ca7188de8d682a3fad85c9e9e732df20a4d99da177263a7fa4.bf.gz
INFO [02-26|14:03:40.428] Pruning state data                       nodes=837506 size=218.19MiB elapsed=8.000s       eta=208h26m17.558s
...
INFO [02-26|14:12:37.865] Pruning state data                       nodes=47986709 size=12.22GiB  elapsed=9m5.437s     eta=2h31m5.014s
...
INFO [02-26|14:39:12.435] Pruning state data                       nodes=126099024 size=32.11GiB  elapsed=35m40.006s   eta=22.854s
INFO [02-26|14:39:15.933] Pruned state data                        nodes=126503269 size=32.21GiB  elapsed=35m43.504s
INFO [02-26|14:39:15.933] Compacting database                      range=0x00-0x10 elapsed="3.017µs"
INFO [02-26|14:39:15.933] Compacting database                      range=0x00-0x10 elapsed="3.017µs"
INFO [02-26|14:41:09.154] Compacting database                      range=0x10-0x20 elapsed=1m53.221s
INFO [02-26|14:43:35.951] Compacting database                      range=0x20-0x30 elapsed=4m20.018s
INFO [02-26|14:45:44.143] Compacting database                      range=0x30-0x40 elapsed=6m28.210s
INFO [02-26|14:47:46.132] Compacting database                      range=0x40-0x50 elapsed=8m30.199s
INFO [02-26|14:50:25.607] Compacting database                      range=0x50-0x60 elapsed=11m9.673s
INFO [02-26|14:52:25.063] Compacting database                      range=0x60-0x70 elapsed=13m9.130s
INFO [02-26|15:20:52.932] Compacting database                      range=0x70-0x80 elapsed=41m36.999s
INFO [02-26|15:24:53.845] Compacting database                      range=0x80-0x90 elapsed=45m37.911s
INFO [02-26|15:27:02.548] Compacting database                      range=0x90-0xa0 elapsed=47m46.615s
INFO [02-26|15:29:11.003] Compacting database                      range=0xa0-0xb0 elapsed=49m55.070s
INFO [02-26|15:31:20.325] Compacting database                      range=0xb0-0xc0 elapsed=52m4.391s
INFO [02-26|15:33:29.416] Compacting database                      range=0xc0-0xd0 elapsed=54m13.483s
INFO [02-26|15:35:44.930] Compacting database                      range=0xd0-0xe0 elapsed=56m28.996s
INFO [02-26|15:38:02.401] Compacting database                      range=0xe0-0xf0 elapsed=58m46.468s
INFO [02-26|15:40:21.008] Compacting database                      range=0xf0-     elapsed=1h1m5.075s
INFO [02-26|15:42:41.153] Database compaction finished             elapsed=1h3m25.220s
INFO [02-26|15:42:41.153] State pruning successful                 pruned=32.21GiB elapsed=3h2m22.427s

^ it's a bit overly pessimistic when estimating the pruning -time, obviously. ETA 208h, finishes pruning in 35m.

I think this PR fixes it.

ps. Interesting note: the compaction range 0x60-0x70 is 10x slower than other ranges, because it includes most of the lowercase ascii range, including things like txLookupIndex, l at 0x69.

@holiman holiman added this to the 1.10.0 milestone Feb 26, 2021
@holiman holiman merged commit 498458b into ethereum:master Feb 26, 2021
Copy link

@Platinumwrist Platinumwrist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core/state/pruner/pruner.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants