Sub-commands for `benchmark` #11164

ggwpez · 2022-04-04T11:55:58Z

Changes:

👉 Breaking: The pallet benchmarking was moved to the benchmark pallet sub-command.
Move shared code into a shared module
Add a test for benchmark block
Unify docs of sub-commands

Many of the changes are just moved files, the interesting part is in node/cli/src/.

The output of the benchmark command now looks like this:

Sub-commands concerned with benchmarking. The pallet benchmarking moved to the `pallet` sub-command

USAGE:
    substrate benchmark <SUBCOMMAND>

OPTIONS:
    -h, --help       Print help information
    -V, --version    Print version information

SUBCOMMANDS:
    block       Benchmark the execution time of historic blocks
    help        Print this message or the help of the given subcommand(s)
    overhead    Benchmark the execution overhead per-block and per-extrinsic
    pallet      Benchmark the extrinsic weight of FRAME Pallets
    storage     Benchmark the storage speed of a chain snapshot

Polkadot companion: paritytech/polkadot#5247
Cumulus companion: paritytech/cumulus#1156
Closes #11140, #11153

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

bin/node-template/node/src/command.rs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

ggwpez · 2022-04-06T15:46:52Z

bot rebase

paritytech-processbot · 2022-04-06T15:46:55Z

Rebasing

emostov · 2022-04-06T15:48:10Z

bin/node-template/node/src/command.rs

+					},
+				_ => Err("Sub-command is not supported".into()),
+			})
+		},


Why not add the other sub commands to the node template? Seems like they would be helpful if someone is basing a new project off the template

Yea that's a good follow-up, I'll put it in an issue. Just wanted to get it done faster 😄

The node-template currently uses the weights from frame_support, so even when you run the benchmarks yourself, it would require some updating to actually use them in the node-template.

Okay I added it now in the last commit since @shawntabrizi also wanted it.
There is infrastructure missing in the node-template, therefore the copy&paste from the substrate node.

utils/frame/benchmarking-cli/src/pallet/mod.rs

utils/frame/benchmarking-cli/src/shared/record.rs

emostov

LGTM - just had some optional suggestions

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

ggwpez · 2022-04-07T19:32:56Z

bot merge

* Restructure benchmark commands Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add benchmark block test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup imports Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * CI Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Review fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Extend error message Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Apply suggestions from code review Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * Review fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add commands to node-template Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

Needed after substrate `benchmark` sub-command change PR paritytech/substrate#11164

paritytech/substrate#11164 paritytech/cumulus#963 paritytech/cumulus#1069 paritytech/substrate#10878

* chore: bump deps to Polkadot v0.9.19 * fix: LengthToFee https://github.com/paritytech/substrate/pull/10785/files * fix: remove u32_trait paritytech/substrate#10850 * fix: client paritytech/substrate#11164 paritytech/cumulus#963 paritytech/cumulus#1069 paritytech/substrate#10878 * fix: clippy runtime * fix: clippy clone * feat: add para runtime enum for client * ci: bump paritytech/ci-linux image * fix: clippy * refactor: benchmark cmd match * fix: peregrune runtime api

* chore: bump deps to Polkadot v0.9.19 * fix: LengthToFee https://github.com/paritytech/substrate/pull/10785/files * fix: remove u32_trait paritytech/substrate#10850 * fix: client paritytech/substrate#11164 paritytech/cumulus#963 paritytech/cumulus#1069 paritytech/substrate#10878 * fix: clippy runtime * fix: clippy clone * feat: add para runtime enum for client * ci: bump paritytech/ci-linux image * fix: clippy * refactor: benchmark cmd match * fix: peregrune runtime api (cherry picked from commit 6f2016b)

* Restructure benchmark commands Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add benchmark block test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup imports Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * CI Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Review fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Extend error message Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Apply suggestions from code review Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * Review fixes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add commands to node-template Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

ggwpez added 2 commits April 4, 2022 13:54

Restructure benchmark commands

012be2a

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Add benchmark block test

1c9468c

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

github-actions bot added the A0-please_review Pull request needs code review. label Apr 4, 2022

ggwpez added A3-in_progress Pull request is in progress. No review needed at this stage. and removed A0-please_review Pull request needs code review. labels Apr 4, 2022

ggwpez mentioned this pull request Apr 4, 2022

Co #11164: Sub-commands for benchmark paritytech/polkadot#5247

Merged

ggwpez requested a review from shawntabrizi April 4, 2022 12:11

ggwpez added 2 commits April 4, 2022 14:15

Fixup imports

9652f07

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

CI

3dd9429

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

shawntabrizi reviewed Apr 4, 2022

View reviewed changes

bin/node-template/node/src/command.rs Outdated Show resolved Hide resolved

ggwpez added 2 commits April 5, 2022 12:14

Review fixes

4e7c202

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Extend error message

7b14cb5

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

shawntabrizi approved these changes Apr 5, 2022

View reviewed changes

shawntabrizi added E5-breaksapi D2-breaksapi labels Apr 5, 2022

emostov reviewed Apr 6, 2022

View reviewed changes

utils/frame/benchmarking-cli/src/pallet/mod.rs Outdated Show resolved Hide resolved

emostov reviewed Apr 6, 2022

View reviewed changes

utils/frame/benchmarking-cli/src/pallet/mod.rs Outdated Show resolved Hide resolved

emostov reviewed Apr 6, 2022

View reviewed changes

utils/frame/benchmarking-cli/src/shared/record.rs Outdated Show resolved Hide resolved

emostov approved these changes Apr 6, 2022

View reviewed changes

ggwpez and others added 2 commits April 6, 2022 18:42

Apply suggestions from code review

e4e0246

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

Review fixes

6466c4e

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

ggwpez added 2 commits April 6, 2022 18:54

Merge remote-tracking branch 'origin/master' into oty-restructure-bench

0bba6a3

Add commands to node-template

e2201af

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

ggwpez mentioned this pull request Apr 7, 2022

Co #11164: Sub-commands for benchmark paritytech/cumulus#1156

Merged

Merge branch 'master' into oty-restructure-bench

f390166

paritytech-processbot bot merged commit 174735e into master Apr 7, 2022

paritytech-processbot bot deleted the oty-restructure-bench branch April 7, 2022 19:33

georgesdib mentioned this pull request Apr 15, 2022

Fixing the bench-bot after substrate changes paritytech/bench-bot#91

Merged

joao-paulo-parity pushed a commit to paritytech/bench-bot that referenced this pull request Apr 15, 2022

Fixing the bench-bot (#91)

cd3b294

Needed after substrate `benchmark` sub-command change PR paritytech/substrate#11164

github-actions bot mentioned this pull request Apr 20, 2022

Update substrate/polkadot/cumulus from v0.9.18 to v0.9.19 moonbeam-foundation/moonbeam#1428

Closed

gilescope mentioned this pull request Apr 20, 2022

benchmark now takes a command paritytech/cumulus#1191

Merged

wischli added a commit to KILTprotocol/kilt-node that referenced this pull request Apr 21, 2022

fix: client

ea93d7a

paritytech/substrate#11164 paritytech/cumulus#963 paritytech/cumulus#1069 paritytech/substrate#10878

wischli mentioned this pull request Apr 22, 2022

feat: upgrade to Polkadot v0.9.19 KILTprotocol/kilt-node#357

Merged

8 tasks

ggera mentioned this pull request Jun 7, 2022

Fixing the bench-bot (#91) KILTprotocol/bench-bot#3

Merged

5 tasks

bernardoaraujor mentioned this pull request Jun 20, 2022

fix benchmarking subcommand polkadot-developers/substrate-docs#1144

Merged

Dengjianping mentioned this pull request Jul 7, 2022

Bump deps to v0.9.22 Manta-Network/Manta#571

Merged

10 tasks

github-actions bot mentioned this pull request Jul 18, 2022

Update substrate/polkadot/cumulus from v0.9.18 to v0.9.19 imstar15/moonbeam#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sub-commands for `benchmark` #11164

Sub-commands for `benchmark` #11164

ggwpez commented Apr 4, 2022 •

edited

Loading

ggwpez commented Apr 6, 2022

paritytech-processbot bot commented Apr 6, 2022

emostov Apr 6, 2022

ggwpez Apr 6, 2022

ggwpez Apr 6, 2022

emostov left a comment

ggwpez commented Apr 7, 2022

Sub-commands for benchmark #11164

Sub-commands for benchmark #11164

Conversation

ggwpez commented Apr 4, 2022 • edited Loading

ggwpez commented Apr 6, 2022

paritytech-processbot bot commented Apr 6, 2022

emostov Apr 6, 2022

Choose a reason for hiding this comment

ggwpez Apr 6, 2022

Choose a reason for hiding this comment

ggwpez Apr 6, 2022

Choose a reason for hiding this comment

emostov left a comment

Choose a reason for hiding this comment

ggwpez commented Apr 7, 2022

Sub-commands for `benchmark` #11164

Sub-commands for `benchmark` #11164

ggwpez commented Apr 4, 2022 •

edited

Loading