Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[dnm] testing benching all pallets #53

Closed
wants to merge 10 commits into from
Closed

[dnm] testing benching all pallets #53

wants to merge 10 commits into from

Conversation

mordamax
Copy link

@mordamax mordamax commented Oct 9, 2024

No description provided.

mordamax and others added 10 commits October 8, 2024 15:20
@mordamax
[DNM] Update README.md
0e1717e
@actions-user
Update from mordamax running command 'prdoc --audience node_dev --bum…
bcf5064 
…p patch'
@mordamax
flip continue-on-fail to fail-fast, save output to file + artifacts
75f511e
@mordamax
Update README.md
a4d5618
@mordamax
Delete pr_5977.prdoc
23c6552
@mordamax
add timeout to cmd
6c5cb68
@pepoviola @bkchr @mordamax
bump zombienet version and set request for k8s (paritytech#5968)
a61d700 
Bump zombienet version, including fixes for `ci` and set _resources
requests_ for the runner.

Thx!

Co-authored-by: Bastian Köcher <git@kchr.de>
@bkchr @ggwpez
@shawntabrizi @seadanda @mordamax
Bump some dependencies (paritytech#5886)
aecfebf 
This bumps `ethbloom`, `ethereum-types`, `primitive-types` and `rlp` to
their latest version.

Fixes: paritytech#5870

---------

Co-authored-by: command-bot <>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
@pepoviola @mordamax
Disable flaky tests reported in 5972/5973/5974 (paritytech#5976)
8eda10a 
Disable flaky tests reported in:
paritytech#5972 
paritytech#5973 
paritytech#5974
@mordamax
Update README.md
d63cbdf
@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd bench

@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd bench --runtime dev

@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd prdoc

@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd prdoc --audience bla

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 9, 2024

Command "prdoc" has failed ❌! See logs here

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 9, 2024

Command "prdoc --audience bla" has failed ❌! See logs here

@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd bench --runtime dev --pallet pallet_blabla

@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd bench --help

@mordamax
Copy link
Author

mordamax commented Oct 9, 2024

/cmd --help

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 9, 2024

Command help: 
usage: /cmd bench [-h] [--quiet] [--clean] [--image IMAGE]
                  [--runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,contracts-rococo,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]]
                  [--pallet [PALLET ...]] [--fail-fast]

options:
  -h, --help            show this help message and exit
  --quiet               Won't print start/end/failed messages in PR
  --clean               Clean up the previous bot's & author's comments in PR
  --image IMAGE         Override docker image '--image
                        docker.io/paritytech/ci-unified:latest'
  --runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,contracts-rococo,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]
                        Runtime(s) space separated
  --pallet [PALLET ...]
                        Pallet(s) space separated
  --fail-fast           Fail fast on first failed benchmark

**Examples**:
 Runs all benchmarks 
 /cmd bench

 Runs benchmarks for pallet_balances and pallet_multisig for all runtimes which have these pallets. **--quiet** makes it to output nothing to PR but reactions
 /cmd bench --pallet pallet_balances pallet_xcm_benchmarks::generic --quiet
 
 Runs bench for all pallets for westend runtime and fails fast on first failed benchmark
 /cmd bench --runtime westend --fail-fast
 
 Does not output anything and cleans up the previous bot's & author command triggering comments in PR 
 /cmd bench --runtime westend rococo --pallet pallet_balances pallet_multisig --quiet --clean

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 9, 2024

Command help: 
usage: /cmd  [--help] [--quiet] [--clean] [--image IMAGE]
             {bench,fmt,update-ui,prdoc} ...

A command runner for polkadot-sdk repo

positional arguments:
  {bench,fmt,update-ui,prdoc}
                        a command to run
    bench               Runs benchmarks
    fmt                 Formats code (cargo +nightly-VERSION fmt) and configs
                        (taplo format)
    update-ui           Updates UI tests
    prdoc               Generates PR documentation

options:
  --help                help for help if you need some help
  --quiet               Won't print start/end/failed messages in PR
  --clean               Clean up the previous bot's & author's comments in PR
  --image IMAGE         Override docker image '--image
                        docker.io/paritytech/ci-unified:latest'

### Command 'bench'
usage: /cmd bench [-h] [--quiet] [--clean] [--image IMAGE]
                  [--runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,contracts-rococo,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]]
                  [--pallet [PALLET ...]] [--fail-fast]

options:
  -h, --help            show this help message and exit
  --quiet               Won't print start/end/failed messages in PR
  --clean               Clean up the previous bot's & author's comments in PR
  --image IMAGE         Override docker image '--image
                        docker.io/paritytech/ci-unified:latest'
  --runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,contracts-rococo,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]
                        Runtime(s) space separated
  --pallet [PALLET ...]
                        Pallet(s) space separated
  --fail-fast           Fail fast on first failed benchmark

**Examples**:
 Runs all benchmarks 
 /cmd bench

 Runs benchmarks for pallet_balances and pallet_multisig for all runtimes which have these pallets. **--quiet** makes it to output nothing to PR but reactions
 /cmd bench --pallet pallet_balances pallet_xcm_benchmarks::generic --quiet
 
 Runs bench for all pallets for westend runtime and fails fast on first failed benchmark
 /cmd bench --runtime westend --fail-fast
 
 Does not output anything and cleans up the previous bot's & author command triggering comments in PR 
 /cmd bench --runtime westend rococo --pallet pallet_balances pallet_multisig --quiet --clean


### Command 'fmt'
usage: /cmd fmt [-h] [--quiet] [--clean] [--image IMAGE]

options:
  -h, --help     show this help message and exit
  --quiet        Won't print start/end/failed messages in PR
  --clean        Clean up the previous bot's & author's comments in PR
  --image IMAGE  Override docker image '--image docker.io/paritytech/ci-
                 unified:latest'


### Command 'update-ui'
usage: /cmd update-ui [-h] [--quiet] [--clean] [--image IMAGE]

options:
  -h, --help     show this help message and exit
  --quiet        Won't print start/end/failed messages in PR
  --clean        Clean up the previous bot's & author's comments in PR
  --image IMAGE  Override docker image '--image docker.io/paritytech/ci-
                 unified:latest'


### Command 'prdoc'
usage: /cmd prdoc [-h] [--pr PR]
                  [--audience [{runtime_dev,runtime_user,node_dev,node_operator} ...]]
                  [--bump {patch,minor,major,silent,ignore,no_change}]
                  [--force]

options:
  -h, --help            show this help message and exit
  --pr PR               The PR number to generate the PrDoc for.
  --audience [{runtime_dev,runtime_user,node_dev,node_operator} ...]
                        The audience of whom the changes may concern. Example:
                        --audience runtime_dev node_dev
  --bump {patch,minor,major,silent,ignore,no_change}
                        A default bump level for all crates. Example: --bump
                        patch
  --force               Whether to overwrite any existing PrDoc.

@github-actions GitHub Actions
Copy link

Command "bench --runtime dev" has started 🚀 See logs here

@github-actions GitHub Actions
Copy link

Command "bench --runtime dev --pallet pallet_blabla" has started 🚀 See logs here

@github-actions GitHub Actions
Copy link

Command "bench --runtime dev --pallet pallet_blabla" has failed ❌! See logs here

@github-actions GitHub Actions
Copy link

Command "bench --runtime dev" has failed ❌! See logs here

Command output:

❌ Failed benchmarks of runtimes/pallets:

@mordamax mordamax closed this Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mordamax @actions-user @pepoviola @bkchr