Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

remove Default from CandidateDescriptor #4484

Merged

Conversation

drahnr
Copy link
Contributor

@drahnr drahnr commented Dec 7, 2021

Introduces a set of helpers for testing to create Candidate*-types with ease. The purpose of these types twofold, 1) subsystem unit tests which are in need of unified fillers, that create types which can have arbitrary content 2) for benchmarking where the content has influence on the measured runtime.

@emostov please take over and drive this to completion ASAP

TODOs

  • remove remainaing fn dummy() -> Self implementations @emostov
  • introduced a test failure `sr25519_sign_version_1` called outside of an Externalities-provided environment.', needs a fix
  • move relevant fns into the the test helper crate (i.e. CandidateBuilder is a good candidate, there are 4 or 5 copies of that. Also SessionInfo) @drahnr , can be defered to a follow up PR
  • verify the runtime benchmarks covers the input parameter space sufficiently, no dummy_* calls in there @emostov
  • Avoid x: Default::default() usage as much as possible @drahnr
  • format

Ref paritytech/substrate#10403

skip check-dependent-cumulus

@github-actions github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Dec 7, 2021
@drahnr drahnr added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Dec 8, 2021
@ordian ordian added the D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. label Dec 8, 2021

/// Create meaningless dummy hash.
pub fn dummy_hash() -> Hash {
// TODO make this PRNG based
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drahnr are you ok with leaving this out for now and leaving for a follow up?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would make things a little more complicated because we would no longer be able to use this when we are asserting the output

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the TODOs, it's fine for right now, and I think we should introduce a separate fn with _rand suffix

@emostov
Copy link
Contributor

emostov commented Dec 10, 2021

/benchmark runtime kusama runtime_parachains::paras_inherent

@parity-benchapp
Copy link

parity-benchapp bot commented Dec 10, 2021

Benchmark Runtime Kusama Pallet for branch "bernhard-avoid-default-for-bench-and-tests" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

Results
Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_variable_disputes", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParaSessionInfo Sessions (r:1 w:0)
Storage: ParasDisputes Disputes (r:1 w:1)
Storage: ParasDisputes Included (r:1 w:1)
Storage: ParasDisputes SpamSlots (r:1 w:1)
Storage: ParasDisputes Frozen (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    485.6
    + v    49.26
              µs

Reads = 28 + (0 * v)
Writes = 17 + (0 * v)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v   mean µs  sigma µs       %
   10     980.4     7.934    0.8%
   13      1127     8.391    0.7%
   16      1267     6.207    0.4%
   19      1429     9.234    0.6%
   22      1564     5.826    0.3%
   25      1718     11.42    0.6%
   28      1879     16.63    0.8%
   31      2013     9.452    0.4%
   34      2169     12.53    0.5%
   37      2312     11.74    0.5%
   40      2455     14.53    0.5%
   43      2605     7.505    0.2%
   46      2755      17.6    0.6%
   49      2900     14.75    0.5%
   52      3035     12.15    0.4%
   55      3189     18.43    0.5%
   58      3345     20.95    0.6%
   61      3503     22.24    0.6%
   64      3644      7.82    0.2%
   67      3779     11.01    0.2%
   70      3934     5.457    0.1%
   73      4078     12.22    0.2%
   76      4242      20.2    0.4%
   79      4379     11.21    0.2%
   82      4517     15.62    0.3%
   85      4675     9.753    0.2%
   88      4814     24.05    0.4%
   91      4971     14.16    0.2%
   94      5133     20.06    0.3%
   97      5263     12.43    0.2%
  100      5414     9.529    0.1%
  103      5543     9.181    0.1%
  106      5711     21.64    0.3%
  109      5850      9.27    0.1%
  112      5998     21.73    0.3%
  115      6156     25.18    0.4%
  118      6310     19.31    0.3%
  121      6435     21.82    0.3%
  124      6610      31.6    0.4%
  127      6725     14.16    0.2%
  130      6899      23.8    0.3%
  133      7038     35.37    0.5%
  136      7181     13.23    0.1%
  139      7340     21.16    0.2%
  142      7479     26.53    0.3%
  145      7620     20.78    0.2%
  148      7765      18.4    0.2%
  151      7925     29.54    0.3%
  154      8086     24.38    0.3%
  157      8213     29.86    0.3%
  160      8389     25.05    0.2%
  163      8550     17.73    0.2%
  166      8670     21.84    0.2%
  169      8815     20.08    0.2%
  172      8947     22.46    0.2%
  175      9099     16.81    0.1%
  178      9251     17.96    0.1%
  181      9407     28.44    0.3%
  184      9588     24.23    0.2%
  187      9700     18.84    0.1%
  190      9852     20.33    0.2%
  193      9991     12.05    0.1%
  196     10150     34.26    0.3%
  199     10310     30.01    0.2%

Quality and confidence:
param     error
v         0.015

Model:
Time ~=    484.5
    + v     49.3
              µs

Reads = 28 + (0 * v)
Writes = 17 + (0 * v)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_bitfields", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasDisputes Frozen (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParasDisputes Disputes (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParasDisputes Included (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=      459
              µs

Reads = 25
Writes = 16

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=      459
              µs

Reads = 25
Writes = 16

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_candidates_variable", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasDisputes Frozen (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParasDisputes Disputes (r:2 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParasDisputes Included (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=     1132
    + v    49.17
              µs

Reads = 28 + (0 * v)
Writes = 15 + (0 * v)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v   mean µs  sigma µs       %
  101      6085      12.1    0.1%
  102      6154     15.28    0.2%
  103      6199     19.71    0.3%
  104      6244     13.82    0.2%
  105      6289     18.21    0.2%
  106      6348     13.24    0.2%
  107      6408     19.85    0.3%
  108      6445     21.82    0.3%
  109      6480     21.51    0.3%
  110      6539     20.98    0.3%
  111      6578     12.96    0.1%
  112      6635     19.13    0.2%
  113      6682     19.76    0.2%
  114      6745     26.44    0.3%
  115      6799     15.13    0.2%
  116      6858      24.2    0.3%
  117      6900     21.41    0.3%
  118      6940     16.26    0.2%
  119      6984     14.36    0.2%
  120      7047     13.57    0.1%
  121      7081     18.68    0.2%
  122      7149     25.45    0.3%
  123      7186     26.32    0.3%
  124      7244     20.71    0.2%
  125      7286     19.83    0.2%
  126      7361     26.12    0.3%
  127      7391     24.98    0.3%
  128      7428     11.04    0.1%
  129      7481     24.15    0.3%
  130      7517     26.44    0.3%
  131      7578      26.7    0.3%
  132      7640     32.29    0.4%
  133      7673     18.17    0.2%
  134      7709     11.18    0.1%
  135      7765     13.96    0.1%
  136      7800     14.57    0.1%
  137      7887     31.75    0.4%
  138      7926     26.74    0.3%
  139      7978     31.74    0.3%
  140      8034     30.92    0.3%
  141      8066     20.39    0.2%
  142      8119     21.46    0.2%
  143      8162     21.92    0.2%
  144      8247     25.88    0.3%
  145      8273     32.55    0.3%
  146      8317     26.03    0.3%
  147      8367     24.67    0.2%
  148      8424     17.82    0.2%
  149      8479     31.14    0.3%
  150      8514     22.02    0.2%
  151      8570     25.73    0.3%
  152      8609     9.271    0.1%
  153      8651     20.14    0.2%
  154      8706     23.32    0.2%
  155      8749     19.76    0.2%
  156      8802     11.29    0.1%
  157      8847     12.78    0.1%
  158      8909     33.74    0.3%
  159      8957     12.02    0.1%
  160      8987     13.21    0.1%
  161      9060     16.09    0.1%
  162      9084     14.95    0.1%
  163      9161      24.6    0.2%
  164      9222     18.04    0.1%
  165      9240     13.49    0.1%
  166      9295     15.86    0.1%
  167      9382     46.69    0.4%
  168      9379     24.34    0.2%
  169      9437      17.3    0.1%
  170      9498     17.14    0.1%
  171      9558     19.68    0.2%
  172      9581     34.55    0.3%
  173      9645     29.94    0.3%
  174      9697     39.88    0.4%
  175      9737     14.58    0.1%
  176      9794     18.87    0.1%
  177      9856     31.47    0.3%
  178      9893     17.83    0.1%
  179      9938     21.09    0.2%
  180      9972     12.28    0.1%
  181     10070      36.3    0.3%
  182     10090     28.78    0.2%
  183     10120     20.82    0.2%
  184     10180     26.57    0.2%
  185     10220     26.81    0.2%
  186     10270      26.8    0.2%
  187     10310     12.85    0.1%
  188     10360     23.18    0.2%
  189     10420     37.36    0.3%
  190     10460     14.07    0.1%
  191     10520     20.33    0.1%
  192     10570     33.54    0.3%
  193     10620      22.6    0.2%
  194     10690     21.07    0.1%
  195     10720     23.79    0.2%
  196     10760     25.48    0.2%
  197     10880     37.03    0.3%
  198     10870     28.49    0.2%
  199     10900     23.32    0.2%
  200     11000     17.16    0.1%

Quality and confidence:
param     error
v         0.029

Model:
Time ~=     1134
    + v    49.19
              µs

Reads = 28 + (0 * v)
Writes = 15 + (0 * v)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_candidate_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasDisputes Frozen (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParasDisputes Disputes (r:2 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras FutureCodeHash (r:1 w:0)
Storage: Paras UpgradeRestrictionSignal (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParasDisputes Included (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    43200
              µs

Reads = 30
Writes = 15

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    43200
              µs

Reads = 30
Writes = 15


…bernhard-avoid-default-for-bench-and-tests
…k --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
// Standard Error: 26_000
.saturating_add((49_051_000 as Weight).saturating_mul(v as Weight))
.saturating_add(T::DbWeight::get().reads(29 as Weight))
(1_134_397_000 as Weight)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3x increase to the static weight here is quite significant

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand where that comes from

@emostov
Copy link
Contributor

emostov commented Dec 10, 2021

/benchmark runtime polkadot runtime_parachains::paras_inherent

@parity-benchapp
Copy link

parity-benchapp bot commented Dec 10, 2021

Benchmark Runtime Polkadot Pallet for branch "bernhard-avoid-default-for-bench-and-tests" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs

Results
Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_variable_disputes", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    403.1
    + v    0.307
              µs

Reads = 23 + (0 * v)
Writes = 14 + (0 * v)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v   mean µs  sigma µs       %
   10     408.9     5.351    1.3%
   13     407.6     4.288    1.0%
   16     405.6     3.891    0.9%
   19     412.8     4.131    1.0%
   22     409.3     3.331    0.8%
   25     417.2     9.542    2.2%
   28     412.4     4.972    1.2%
   31       415     5.417    1.3%
   34     410.8     3.635    0.8%
   37     415.3      4.92    1.1%
   40     416.5     3.739    0.8%
   43     414.6     2.319    0.5%
   46     416.5     4.258    1.0%
   49     427.3     7.502    1.7%
   52     418.3     4.286    1.0%
   55     419.2     6.073    1.4%
   58       422     4.235    1.0%
   61     422.8     6.057    1.4%
   64     424.9     6.036    1.4%
   67     423.4     5.373    1.2%
   70     427.2     2.071    0.4%
   73     428.6     6.319    1.4%
   76     429.8     5.318    1.2%
   79     426.5     2.827    0.6%
   82       429     7.257    1.6%
   85     427.1     2.664    0.6%
   88     427.4     5.196    1.2%
   91     432.6     3.043    0.7%
   94     436.5     7.981    1.8%
   97     428.5     2.483    0.5%
  100     433.5     2.902    0.6%
  103     433.8     1.911    0.4%
  106     433.4     3.469    0.8%
  109     439.7     7.832    1.7%
  112     438.5     5.578    1.2%
  115       439     4.397    1.0%
  118     436.9      2.55    0.5%
  121     439.9     4.069    0.9%
  124     437.6     4.522    1.0%
  127     446.7     6.522    1.4%
  130     444.2     6.218    1.3%
  133     446.7      5.31    1.1%
  136     442.9     2.004    0.4%
  139     449.9     6.017    1.3%
  142     450.8     6.218    1.3%
  145     451.3     4.496    0.9%
  148     450.2     5.375    1.1%
  151     453.3     4.355    0.9%
  154     451.4     3.954    0.8%
  157     454.7     5.321    1.1%
  160     458.6     7.439    1.6%
  163     459.2     4.478    0.9%
  166       458     4.841    1.0%
  169     459.3     5.852    1.2%
  172     458.9     7.624    1.6%
  175     454.7     3.365    0.7%
  178     456.8      3.45    0.7%
  181     458.2     3.846    0.8%
  184     455.6     3.489    0.7%
  187     468.7     10.16    2.1%
  190     462.6     6.859    1.4%
  193     462.7     6.239    1.3%
  196     459.7     4.016    0.8%
  199     461.9     4.292    0.9%

Quality and confidence:
param     error
v         0.004

Model:
Time ~=      404
    + v    0.309
              µs

Reads = 23 + (0 * v)
Writes = 14 + (0 * v)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_bitfields", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    446.6
              µs

Reads = 23
Writes = 15

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    446.6
              µs

Reads = 23
Writes = 15

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_candidates_variable", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=     1102
    + v     49.1
              µs

Reads = 25 + (0 * v)
Writes = 14 + (0 * v)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v   mean µs  sigma µs       %
  101      6071     9.779    0.1%
  102      6115     13.68    0.2%
  103      6144     5.733    0.0%
  104      6212      19.5    0.3%
  105      6261     15.83    0.2%
  106      6315     25.91    0.4%
  107      6362     14.67    0.2%
  108      6414     16.47    0.2%
  109      6455     13.07    0.2%
  110      6490     12.77    0.1%
  111      6566     16.98    0.2%
  112      6610     21.18    0.3%
  113      6654     17.69    0.2%
  114      6709      32.2    0.4%
  115      6776     41.38    0.6%
  116      6788     12.98    0.1%
  117      6840     15.52    0.2%
  118      6916     18.67    0.2%
  119      6962     32.94    0.4%
  120      7022     35.08    0.4%
  121      7041     20.89    0.2%
  122      7093     10.65    0.1%
  123      7146     13.09    0.1%
  124      7178     18.25    0.2%
  125      7261     27.91    0.3%
  126      7288     17.89    0.2%
  127      7340     26.95    0.3%
  128      7375     21.87    0.2%
  129      7426     21.37    0.2%
  130      7476     8.951    0.1%
  131      7552     18.23    0.2%
  132      7586     10.36    0.1%
  133      7647     13.15    0.1%
  134      7676     16.09    0.2%
  135      7746     34.38    0.4%
  136      7779     10.99    0.1%
  137      7840     29.11    0.3%
  138      7884      12.6    0.1%
  139      7919     23.25    0.2%
  140      7977     26.52    0.3%
  141      8033     27.69    0.3%
  142      8087     15.43    0.1%
  143      8119     23.09    0.2%
  144      8190     30.18    0.3%
  145      8222     26.37    0.3%
  146      8258     13.35    0.1%
  147      8327     16.26    0.1%
  148      8369     19.95    0.2%
  149      8427     13.53    0.1%
  150      8470     21.77    0.2%
  151      8528     21.84    0.2%
  152      8591     20.98    0.2%
  153      8612      12.9    0.1%
  154      8662     19.15    0.2%
  155      8722     19.57    0.2%
  156      8772     27.69    0.3%
  157      8796     20.67    0.2%
  158      8864     25.86    0.2%
  159      8918     32.53    0.3%
  160      8981     24.88    0.2%
  161      9015        27    0.2%
  162      9052      21.9    0.2%
  163      9103     21.32    0.2%
  164      9156     29.38    0.3%
  165      9267     22.04    0.2%
  166      9253     22.98    0.2%
  167      9310     27.91    0.2%
  168      9343     16.89    0.1%
  169      9391      23.6    0.2%
  170      9471     44.68    0.4%
  171      9508     23.15    0.2%
  172      9547     21.56    0.2%
  173      9611     34.79    0.3%
  174      9652     23.19    0.2%
  175      9710     39.79    0.4%
  176      9762     21.92    0.2%
  177      9785     28.88    0.2%
  178      9848      27.5    0.2%
  179      9889     23.12    0.2%
  180      9942     11.64    0.1%
  181      9980     14.85    0.1%
  182     10030     10.93    0.1%
  183     10070     16.26    0.1%
  184     10130     19.28    0.1%
  185     10190     13.38    0.1%
  186     10250     21.37    0.2%
  187     10290     20.53    0.1%
  188     10340     35.53    0.3%
  189     10370      22.1    0.2%
  190     10410     12.57    0.1%
  191     10460     22.08    0.2%
  192     10590     32.65    0.3%
  193     10590     20.16    0.1%
  194     10640     30.17    0.2%
  195     10680     27.12    0.2%
  196     10710      16.6    0.1%
  197     10790     26.79    0.2%
  198     10830     24.82    0.2%
  199     10880     33.47    0.3%
  200     10910     21.13    0.1%

Quality and confidence:
param     error
v         0.029

Model:
Time ~=     1103
    + v    49.12
              µs

Reads = 25 + (0 * v)
Writes = 14 + (0 * v)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_candidate_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:2 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras FutureCodeHash (r:1 w:0)
Storage: Paras UpgradeRestrictionSignal (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    42700
              µs

Reads = 27
Writes = 14

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    42700
              µs

Reads = 27
Writes = 14


…k --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
@@ -47,7 +47,7 @@ fn dummy_validation_code() -> ValidationCode {
/// "features = runtime-benchmarks".
fn account<AccountId: Decode + Default>(name: &'static str, index: u32, seed: u32) -> AccountId {
let entropy = (name, index, seed).using_encoded(sp_io::hashing::blake2_256);
AccountId::decode(&mut &entropy[..]).unwrap_or_default()
AccountId::decode(&mut &entropy[..]).expect("256 bit input is valid. qed.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine for now, but in general it's best to use TrailZeroInput to ensure that AccountId::decode doesn't run out of input data. In the future, we should just have a trait function for fn from_entropy(e: impl Encode) -> Self.

(201_142_000 as Weight)
// Standard Error: 2_000
.saturating_add((304_000 as Weight).saturating_mul(v as Weight))
(404_051_000 as Weight)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ouch - 2x increase here :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear to what caused this, or if other changes on master caused the change, to be checked.

Copy link
Member

@eskimor eskimor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks like a great improvement!

node/core/dispute-coordinator/src/real/ordering/tests.rs Outdated Show resolved Hide resolved
@drahnr
Copy link
Contributor Author

drahnr commented Dec 10, 2021

bot merge

@paritytech-processbot
Copy link

Waiting for commit status.

@drahnr
Copy link
Contributor Author

drahnr commented Dec 10, 2021

bot merge

@paritytech-processbot
Copy link

Waiting for commit status.

@paritytech-processbot paritytech-processbot bot merged commit 8c4a7da into master Dec 10, 2021
@paritytech-processbot paritytech-processbot bot deleted the bernhard-avoid-default-for-bench-and-tests branch December 10, 2021 12:12
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants