Add PoV size to benchmark overhead

[ggwpez]

Closes paritytech/polkadot-sdk#384

Adds the Storage proof size to the the overhead weight template.
New rendered extrinsic_weights.rs looks like this:

/// Time to execute a NO-OP extrinsic. For example `System::remark`.
/// Calculated by multiplying the *Average* with `1.0` and adding `0`.
///
...
pub const ExtrinsicBaseRefTime: u64 = WEIGHT_PER_NANOS.ref_time().saturating_mul(97_610);

/// Weight to execute a NO-OP extrinsic. For example `System::remark`.
pub const ExtrinsicBaseWeight: Weight = Weight::from_components(
	ExtrinsicBaseRefTime::get(),
	// There is no proof size consumed by a NO-OP extrinsic.
	0,
);

... and block_weights.rs:

/// Time to execute an empty block.
/// Calculated by multiplying the *Average* with `1.0` and adding `0`.
///
...
pub const BlockExecutionRefTime: u64 = WEIGHT_PER_NANOS.ref_time().saturating_mul(342_097);

/// Storage proof size to prove the execution of an empty block.
///
/// THIS IS NOT THE POV SIZE. The PoV size additionally includes the block size.
///
/// Compaction would result in `3_473` byte.
/// Compaction and compression would result in `3_163` byte.
pub const EmptyBlockProofSize: u64 = 4_857;

/// Storage proof size to prove the execution of a block with at least one extrinsic.
///
/// THIS IS NOT THE POV SIZE. The PoV size additionally includes the block size.
///
/// Compaction would result in `3_879` byte.
/// Compaction and compression would result in `3_365` byte.
pub const NonEmptyBlockProofSize: u64 = 5_422;

/// Weight to execute an empty block.
///
/// We do not distinguish between the weight of empty or non-empty blocks and
/// therefore use the assumed larger one, being non-empty.
pub const BlockExecutionWeight: Weight = Weight::from_components(
	BlockExecutionRefTime::get(),
	NonEmptyBlockProofSize::get(),
);

Further changes:

• Handle overflow in calc_ref_time
• Tests the rendered template output files
• Remove the {{header}} HBS variable as it introduces an empty newline at the top of the rendered file when there is no header. Any header will be prepended directly.
• Use the direct header to execute a block. Formerly it would assume genesis.

Open points

• I added a test that applies rustfmt to a rendered template and checks that it would not modify it. I am fine with not merging it if that is too hacky.

[shawntabrizi]

Looks like a good starting point. Please add an issue to follow up on some things here.

Perhaps a TODO on top of the zstd to perhaps have the API support generating a compressed proof, so we dont assume how compression happens here?

[shawntabrizi]

👉 Question is how do we now get the base proof size of an extrinsic?

I don't think an extrinsic has a base storage proof size. So to calculate the impact to PoV (storage proof + block size) we add:

1. the length of an extrinsic
2. any storage it reads during execution

By default, (2) can be zero, and (1) can be as small as 2 bytes and a signature.

[ggwpez]

Somehow the block weights reduced a lot. This is also on master. It was fixed in Polkadot by this #12232 but not here?!

PS: I dienered Polkadot and the weights there are as expected. Not sure why just Substrate is reduced.

[ggwpez]

Not sure how to handle this or if we want it.
It is pretty useful though since it is easy to mess up the template which then results in downstream fmt failure.

[stale]

Hey, is anyone still working on this? Due to the inactivity this issue has been automatically marked as stale. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Hey, is anyone still working on this? Due to the inactivity this issue has been automatically marked as stale. It will be closed if no further activity occurs. Thank you for your contributions.