Re-run weight baseline #13336
Re-run weight baseline #13336
Conversation
frame/contracts/src/weights.rs
Outdated
| // Standard Error: 8_433 | ||
| .saturating_add(Weight::from_ref_time(12_773_098).saturating_mul(n.into())) | ||
| // Minimum execution time: 341_790 nanoseconds. | ||
| Weight::from_ref_time(648_404_995) |
There was a problem hiding this comment.
wow, that's a significant increase
|
/cmd queue -c bench $ pallet dev contracts |
|
Looks like 13 cases went up by over 20% while the other 104 stayed consistent. Maybe we have to increase the |
|
@Robbepop Could it be still the problem with cranelift producing unaligned memory operations? I think we fixed it in wasmi by not zeroing a huge stack on every instantiate? But this was only a 10% spread.
The thing is: It was working before. We can experiment with those. But this shouldn't be done on a live codebase. My work is completely stalled now. So please roll back those changes at least for substrate. Then you can find a working config on a branch. Once everything works we can switch over. But right now I have pressing deadlines and need to merge stuff. And I can't do that while flying blind with benchmarks. |
Yea, maybe we can swap the labels back to the old machines @oleg-plakida? Then fix it in #13316 and merge after it stays consistent. |
|
@athei you can run benchmarks on the old runner with command Does this unblock you? |
|
/cmd queue -c bench $ pallet dev pallet_contracts |
I cannot remember to have noticed that the unaligned memory issue has been fixed in Wasmtime. Not so much has changed in |
I know it isn't fixed in cranelift. But I think we shrank the initial stack size in wasmi massively to it doesn't have a big impact anymore (or at least make it configurable). |
|
/bot bench $ pallet dev pallet_contracts Third run is a bit more consistent over-all with one outlier to |
|
bot bench $ pallet dev pallet_contracts |
|
bot clean |
|
@ggwpez Command |
| // Standard Error: 68 | ||
| .saturating_add(Weight::from_ref_time(57_263).saturating_mul(c.into())) | ||
| // Minimum execution time: 36_506 nanoseconds. | ||
| Weight::from_ref_time(40_743_562) |
| // Standard Error: 33 | ||
| .saturating_add(Weight::from_ref_time(35_317).saturating_mul(c.into())) | ||
| // Minimum execution time: 471_172 nanoseconds. | ||
| Weight::from_ref_time(499_386_220) |
| // Standard Error: 378 | ||
| .saturating_add(Weight::from_ref_time(124_393).saturating_mul(c.into())) | ||
| // Minimum execution time: 4_501_838 nanoseconds. | ||
| Weight::from_ref_time(899_786_250) |
| // Standard Error: 12 | ||
| .saturating_add(Weight::from_ref_time(2_401).saturating_mul(s.into())) | ||
| // Minimum execution time: 2_359_219 nanoseconds. | ||
| Weight::from_ref_time(338_145_767) |
Re-run the benchmarks to check if they stay consistent.
https://weights.tasty.limo/compare?repo=substrate&threshold=10&path_pattern=frame%2F**%2Fsrc%2Fweights.rs&method=guess-worst&ignore_errors=true&unit=time&old=as-weights-gcp&new=at/weights