Add Proof Size to Weight Output (#11637)

* initial impl

* add template test

* linear fit proof size

* always record proof when tracking storage

* calculate worst case pov

* remove duplicate worst case

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* more comment output

* add cli for worst case map size

* update name

* clap does not support underscores

* rename

* expose worst case map values

* improve some comments

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* update template

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix fmt

* more fmt

* more fmt

* Dont panic when there is no proof

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Whitelist :extrinsic_index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use whitelist when recording proof

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add logs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add PoV testing pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Deploy PoV testing pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Storage benches reside in the PoV pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Linear regress PoV per component

Splits the PoV calculation into "measured" and "estimated".
The measured part is reported by the Proof recorder and linear
regressed over all components at once.
The estimated part is calculated as worst-case by using the max
PoV size per storage access and calculating one linear regress per
component. This gives each component a (possibly) independent PoV.
For now the measured size will always be lower than the PoV on
Polkadot since it is measured on an empty snapshot. The measured
part is therefor only used as diagnostic for debugging.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Put PoV into the weight templates

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Extra alanysis choise for PoV

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add+Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make benches faster

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use same template comments

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_balances

* ".git/.scripts/bench-bot.sh" pallet dev pallet_democracy

* Update referenda mock BlockWeights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Take measured value size into account

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_scheduler

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* proof_size: None

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ugly, but works

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* wup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add pov_mode attribute to the benchmarks! macro

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use pov_mode attribute in PoV benchmarking

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Scheduler, Whitelist: Add pov_mode attr

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update PoV weights

* Add CLI arg: default-pov-mode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Update PoV weights"

This reverts commit 2f3ac2387396470b118122a6ff8fa4ee12216f4b.

* Revert "WIP"

This reverts commit c34b538cd2bc45da4544e887180184e30957904a.

* Revert first approach

This reverts commit range 8ddaa2fffe5930f225a30bee314d0b7c94c344dd^..4c84f8748e5395852a9e0e25b0404953fee1a59e

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add extra benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_alliance

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_whitelist

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_scheduler

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy 🤦

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add reference benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix doc comments

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Undo logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add 'Ignored' pov_mode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Allow multiple attributes per benchmark

Turns out that the current benchmarking syntax does not support
multiple attributes per bench 🤦. Changing it to support that
since otherwise the `pov_mode` would conflict with the others.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Validate pov_mode syntax

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Ignore PoV for all contract benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Bump macro recursion limit

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update contract weights

They dont have a PoV component anymore.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix test ffs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* pov_mode is unsupported in V2 syntax

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix pallet ui tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* update pallet ui

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix pallet ui tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Your Name <you@example.com>
This commit is contained in:
Shawn Tabrizi
2023-01-26 19:35:39 -03:00
committed by GitHub
parent 41f819eb48
commit 1b27ae9549
74 changed files with 18668 additions and 9111 deletions
+111 -62
View File
@@ -1,6 +1,6 @@
// This file is part of Substrate.
// Copyright (C) 2022 Parity Technologies (UK) Ltd.
// Copyright (C) 2023 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
@@ -18,7 +18,8 @@
//! Autogenerated weights for frame_system
//!
//! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 4.0.0-dev
//! DATE: 2022-11-07, STEPS: `50`, REPEAT: 20, LOW RANGE: `[]`, HIGH RANGE: `[]`
//! DATE: 2023-01-24, STEPS: `50`, REPEAT: `20`, LOW RANGE: `[]`, HIGH RANGE: `[]`
//! WORST CASE MAP SIZE: `1000000`
//! HOSTNAME: `bm2`, CPU: `Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz`
//! EXECUTION: Some(Wasm), WASM-EXECUTION: Compiled, CHAIN: Some("dev"), DB CACHE: 1024
@@ -60,52 +61,76 @@ pub struct SubstrateWeight<T>(PhantomData<T>);
impl<T: crate::Config> WeightInfo for SubstrateWeight<T> {
/// The range of component `b` is `[0, 3932160]`.
fn remark(b: u32, ) -> Weight {
// Minimum execution time: 3_951 nanoseconds.
Weight::from_ref_time(1_307_232 as u64)
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 2_018 nanoseconds.
Weight::from_ref_time(2_091_000)
// Standard Error: 0
.saturating_add(Weight::from_ref_time(363 as u64).saturating_mul(b as u64))
.saturating_add(Weight::from_ref_time(362).saturating_mul(b.into()))
}
/// The range of component `b` is `[0, 3932160]`.
fn remark_with_event(b: u32, ) -> Weight {
// Minimum execution time: 14_880 nanoseconds.
Weight::from_ref_time(15_173_000 as u64)
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 7_449 nanoseconds.
Weight::from_ref_time(7_748_000)
// Standard Error: 0
.saturating_add(Weight::from_ref_time(1_424 as u64).saturating_mul(b as u64))
.saturating_add(Weight::from_ref_time(1_423).saturating_mul(b.into()))
}
// Storage: System Digest (r:1 w:1)
// Storage: unknown [0x3a686561707061676573] (r:0 w:1)
/// Storage: System Digest (r:1 w:1)
/// Proof Skipped: System Digest (max_values: Some(1), max_size: None, mode: Measured)
/// Storage: unknown `0x3a686561707061676573` (r:0 w:1)
/// Proof Skipped: unknown `0x3a686561707061676573` (r:0 w:1)
fn set_heap_pages() -> Weight {
// Minimum execution time: 9_819 nanoseconds.
Weight::from_ref_time(10_513_000 as u64)
.saturating_add(T::DbWeight::get().reads(1 as u64))
.saturating_add(T::DbWeight::get().writes(2 as u64))
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `495`
// Minimum execution time: 4_440 nanoseconds.
Weight::from_parts(4_605_000, 495)
.saturating_add(T::DbWeight::get().reads(1_u64))
.saturating_add(T::DbWeight::get().writes(2_u64))
}
// Storage: Skipped Metadata (r:0 w:0)
/// Storage: Skipped Metadata (r:0 w:0)
/// Proof Skipped: Skipped Metadata (max_values: None, max_size: None, mode: Measured)
/// The range of component `i` is `[0, 1000]`.
fn set_storage(i: u32, ) -> Weight {
// Minimum execution time: 4_038 nanoseconds.
Weight::from_ref_time(4_098_000 as u64)
// Standard Error: 710
.saturating_add(Weight::from_ref_time(620_813 as u64).saturating_mul(i as u64))
.saturating_add(T::DbWeight::get().writes((1 as u64).saturating_mul(i as u64)))
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 1_981 nanoseconds.
Weight::from_ref_time(2_114_000)
// Standard Error: 804
.saturating_add(Weight::from_ref_time(631_438).saturating_mul(i.into()))
.saturating_add(T::DbWeight::get().writes((1_u64).saturating_mul(i.into())))
}
// Storage: Skipped Metadata (r:0 w:0)
/// Storage: Skipped Metadata (r:0 w:0)
/// Proof Skipped: Skipped Metadata (max_values: None, max_size: None, mode: Measured)
/// The range of component `i` is `[0, 1000]`.
fn kill_storage(i: u32, ) -> Weight {
// Minimum execution time: 3_972 nanoseconds.
Weight::from_ref_time(4_082_000 as u64)
// Standard Error: 884
.saturating_add(Weight::from_ref_time(536_923 as u64).saturating_mul(i as u64))
.saturating_add(T::DbWeight::get().writes((1 as u64).saturating_mul(i as u64)))
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 2_061 nanoseconds.
Weight::from_ref_time(2_153_000)
// Standard Error: 952
.saturating_add(Weight::from_ref_time(502_629).saturating_mul(i.into()))
.saturating_add(T::DbWeight::get().writes((1_u64).saturating_mul(i.into())))
}
// Storage: Skipped Metadata (r:0 w:0)
/// Storage: Skipped Metadata (r:0 w:0)
/// Proof Skipped: Skipped Metadata (max_values: None, max_size: None, mode: Measured)
/// The range of component `p` is `[0, 1000]`.
fn kill_prefix(p: u32, ) -> Weight {
// Minimum execution time: 5_703 nanoseconds.
Weight::from_ref_time(5_763_000 as u64)
// Standard Error: 1_248
.saturating_add(Weight::from_ref_time(1_126_062 as u64).saturating_mul(p as u64))
.saturating_add(T::DbWeight::get().writes((1 as u64).saturating_mul(p as u64)))
// Proof Size summary in bytes:
// Measured: `116 + p * (69 ±0)`
// Estimated: `121 + p * (70 ±0)`
// Minimum execution time: 4_026 nanoseconds.
Weight::from_parts(4_174_000, 121)
// Standard Error: 1_148
.saturating_add(Weight::from_ref_time(1_093_099).saturating_mul(p.into()))
.saturating_add(T::DbWeight::get().writes((1_u64).saturating_mul(p.into())))
.saturating_add(Weight::from_proof_size(70).saturating_mul(p.into()))
}
}
@@ -113,51 +138,75 @@ impl<T: crate::Config> WeightInfo for SubstrateWeight<T> {
impl WeightInfo for () {
/// The range of component `b` is `[0, 3932160]`.
fn remark(b: u32, ) -> Weight {
// Minimum execution time: 3_951 nanoseconds.
Weight::from_ref_time(1_307_232 as u64)
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 2_018 nanoseconds.
Weight::from_ref_time(2_091_000)
// Standard Error: 0
.saturating_add(Weight::from_ref_time(363 as u64).saturating_mul(b as u64))
.saturating_add(Weight::from_ref_time(362).saturating_mul(b.into()))
}
/// The range of component `b` is `[0, 3932160]`.
fn remark_with_event(b: u32, ) -> Weight {
// Minimum execution time: 14_880 nanoseconds.
Weight::from_ref_time(15_173_000 as u64)
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 7_449 nanoseconds.
Weight::from_ref_time(7_748_000)
// Standard Error: 0
.saturating_add(Weight::from_ref_time(1_424 as u64).saturating_mul(b as u64))
.saturating_add(Weight::from_ref_time(1_423).saturating_mul(b.into()))
}
// Storage: System Digest (r:1 w:1)
// Storage: unknown [0x3a686561707061676573] (r:0 w:1)
/// Storage: System Digest (r:1 w:1)
/// Proof Skipped: System Digest (max_values: Some(1), max_size: None, mode: Measured)
/// Storage: unknown `0x3a686561707061676573` (r:0 w:1)
/// Proof Skipped: unknown `0x3a686561707061676573` (r:0 w:1)
fn set_heap_pages() -> Weight {
// Minimum execution time: 9_819 nanoseconds.
Weight::from_ref_time(10_513_000 as u64)
.saturating_add(RocksDbWeight::get().reads(1 as u64))
.saturating_add(RocksDbWeight::get().writes(2 as u64))
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `495`
// Minimum execution time: 4_440 nanoseconds.
Weight::from_parts(4_605_000, 495)
.saturating_add(RocksDbWeight::get().reads(1_u64))
.saturating_add(RocksDbWeight::get().writes(2_u64))
}
// Storage: Skipped Metadata (r:0 w:0)
/// Storage: Skipped Metadata (r:0 w:0)
/// Proof Skipped: Skipped Metadata (max_values: None, max_size: None, mode: Measured)
/// The range of component `i` is `[0, 1000]`.
fn set_storage(i: u32, ) -> Weight {
// Minimum execution time: 4_038 nanoseconds.
Weight::from_ref_time(4_098_000 as u64)
// Standard Error: 710
.saturating_add(Weight::from_ref_time(620_813 as u64).saturating_mul(i as u64))
.saturating_add(RocksDbWeight::get().writes((1 as u64).saturating_mul(i as u64)))
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 1_981 nanoseconds.
Weight::from_ref_time(2_114_000)
// Standard Error: 804
.saturating_add(Weight::from_ref_time(631_438).saturating_mul(i.into()))
.saturating_add(RocksDbWeight::get().writes((1_u64).saturating_mul(i.into())))
}
// Storage: Skipped Metadata (r:0 w:0)
/// Storage: Skipped Metadata (r:0 w:0)
/// Proof Skipped: Skipped Metadata (max_values: None, max_size: None, mode: Measured)
/// The range of component `i` is `[0, 1000]`.
fn kill_storage(i: u32, ) -> Weight {
// Minimum execution time: 3_972 nanoseconds.
Weight::from_ref_time(4_082_000 as u64)
// Standard Error: 884
.saturating_add(Weight::from_ref_time(536_923 as u64).saturating_mul(i as u64))
.saturating_add(RocksDbWeight::get().writes((1 as u64).saturating_mul(i as u64)))
// Proof Size summary in bytes:
// Measured: `0`
// Estimated: `0`
// Minimum execution time: 2_061 nanoseconds.
Weight::from_ref_time(2_153_000)
// Standard Error: 952
.saturating_add(Weight::from_ref_time(502_629).saturating_mul(i.into()))
.saturating_add(RocksDbWeight::get().writes((1_u64).saturating_mul(i.into())))
}
// Storage: Skipped Metadata (r:0 w:0)
/// Storage: Skipped Metadata (r:0 w:0)
/// Proof Skipped: Skipped Metadata (max_values: None, max_size: None, mode: Measured)
/// The range of component `p` is `[0, 1000]`.
fn kill_prefix(p: u32, ) -> Weight {
// Minimum execution time: 5_703 nanoseconds.
Weight::from_ref_time(5_763_000 as u64)
// Standard Error: 1_248
.saturating_add(Weight::from_ref_time(1_126_062 as u64).saturating_mul(p as u64))
.saturating_add(RocksDbWeight::get().writes((1 as u64).saturating_mul(p as u64)))
// Proof Size summary in bytes:
// Measured: `116 + p * (69 ±0)`
// Estimated: `121 + p * (70 ±0)`
// Minimum execution time: 4_026 nanoseconds.
Weight::from_parts(4_174_000, 121)
// Standard Error: 1_148
.saturating_add(Weight::from_ref_time(1_093_099).saturating_mul(p.into()))
.saturating_add(RocksDbWeight::get().writes((1_u64).saturating_mul(p.into())))
.saturating_add(Weight::from_proof_size(70).saturating_mul(p.into()))
}
}