Companion for Store voters in unsorted bags to get good stake iteration properties (#3413)

* add voter bags generation script and generated values for all runtimes

* VoterBagThresholds for test-runtime

* semicolon

* turns out the header is called file_header.txt

* perform benchmarks to get real numbers

* add note about when to re-run this script

* Suggestions for #3413 (Store voters in unsorted bags companion) (#3488)

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: parity-processbot <>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: CI system <>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Lldenaurois <ljdenaurois@gmail.com>
Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com>
Co-authored-by: Seun Lanlege <seun@parity.io>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Denis Tsai <denistsai@aetheras.io>
Co-authored-by: Pierre Besson <pierre.besson@parity.io>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: ferrell-code <automatedcharlesferrell@gmail.com>
Co-authored-by: Shaun W <spxwang@gmail.com>

* Try resolve Cargo.lock conflicts

* Add temp VoterSnapshotPerBlock value to compile

* Improve voter_bags test with nominator/validator breakdown

* Remove miner.log

* Update README

* Module doc comments

* Build working

* remote-ext-tests very  WIP

* Remove system module query

* formatting and log target

* clean up

* Save

* Fix issues with polkadot runtime after mergin

* Save

* Some updates to runtimes to make sure things compile

* save

* udpate weights

* Use UseNominatorsAndUpdateBagsList

* make remote-ext tests a bin

* Adjust remote-ext test to use pub; update some comments

* Use list_bags_get

* Try fix cargo.lock

* Gate kusama feat code

* Carg.lock .. again

* point bags-list deps to master

* Update voter-bags  CLI to take total issuance and min balance as args

* Switch polkadot runtime to UseNominatorMap

* Some tweaks to the remote-test

* Delete some stuff for polkadot; comments

* Various fixes

* Add comment for VoterSnapshotPerBlock

* Remove bags-list import from polakdot cargo

* use benchmarking feature for pallet-bags-list

* Add custom migration for westend and kusama for bags-list

* :facepalm

* Fix kusama runtime

* Small clean up

* Be more alphabetical ordered

* Use 22_500 for VoterSnapshotPerBlock

* point back to master

* remove unused type

* SortedListProvider for staking runtime

* Remove polkadot for remote-ext bags test

* Revert "Remove polkadot for remote-ext bags test"

This reverts commit 3608c466002063761cd669880a00b2e05c1ff7d1.

* Remove polkadot for remote-ext bags test

* update Substrate

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Lldenaurois <ljdenaurois@gmail.com>
Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com>
Co-authored-by: Seun Lanlege <seun@parity.io>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Denis Tsai <denistsai@aetheras.io>
Co-authored-by: Pierre Besson <pierre.besson@parity.io>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: ferrell-code <automatedcharlesferrell@gmail.com>
Co-authored-by: Shaun W <spxwang@gmail.com>
Co-authored-by: parity-processbot <>
This commit is contained in:
Peter Goodspeed-Niklaus
2021-09-17 05:24:59 +02:00
committed by GitHub
parent 608ad0f6bf
commit 9a6f43ab8f
25 changed files with 1479 additions and 268 deletions
@@ -16,6 +16,7 @@
//! A list of the different weight modules for our runtime.
pub mod frame_system;
pub mod pallet_bags_list;
pub mod pallet_balances;
pub mod pallet_election_provider_multi_phase;
pub mod pallet_identity;
@@ -0,0 +1,66 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Autogenerated weights for pallet_bags_list
//!
//! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 4.0.0-dev
//! DATE: 2021-09-02, STEPS: `50`, REPEAT: 20, LOW RANGE: `[]`, HIGH RANGE: `[]`
//! EXECUTION: Some(Wasm), WASM-EXECUTION: Compiled, CHAIN: Some("dev"), DB CACHE: 128
// Executed Command:
// target/release/substrate
// benchmark
// --chain=dev
// --steps=50
// --repeat=20
// --pallet=pallet_bags_list
// --extrinsic=*
// --execution=wasm
// --wasm-execution=compiled
// --heap-pages=4096
// --output=./frame/bags-list/src/weights.rs
// --template=./.maintain/frame-weight-template.hbs
// NOTE: this is copy pasta from substrate % some tweaks
#![cfg_attr(rustfmt, rustfmt_skip)]
#![allow(unused_parens)]
#![allow(unused_imports)]
use frame_support::{traits::Get, weights::{Weight, constants::RocksDbWeight}};
use sp_std::marker::PhantomData;
/// Weight functions needed for pallet_bags_list.
/// Weights for pallet_bags_list using the Substrate node and recommended hardware.
pub struct WeightInfo<T>(PhantomData<T>);
impl<T: frame_system::Config> pallet_bags_list::WeightInfo for WeightInfo<T> {
fn rebag_non_terminal() -> Weight {
(75_718_000 as Weight)
.saturating_add(T::DbWeight::get().reads(7 as Weight))
.saturating_add(T::DbWeight::get().writes(5 as Weight))
}
// Storage: Staking Bonded (r:1 w:0)
// Storage: Staking Ledger (r:1 w:0)
// Storage: BagsList ListNodes (r:4 w:4)
// Storage: BagsList ListBags (r:1 w:1)
fn rebag_terminal() -> Weight {
(75_718_000 as Weight)
.saturating_add(T::DbWeight::get().reads(7 as Weight))
.saturating_add(T::DbWeight::get().writes(5 as Weight))
}
}
@@ -16,7 +16,7 @@
//! Autogenerated weights for `pallet_staking`
//!
//! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 3.0.0
//! DATE: 2021-07-02, STEPS: `[50, ]`, REPEAT: 20, LOW RANGE: `[]`, HIGH RANGE: `[]`
//! DATE: 2021-07-07, STEPS: `[50, ]`, REPEAT: 20, LOW RANGE: `[]`, HIGH RANGE: `[]`
//! EXECUTION: Some(Wasm), WASM-EXECUTION: Compiled, CHAIN: Some("westend-dev"), DB CACHE: 128
// Executed Command:
@@ -31,7 +31,7 @@
// --wasm-execution=compiled
// --heap-pages=4096
// --header=./file_header.txt
// --output=./runtime/westend/src/weights/
// --output=runtime/westend/src/weights/
#![allow(unused_parens)]
#![allow(unused_imports)]
@@ -43,24 +43,24 @@ use sp_std::marker::PhantomData;
pub struct WeightInfo<T>(PhantomData<T>);
impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
fn bond() -> Weight {
(71_835_000 as Weight)
(70_644_000 as Weight)
.saturating_add(T::DbWeight::get().reads(5 as Weight))
.saturating_add(T::DbWeight::get().writes(4 as Weight))
}
fn bond_extra() -> Weight {
(54_894_000 as Weight)
.saturating_add(T::DbWeight::get().reads(3 as Weight))
(57_596_000 as Weight)
.saturating_add(T::DbWeight::get().reads(4 as Weight))
.saturating_add(T::DbWeight::get().writes(2 as Weight))
}
fn unbond() -> Weight {
(59_419_000 as Weight)
(57_410_000 as Weight)
.saturating_add(T::DbWeight::get().reads(6 as Weight))
.saturating_add(T::DbWeight::get().writes(3 as Weight))
}
fn withdraw_unbonded_update(s: u32) -> Weight {
(50_384_000 as Weight)
// Standard Error: 0
.saturating_add((26_000 as Weight).saturating_mul(s as Weight))
.saturating_add((33_000 as Weight).saturating_mul(s as Weight))
.saturating_add(T::DbWeight::get().reads(4 as Weight))
.saturating_add(T::DbWeight::get().writes(3 as Weight))
}
@@ -73,9 +73,9 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
.saturating_add(T::DbWeight::get().writes((1 as Weight).saturating_mul(s as Weight)))
}
fn validate() -> Weight {
(33_672_000 as Weight)
.saturating_add(T::DbWeight::get().reads(6 as Weight))
.saturating_add(T::DbWeight::get().writes(2 as Weight))
(69_191_000 as Weight)
.saturating_add(T::DbWeight::get().reads(10 as Weight))
.saturating_add(T::DbWeight::get().writes(6 as Weight))
}
fn kick(k: u32) -> Weight {
(11_371_000 as Weight)
@@ -91,18 +91,18 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
.saturating_add((5_341_000 as Weight).saturating_mul(n as Weight))
.saturating_add(T::DbWeight::get().reads(7 as Weight))
.saturating_add(T::DbWeight::get().reads((1 as Weight).saturating_mul(n as Weight)))
.saturating_add(T::DbWeight::get().writes(2 as Weight))
.saturating_add(T::DbWeight::get().writes(7 as Weight))
}
fn chill() -> Weight {
(17_740_000 as Weight).saturating_add(T::DbWeight::get().reads(3 as Weight))
}
fn set_payee() -> Weight {
(11_624_000 as Weight)
(11_867_000 as Weight)
.saturating_add(T::DbWeight::get().reads(1 as Weight))
.saturating_add(T::DbWeight::get().writes(1 as Weight))
}
fn set_controller() -> Weight {
(25_480_000 as Weight)
(26_102_000 as Weight)
.saturating_add(T::DbWeight::get().reads(3 as Weight))
.saturating_add(T::DbWeight::get().writes(3 as Weight))
}
@@ -121,15 +121,15 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
fn set_invulnerables(v: u32) -> Weight {
(2_352_000 as Weight)
// Standard Error: 0
.saturating_add((27_000 as Weight).saturating_mul(v as Weight))
.saturating_add((5_000 as Weight).saturating_mul(v as Weight))
.saturating_add(T::DbWeight::get().writes(1 as Weight))
}
fn force_unstake(s: u32) -> Weight {
(58_375_000 as Weight)
// Standard Error: 1_000
.saturating_add((2_282_000 as Weight).saturating_mul(s as Weight))
.saturating_add(T::DbWeight::get().reads(6 as Weight))
.saturating_add(T::DbWeight::get().writes(6 as Weight))
.saturating_add((2_220_000 as Weight).saturating_mul(s as Weight))
.saturating_add(T::DbWeight::get().reads(8 as Weight))
.saturating_add(T::DbWeight::get().writes(7 as Weight))
.saturating_add(T::DbWeight::get().writes((1 as Weight).saturating_mul(s as Weight)))
}
fn cancel_deferred_slash(s: u32) -> Weight {
@@ -166,8 +166,8 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
}
fn set_history_depth(e: u32) -> Weight {
(0 as Weight)
// Standard Error: 63_000
.saturating_add((33_032_000 as Weight).saturating_mul(e as Weight))
// Standard Error: 76_000
.saturating_add((33_612_000 as Weight).saturating_mul(e as Weight))
.saturating_add(T::DbWeight::get().reads(2 as Weight))
.saturating_add(T::DbWeight::get().writes(4 as Weight))
.saturating_add(T::DbWeight::get().writes((7 as Weight).saturating_mul(e as Weight)))
@@ -175,40 +175,38 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
fn reap_stash(s: u32) -> Weight {
(69_690_000 as Weight)
// Standard Error: 1_000
.saturating_add((2_271_000 as Weight).saturating_mul(s as Weight))
.saturating_add(T::DbWeight::get().reads(7 as Weight))
.saturating_add(T::DbWeight::get().writes(8 as Weight))
.saturating_add((2_214_000 as Weight).saturating_mul(s as Weight))
.saturating_add(T::DbWeight::get().reads(11 as Weight))
.saturating_add(T::DbWeight::get().writes(12 as Weight))
.saturating_add(T::DbWeight::get().writes((1 as Weight).saturating_mul(s as Weight)))
}
fn new_era(v: u32, n: u32) -> Weight {
(0 as Weight)
// Standard Error: 787_000
.saturating_add((310_720_000 as Weight).saturating_mul(v as Weight))
// Standard Error: 39_000
.saturating_add((48_687_000 as Weight).saturating_mul(n as Weight))
.saturating_add(T::DbWeight::get().reads(10 as Weight))
.saturating_add(T::DbWeight::get().reads((3 as Weight).saturating_mul(v as Weight)))
.saturating_add(T::DbWeight::get().reads((3 as Weight).saturating_mul(n as Weight)))
// Standard Error: 1_065_000
.saturating_add((309_528_000 as Weight).saturating_mul(v as Weight))
// Standard Error: 53_000
.saturating_add((52_832_000 as Weight).saturating_mul(n as Weight))
.saturating_add(T::DbWeight::get().reads(187 as Weight))
.saturating_add(T::DbWeight::get().reads((4 as Weight).saturating_mul(v as Weight)))
.saturating_add(T::DbWeight::get().reads((4 as Weight).saturating_mul(n as Weight)))
.saturating_add(T::DbWeight::get().writes(4 as Weight))
.saturating_add(T::DbWeight::get().writes((3 as Weight).saturating_mul(v as Weight)))
}
fn get_npos_voters(v: u32, n: u32, s: u32) -> Weight {
(0 as Weight)
// Standard Error: 110_000
.saturating_add((25_641_000 as Weight).saturating_mul(v as Weight))
.saturating_add((25_393_000 as Weight).saturating_mul(v as Weight))
// Standard Error: 110_000
.saturating_add((27_889_000 as Weight).saturating_mul(n as Weight))
// Standard Error: 3_749_000
.saturating_add((15_769_000 as Weight).saturating_mul(s as Weight))
.saturating_add(T::DbWeight::get().reads(3 as Weight))
.saturating_add((34_992_000 as Weight).saturating_mul(n as Weight))
.saturating_add(T::DbWeight::get().reads(179 as Weight))
.saturating_add(T::DbWeight::get().reads((3 as Weight).saturating_mul(v as Weight)))
.saturating_add(T::DbWeight::get().reads((3 as Weight).saturating_mul(n as Weight)))
.saturating_add(T::DbWeight::get().reads((4 as Weight).saturating_mul(n as Weight)))
.saturating_add(T::DbWeight::get().reads((1 as Weight).saturating_mul(s as Weight)))
}
fn get_npos_targets(v: u32) -> Weight {
(0 as Weight)
// Standard Error: 36_000
.saturating_add((10_960_000 as Weight).saturating_mul(v as Weight))
// Standard Error: 42_000
.saturating_add((11_340_000 as Weight).saturating_mul(v as Weight))
.saturating_add(T::DbWeight::get().reads(1 as Weight))
.saturating_add(T::DbWeight::get().reads((1 as Weight).saturating_mul(v as Weight)))
}
@@ -216,8 +214,8 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
(5_696_000 as Weight).saturating_add(T::DbWeight::get().writes(5 as Weight))
}
fn chill_other() -> Weight {
(41_216_000 as Weight)
.saturating_add(T::DbWeight::get().reads(7 as Weight))
.saturating_add(T::DbWeight::get().writes(2 as Weight))
(84_796_000 as Weight)
.saturating_add(T::DbWeight::get().reads(11 as Weight))
.saturating_add(T::DbWeight::get().writes(6 as Weight))
}
}