[NPoS] Implements dynamic number of nominators (#12970)

* Implements dynamic nominations per nominator

* Adds SnapshotBounds and ElectionSizeTracker

* Changes the ElectionDataProvider interface to receive ElectionBounds as input

* Implements get_npos_voters with ElectionBounds

* Implements get_npos_targets with ElectionBounds

* Adds comments

* tests

* Truncates nomninations that exceed nominations quota; Old tests passing

* Uses DataProviderBounds and ElectionBounds (to continue)

* Finishes conversions - tests passing

* Refactor staking in babe mocks

* Replaces MaxElectableTargets and MaxElectingVoters with ElectionBounds; Adds more tests

* Fixes nits; node compiling

* bechmarks

* removes nomination_quota extrinsic to request the nomination quota

* Lazy quota check, ie. at nominate time only

* remove non-working test (for now)

* tests lazy nominations quota when quota is lower than current number of nominated targets

* Adds runtime API and custom RPC call for clients to query the nominations quota for a given balance

* removes old rpc

* Cosmetic touches

* All mocks working

* Fixes benchmarking mocks

* nits

* more tests

* renames trait methods

* nit

* ".git/.scripts/commands/fmt/fmt.sh"

* Fix V2 PoV benchmarking (#13485)

* Bump default 'additional_trie_layers' to two

The default here only works for extremely small runtimes, which have
no more than 16 storage prefices. This is changed to a "sane" default
of 2, which is save for runtimes with up to 4096 storage prefices (eg StorageValue).

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update tests and test weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix PoV weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_balances

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_message_queue

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton

* Fix sanity check

>0 would also do as a check, but let's try this.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

* Move BEEFY code to consensus (#13484)

* Move beefy primitives to consensus dir
* Move beefy gadget to client consensus folder
* Rename beefy crates

* chore: move genesis block builder to chain-spec crate. (#13427)

* chore: move genesis block builder to block builder crate.

* add missing file

* chore: move genesis block builder to sc-chain-spec

* Update client/chain-spec/src/genesis.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update test-utils/runtime/client/src/lib.rs

* fix warnings

* fix warnings

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

* Speed up storage iteration from within the runtime (#13479)

* Speed up storage iteration from within the runtime

* Move the cached iterator into an `Option`

* Use `RefCell` in no_std

* Simplify the code slightly

* Use `Option::replace`

* Update doc comment for `next_storage_key_slow`

* Make unbounded channels size warning exact (part 1) (#13490)

* Replace `futures-channel` with `async-channel` in `out_events`

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Also print the backtrace of `send()` call

* Switch from `backtrace` crate to `std::backtrace`

* Remove outdated `backtrace` dependency

* Remove `backtrace` from `Cargo.lock`

---------

Co-authored-by: Koute <koute@users.noreply.github.com>

* Removal of Prometheus alerting rules deployment in cloud-infra (#13499)

* sp-consensus: remove unused error variants (#13495)

* Expose `ChargedAmount` (#13488)

* Expose `ChargedAmount`

* Fix imports

* sc-consensus-beefy: fix metrics: use correct names (#13494)


Signed-off-by: acatangiu <adrian@parity.io>

* clippy fix

* removes NominationsQuotaExceeded event

* Update frame/staking/src/lib.rs

Co-authored-by: Ross Bulat <ross@parity.io>

* adds back the npos_max_iter

* remove duplicate imports added after merge

* fmt

* Adds comment in public struct; Refactors CountBound and SizeCount to struct

* addresses various pr comments

* PR comment reviews

* Fixes on-chain election bounds and related code

* EPM checks the size of the voter list returned by the data provider

* cosmetic changes

* updates e2e tests mock

* Adds more tests for size tracker and refactors code

* Adds back only_iterates_max_2_times_max_allowed_len test

* Refactor

* removes unecessary dependency

* empty commit -- restart all stuck CI jobs

* restarts ci jobs

* Renames ElectionBounds -> Bounds in benchmarking mocks et al

* updates mocks

* Update frame/election-provider-support/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/staking/src/pallet/impls.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/election-provider-support/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/staking/src/tests.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* more checks in api_nominations_quota in tests

* Improves docs

* fixes e2e tests

* Uses size_hint rather than mem::size_of in size tracker; Refactor size tracker to own module

* nits from reviews

* Refactors bounds to own module; improves docs

* More tests and docs

* fixes docs

* Fixes benchmarks

* Fixes rust docs

* fixes bags-list remote-ext-tests

* Simplify bound checks in create_snapshot_external

* Adds target size check in get_npos_targets

* ".git/.scripts/commands/fmt/fmt.sh"

* restart ci

* rust doc fixes and cosmetic nits

* rollback upgrade on parity-scale-codec version (unecessary)

* reset cargo lock, no need to update it

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: acatangiu <adrian@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: yjh <yjh465402634@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Anthony Lazam <lazam@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Piotr Mikołajczyk <piomiko41@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
This commit is contained in:
Gonçalo Pestana
2023-08-10 09:45:55 +02:00
committed by GitHub
parent 314109d87b
commit 93754780b1
30 changed files with 1415 additions and 307 deletions
+16 -14
View File
@@ -22,7 +22,7 @@ use crate::{ConfigOp, Pallet as Staking};
use testing_utils::*;
use codec::Decode;
use frame_election_provider_support::SortedListProvider;
use frame_election_provider_support::{bounds::DataProviderBounds, SortedListProvider};
use frame_support::{
dispatch::UnfilteredDispatchable,
pallet_prelude::*,
@@ -338,7 +338,7 @@ benchmarks! {
validate {
let (stash, controller) = create_stash_controller::<T>(
T::MaxNominations::get() - 1,
MaxNominationsOf::<T>::get() - 1,
100,
Default::default(),
)?;
@@ -362,11 +362,11 @@ benchmarks! {
// these are the other validators; there are `T::MaxNominations::get() - 1` of them, so
// there are a total of `T::MaxNominations::get()` validators in the system.
let rest_of_validators = create_validators_with_seed::<T>(T::MaxNominations::get() - 1, 100, 415)?;
let rest_of_validators = create_validators_with_seed::<T>(MaxNominationsOf::<T>::get() - 1, 100, 415)?;
// this is the validator that will be kicking.
let (stash, controller) = create_stash_controller::<T>(
T::MaxNominations::get() - 1,
MaxNominationsOf::<T>::get() - 1,
100,
Default::default(),
)?;
@@ -381,7 +381,7 @@ benchmarks! {
for i in 0 .. k {
// create a nominator stash.
let (n_stash, n_controller) = create_stash_controller::<T>(
T::MaxNominations::get() + i,
MaxNominationsOf::<T>::get() + i,
100,
Default::default(),
)?;
@@ -418,7 +418,7 @@ benchmarks! {
// Worst case scenario, T::MaxNominations::get()
nominate {
let n in 1 .. T::MaxNominations::get();
let n in 1 .. MaxNominationsOf::<T>::get();
// clean up any existing state.
clear_validators_and_nominators::<T>();
@@ -429,7 +429,7 @@ benchmarks! {
// we are just doing an insert into the origin position.
let scenario = ListScenario::<T>::new(origin_weight, true)?;
let (stash, controller) = create_stash_controller_with_balance::<T>(
SEED + T::MaxNominations::get() + 1, // make sure the account does not conflict with others
SEED + MaxNominationsOf::<T>::get() + 1, // make sure the account does not conflict with others
origin_weight,
Default::default(),
).unwrap();
@@ -711,7 +711,7 @@ benchmarks! {
create_validators_with_nominators_for_era::<T>(
v,
n,
<T as Config>::MaxNominations::get() as usize,
MaxNominationsOf::<T>::get() as usize,
false,
None,
)?;
@@ -729,7 +729,7 @@ benchmarks! {
create_validators_with_nominators_for_era::<T>(
v,
n,
<T as Config>::MaxNominations::get() as usize,
MaxNominationsOf::<T>::get() as usize,
false,
None,
)?;
@@ -808,7 +808,7 @@ benchmarks! {
let n in (MaxNominators::<T>::get() / 2) .. MaxNominators::<T>::get();
let validators = create_validators_with_nominators_for_era::<T>(
v, n, T::MaxNominations::get() as usize, false, None
v, n, MaxNominationsOf::<T>::get() as usize, false, None
)?
.into_iter()
.map(|v| T::Lookup::lookup(v).unwrap())
@@ -819,7 +819,8 @@ benchmarks! {
let num_voters = (v + n) as usize;
}: {
let voters = <Staking<T>>::get_npos_voters(None);
// default bounds are unbounded.
let voters = <Staking<T>>::get_npos_voters(DataProviderBounds::default());
assert_eq!(voters.len(), num_voters);
}
@@ -830,10 +831,11 @@ benchmarks! {
let n = MaxNominators::<T>::get();
let _ = create_validators_with_nominators_for_era::<T>(
v, n, T::MaxNominations::get() as usize, false, None
v, n, MaxNominationsOf::<T>::get() as usize, false, None
)?;
}: {
let targets = <Staking<T>>::get_npos_targets(None);
// default bounds are unbounded.
let targets = <Staking<T>>::get_npos_targets(DataProviderBounds::default());
assert_eq!(targets.len() as u32, v);
}
@@ -961,7 +963,7 @@ mod tests {
create_validators_with_nominators_for_era::<Test>(
v,
n,
<Test as Config>::MaxNominations::get() as usize,
MaxNominationsOf::<Test>::get() as usize,
false,
None,
)