Files
pezkuwi-subxt/substrate/frame/staking/src/mock.rs
T
Zeke Mostov 669972a171 Implement pallet-bags-list and its interfaces with pallet-staking (#9507)
* remove extra whitespace

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* only emit rebag event on success

* add doc explaining the term voter

* revamp/simplify rebag test

* ensure genesis accounts are placed into the correct nodes/bags

* bond_extra implicitly rebags

* types at top; doc public type

* start sketching out adjustable thresholds

* add integrity test for voter bag threshold requirements

* get rid of BagIdx

This reorganizes bag storage such that bags are always referred to
by their upper threshold. This in turn means that adding and removing
bags is cheaper; you only need to migrate certain voters, not all of them.

* implement migration logic for when the threshold list changes

* start sketching out threshold proc macros

* further refine macro signatures

* WIP: implement make_ratio macro

* start rethinking the process of producing threshold lists

The macro approach seems to be a non-starter; that only really works
if we're throwing around numeric literals everywhere, and that's just
not nice in this case.

Instead, let's write helper functions and make it really easy to
generate the tables in separate, permanent files, which humans
can then edit.

* write helper functions to emit voter bags module

* WIP: demo generating voter bags for a realistic runtime

This isn't yet done, becuase it seems to take a Very Long Time to run,
and it really shouldn't. Need to look into that.

Still, it's a lot closer than it was this morning.

* rm unnecessary arg_enum

* fix voter bags math

Turns out that when you're working in exponential space, you need
to divide, not subtract, in order to keep the math working properly.

Also neaten up the output a little bit to make it easier to read.

* add computed voter bags thresholds to node

* fixup some docs

* iter from large bags to small, fulfuilling the contract

* make tests compile

* add VoterBagThresholds to some configs

* ensure that iteration covers all voters even with implied final bag

* use sp_std::boxed::Box;

* fix unused import

* add some more voter bags tests

* file_header.txt

* integrity test to ensure min bag exceeds existential weight

* add more debug assertions about node list length

* rm unused imports

* Kian enters

* Update frame/election-provider-support/src/onchain.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Suggestions for #9081 (Store voters in unsorted bags) (#9328)

* Add some debug asserts to node::get and remove_node

* Improve the debug asserts in remove_node

* improve debug asserts

* Space

* Remove bad assertions

* Tests: WIP take_works

* Take test

* Doc comment

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Test storage is cleaned up;

* formatting

* Switch to simpler thresholds

* Update the storage cleanup test

* Remove hardcoded values from benchmark to make it more robust

* Fix tests to acces bags properly

* Sanity check WIP; tests failing

* Update sanity checks to be more correct

* Improve storage cleanup tests

* WIP remote_ext_tests

* Some notes on next steps

* Remove some stuff that was for remote-ext tests

* Some more cleanup to reduce diff

* More :clean:

* Mo cleanin

* small fix

* A lot of changes from kian

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>

* merge fallout

* Run cargo +nightly fmt

* Fix a bunch of stuff, remove not needed runtime arg of make-bags

* add logs

* Glue the new staking bags to the election snapshot (#9415)

* Glue the new staking bags to the election snapshot

* add CheckedRem (#9412)

* add CheckedRem

* fix

* Run fmt

* Test comment

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>

* Update node runtime with VoterSnapshotPerBlock

* Unit test for pallet-staking unsorted bags feature (targets #9081) (#9422)

* impl notional_bag_for_works

* Add tests: insert_as_works & insert_works

* Impl test: remove_works

* Trivial cleaning

* Add test: update_position_for_works

* Write out edge case; probably can delete later

* Add test: bags::get_works

* Add test: remove_node_happy_path_works

* Add test: remove_node_bad_paths_documented

* WIP: voting_data_works

* done

* Improve test voting_data_works

* Add comment

* Fill out test basic_setup_works

* Update: iteration_is_semi_sorted

* Improve remove_works

* Update update_position_for_works; create set_ledger_and_free_balance

* Improve get_works

* Improve storage clean up checks in remove test

* Test: impl rebag_works + insert_and_remove_works

* forgot file -  Test: impl rebag_works + insert_and_remove_works

* Small tweak

* Update voter_bags test to reflect unused bags are removed

* Unbond & Rebond: do_rebag

* Prevent infinite loops with duplicate tail insert

* Check iter.count on voter list in pre-migrate

* undo strang fmt comment stuff

* Add in todo

Co-authored-by: kianenigma <kian@parity.io>

* Try prepare for master merge

* Reduce diff

* Add comment for test to add

* Add in code TODO for update_position efficiency updates

* Initial impl compiles

* impl StakingVoterListStub

* Sample impl<T: Config> VoterListProvider<T> for VoterList<T>

* impl VoterListProvider for voter-bags

* WIP integrate pallet-voter-bags to staking mock

* the trait `pallet_staking::pallet::pallet::Config` is not implemented for `mock::Test`

* random

* pushing my stuff

* Mock working

* WIP voter list tests

* Add bag insert, remove tests

* Add test for bag insert and remove

* Add remaining tests for VoterList

* Add tests for node

* Add rebag works

* Add rebag extrinsic tests

* Rename to bags-list and name the list .. list!

* Rename VoterBagThresholds => BagThresholds

* Add test count_works

* Test on_update_works

* test sanity check

* a round of test fixes

* push a lot of changes

* my last changes

* all bags-list test work; fmt

* Beautify some tests

* Doc comment for bags-list

* Add insert warnings

* Setup initial benchmark

* Wire up WeightInfo

* is_terminal wip; everything broken!

* Is terminal working

* add TODOs for remove_node

* clean up remoe_node

* Fix all staking tests

* retire VoterBagFor

* commit

* bring in stashed changes

* save

* bench pipeline works now, but I can't run stuff

* sabe

* benchmarks now run, but we have a failure

* WIP: Wire up make_bags

* bags-thresholds compiles

* Fix most build issues

* This will fix all the tests

* move bag thresholds to bags-list

* Move bag-thresholds bin to within pallet-bags

* Remove some unnescary TODOs

* Impl tets wrong_rebag_is_noop

* assert remove is a noop with bad data

* Assert integrity test panics

* Return an error when inserting duplicates

* Update to handle error in staking pallet when inserting to list

* Test contains and on_insert error

* Test re-nominate does not mess up list or count

* Everything builds and works, only the benchmark...

* fuck yeah benchmarks

* more cleanup, more hardening.

* use the bags list again

* fix benhc

* Some questions and changs for List::migration

* Fix migration removed_bags and new_bags usage

* Some trivial aesthetic changes

* Some more trivial changes

* tiny changes/

* mega rename

* fix all tests and ci build

* nit

* Test and fix migration

* nit

* fmt

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fmt

* remove unused

* make a few things pub

* make node also pub.. for remote-ext test

* Fix all tests again

* Force bag changes in relevant benchmarks (targets #9507) (#9529)

* force rebag for unbond, rebond, and bond_extra

* nit

* Improve utils

* fmt

* nits

* Move generate_bags to its own pallet

* Get runtime-benchmarks feature setup with prepare_on_update_benchmark

* Withdraw unbonded kill working

* Nominate bench working

* some cleanup

* WIP

* update to check head pre & post conditions

* Add some post condition verification stuff for on_remove

* Update nominate

* fmt

* Improvements

* Fix build

* fix build with polkadot companion

* Update frame/bags-list/src/list/tests.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* move generate-bag from frame to utils

* wip

* refactor WIP

* WIP save

* Refactor working

* some variable renaming

* WIP: prepare to remove head checks

* Finish MvP refactor

* Some cleanup

* Soem more cleanup

* save

* fix a lot of stuff

* Update client/db/src/bench.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Fix some issues that came from trying to merge comments on github

* some small changes

* simplify it

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Build works

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Remove commented out debug assert

* Remove some unused deps and some unused benchmarking stuff

* Fix stakings ElectionDataProvider clear

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bags_list --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bags-list/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bags_list --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bags-list/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bags_list --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bags-list/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bags_list --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bags-list/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Improving staking pallet-bags-list migration

* fix build and some comments;

* comment

* Reduce visibility in bags list components

* make node.bag_upper only accesible to benchmarks

* Address some feedback; comments updates

* use nominator map comment

* fix vec capacity debug assert

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* clarify VoterSnapshotPerBlock

* Reduce diff on create_validators by wrapping with_seed

* Some small improvements to staking benches

* Soem comment updates

* fix vec capacity debug assert ... for real this time

* Reduce ListBags viz

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove supports_eq_unordered & Support eq_unordered

* Update utils/frame/generate-bags/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Make total-issuance & minimium-balance CLI args; Dont use emptry ext

* Improve docs for generate bags CLI args

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Don't use default bags weight in node runtime

* Feature gating sanity_check not working

* Feature gate sanity check by creating duplicate fns

* Fix line wrapping

* Document VoteWeightProvider

* Make bags ext-builder not a module

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* use pallet_bags_list instead of crate in mock

* Make get_bags test helper fn live in List

* use remove_from_storage_unchecked for node removal

* Remove count of ids removed in remove_many

* Add node sanity check, improve list sanity check

* Do a list sanity check after on_update

* List::migrate: clean up debug assert, exit early when no change in thresholds

* Improve public doc comments for pallet_bags_list::list::List

* Improve public doc comments for pallet_bags_list::list::List

* Update generate bags docs

* Fix grammar in bags-list benchmark

* Add benchmark case for `rebag` extrinsic

* Add count parameter to List::clear; WIP adding MaxEncodedLen to list'

* MaxEncodeLen + generate_storage_info not working for Bag or Node

* Get MaxEncodeLen derive to work

* Try to correctly feature gate SortedListProvider::clear

* Use u32::MAX, not u32::max_value

* Get up to nominators_quota noms

* SortedListProvider::clear takes an Option

* Eplicitly ignore SortedListProvider return value

* Fix doc comment

* Update node-runtime voter snapshot per block

* Add test get_max_len_voters_even_if_some_nominators_are_slashed

* Add test only_iterates_max_2_times_nominators_quota

* Fix generate bags cargo.toml

* use sp_std vec

* Remove v8 migration hooks from pallet-staking

* Update npos trait

* Try respect line width

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bags_list --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bags-list/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update frame/bags-list/src/benchmarks.rs

* Unwrap try-runtime error; remove sortedlistprovider pre upgrade len  check

* trigger ci

* restore

* trigger ci

* restore

* trigger ci

* revert

* trigger ci

* revert

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <peter.r.goodspeedniklaus@gmail.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-09-17 02:25:21 +00:00

869 lines
25 KiB
Rust

// This file is part of Substrate.
// Copyright (C) 2018-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Test utilities
use crate::{self as pallet_staking, *};
use frame_election_provider_support::{onchain, SortedListProvider};
use frame_support::{
assert_ok, parameter_types,
traits::{
Currency, FindAuthor, GenesisBuild, Get, Hooks, Imbalance, OnInitialize, OnUnbalanced,
OneSessionHandler,
},
weights::constants::RocksDbWeight,
};
use sp_core::H256;
use sp_io;
use sp_runtime::{
curve::PiecewiseLinear,
testing::{Header, TestXt, UintAuthorityId},
traits::{IdentityLookup, Zero},
};
use sp_staking::offence::{OffenceDetails, OnOffenceHandler};
use std::{cell::RefCell, collections::HashSet};
pub const INIT_TIMESTAMP: u64 = 30_000;
pub const BLOCK_TIME: u64 = 1000;
/// The AccountId alias in this test module.
pub(crate) type AccountId = u64;
pub(crate) type AccountIndex = u64;
pub(crate) type BlockNumber = u64;
pub(crate) type Balance = u128;
thread_local! {
static SESSION: RefCell<(Vec<AccountId>, HashSet<AccountId>)> = RefCell::new(Default::default());
}
/// Another session handler struct to test on_disabled.
pub struct OtherSessionHandler;
impl OneSessionHandler<AccountId> for OtherSessionHandler {
type Key = UintAuthorityId;
fn on_genesis_session<'a, I: 'a>(_: I)
where
I: Iterator<Item = (&'a AccountId, Self::Key)>,
AccountId: 'a,
{
}
fn on_new_session<'a, I: 'a>(_: bool, validators: I, _: I)
where
I: Iterator<Item = (&'a AccountId, Self::Key)>,
AccountId: 'a,
{
SESSION.with(|x| {
*x.borrow_mut() = (validators.map(|x| x.0.clone()).collect(), HashSet::new())
});
}
fn on_disabled(validator_index: usize) {
SESSION.with(|d| {
let mut d = d.borrow_mut();
let value = d.0[validator_index];
d.1.insert(value);
})
}
}
impl sp_runtime::BoundToRuntimeAppPublic for OtherSessionHandler {
type Public = UintAuthorityId;
}
pub fn is_disabled(controller: AccountId) -> bool {
let stash = Staking::ledger(&controller).unwrap().stash;
SESSION.with(|d| d.borrow().1.contains(&stash))
}
type UncheckedExtrinsic = frame_system::mocking::MockUncheckedExtrinsic<Test>;
type Block = frame_system::mocking::MockBlock<Test>;
frame_support::construct_runtime!(
pub enum Test where
Block = Block,
NodeBlock = Block,
UncheckedExtrinsic = UncheckedExtrinsic,
{
System: frame_system::{Pallet, Call, Config, Storage, Event<T>},
Authorship: pallet_authorship::{Pallet, Call, Storage, Inherent},
Timestamp: pallet_timestamp::{Pallet, Call, Storage, Inherent},
Balances: pallet_balances::{Pallet, Call, Storage, Config<T>, Event<T>},
Staking: pallet_staking::{Pallet, Call, Config<T>, Storage, Event<T>},
Session: pallet_session::{Pallet, Call, Storage, Event, Config<T>},
BagsList: pallet_bags_list::{Pallet, Call, Storage, Event<T>},
}
);
/// Author of block is always 11
pub struct Author11;
impl FindAuthor<AccountId> for Author11 {
fn find_author<'a, I>(_digests: I) -> Option<AccountId>
where
I: 'a + IntoIterator<Item = (frame_support::ConsensusEngineId, &'a [u8])>,
{
Some(11)
}
}
parameter_types! {
pub const BlockHashCount: u64 = 250;
pub BlockWeights: frame_system::limits::BlockWeights =
frame_system::limits::BlockWeights::simple_max(
frame_support::weights::constants::WEIGHT_PER_SECOND * 2
);
pub const MaxLocks: u32 = 1024;
pub static SessionsPerEra: SessionIndex = 3;
pub static ExistentialDeposit: Balance = 1;
pub static SlashDeferDuration: EraIndex = 0;
pub static Period: BlockNumber = 5;
pub static Offset: BlockNumber = 0;
}
impl frame_system::Config for Test {
type BaseCallFilter = frame_support::traits::Everything;
type BlockWeights = ();
type BlockLength = ();
type DbWeight = RocksDbWeight;
type Origin = Origin;
type Index = AccountIndex;
type BlockNumber = BlockNumber;
type Call = Call;
type Hash = H256;
type Hashing = ::sp_runtime::traits::BlakeTwo256;
type AccountId = AccountId;
type Lookup = IdentityLookup<Self::AccountId>;
type Header = Header;
type Event = Event;
type BlockHashCount = BlockHashCount;
type Version = ();
type PalletInfo = PalletInfo;
type AccountData = pallet_balances::AccountData<Balance>;
type OnNewAccount = ();
type OnKilledAccount = ();
type SystemWeightInfo = ();
type SS58Prefix = ();
type OnSetCode = ();
}
impl pallet_balances::Config for Test {
type MaxLocks = MaxLocks;
type MaxReserves = ();
type ReserveIdentifier = [u8; 8];
type Balance = Balance;
type Event = Event;
type DustRemoval = ();
type ExistentialDeposit = ExistentialDeposit;
type AccountStore = System;
type WeightInfo = ();
}
parameter_types! {
pub const UncleGenerations: u64 = 0;
pub const DisabledValidatorsThreshold: Perbill = Perbill::from_percent(25);
}
sp_runtime::impl_opaque_keys! {
pub struct SessionKeys {
pub other: OtherSessionHandler,
}
}
impl pallet_session::Config for Test {
type SessionManager = pallet_session::historical::NoteHistoricalRoot<Test, Staking>;
type Keys = SessionKeys;
type ShouldEndSession = pallet_session::PeriodicSessions<Period, Offset>;
type SessionHandler = (OtherSessionHandler,);
type Event = Event;
type ValidatorId = AccountId;
type ValidatorIdOf = crate::StashOf<Test>;
type DisabledValidatorsThreshold = DisabledValidatorsThreshold;
type NextSessionRotation = pallet_session::PeriodicSessions<Period, Offset>;
type WeightInfo = ();
}
impl pallet_session::historical::Config for Test {
type FullIdentification = crate::Exposure<AccountId, Balance>;
type FullIdentificationOf = crate::ExposureOf<Test>;
}
impl pallet_authorship::Config for Test {
type FindAuthor = Author11;
type UncleGenerations = UncleGenerations;
type FilterUncle = ();
type EventHandler = Pallet<Test>;
}
parameter_types! {
pub const MinimumPeriod: u64 = 5;
}
impl pallet_timestamp::Config for Test {
type Moment = u64;
type OnTimestampSet = ();
type MinimumPeriod = MinimumPeriod;
type WeightInfo = ();
}
pallet_staking_reward_curve::build! {
const I_NPOS: PiecewiseLinear<'static> = curve!(
min_inflation: 0_025_000,
max_inflation: 0_100_000,
ideal_stake: 0_500_000,
falloff: 0_050_000,
max_piece_count: 40,
test_precision: 0_005_000,
);
}
parameter_types! {
pub const BondingDuration: EraIndex = 3;
pub const RewardCurve: &'static PiecewiseLinear<'static> = &I_NPOS;
pub const MaxNominatorRewardedPerValidator: u32 = 64;
}
thread_local! {
pub static REWARD_REMAINDER_UNBALANCED: RefCell<u128> = RefCell::new(0);
}
pub struct RewardRemainderMock;
impl OnUnbalanced<NegativeImbalanceOf<Test>> for RewardRemainderMock {
fn on_nonzero_unbalanced(amount: NegativeImbalanceOf<Test>) {
REWARD_REMAINDER_UNBALANCED.with(|v| {
*v.borrow_mut() += amount.peek();
});
drop(amount);
}
}
const THRESHOLDS: [sp_npos_elections::VoteWeight; 9] =
[10, 20, 30, 40, 50, 60, 1_000, 2_000, 10_000];
parameter_types! {
pub static BagThresholds: &'static [sp_npos_elections::VoteWeight] = &THRESHOLDS;
}
impl pallet_bags_list::Config for Test {
type Event = Event;
type WeightInfo = ();
type VoteWeightProvider = Staking;
type BagThresholds = BagThresholds;
}
impl onchain::Config for Test {
type Accuracy = Perbill;
type DataProvider = Staking;
}
impl crate::pallet::pallet::Config for Test {
const MAX_NOMINATIONS: u32 = 16;
type Currency = Balances;
type UnixTime = Timestamp;
type CurrencyToVote = frame_support::traits::SaturatingCurrencyToVote;
type RewardRemainder = RewardRemainderMock;
type Event = Event;
type Slash = ();
type Reward = ();
type SessionsPerEra = SessionsPerEra;
type SlashDeferDuration = SlashDeferDuration;
type SlashCancelOrigin = frame_system::EnsureRoot<Self::AccountId>;
type BondingDuration = BondingDuration;
type SessionInterface = Self;
type EraPayout = ConvertCurve<RewardCurve>;
type NextNewSession = Session;
type MaxNominatorRewardedPerValidator = MaxNominatorRewardedPerValidator;
type ElectionProvider = onchain::OnChainSequentialPhragmen<Self>;
type GenesisElectionProvider = Self::ElectionProvider;
type WeightInfo = ();
// NOTE: consider a macro and use `UseNominatorsMap<Self>` as well.
type SortedListProvider = BagsList;
}
impl<LocalCall> frame_system::offchain::SendTransactionTypes<LocalCall> for Test
where
Call: From<LocalCall>,
{
type OverarchingCall = Call;
type Extrinsic = Extrinsic;
}
pub type Extrinsic = TestXt<Call, ()>;
pub(crate) type StakingCall = crate::Call<Test>;
pub(crate) type TestRuntimeCall = <Test as frame_system::Config>::Call;
pub struct ExtBuilder {
nominate: bool,
validator_count: u32,
minimum_validator_count: u32,
invulnerables: Vec<AccountId>,
has_stakers: bool,
initialize_first_session: bool,
min_nominator_bond: Balance,
min_validator_bond: Balance,
balance_factor: Balance,
status: BTreeMap<AccountId, StakerStatus<AccountId>>,
stakes: BTreeMap<AccountId, Balance>,
stakers: Vec<(AccountId, AccountId, Balance, StakerStatus<AccountId>)>,
}
impl Default for ExtBuilder {
fn default() -> Self {
Self {
nominate: true,
validator_count: 2,
minimum_validator_count: 0,
balance_factor: 1,
invulnerables: vec![],
has_stakers: true,
initialize_first_session: true,
min_nominator_bond: ExistentialDeposit::get(),
min_validator_bond: ExistentialDeposit::get(),
status: Default::default(),
stakes: Default::default(),
stakers: Default::default(),
}
}
}
impl ExtBuilder {
pub fn existential_deposit(self, existential_deposit: Balance) -> Self {
EXISTENTIAL_DEPOSIT.with(|v| *v.borrow_mut() = existential_deposit);
self
}
pub fn nominate(mut self, nominate: bool) -> Self {
self.nominate = nominate;
self
}
pub fn validator_count(mut self, count: u32) -> Self {
self.validator_count = count;
self
}
pub fn minimum_validator_count(mut self, count: u32) -> Self {
self.minimum_validator_count = count;
self
}
pub fn slash_defer_duration(self, eras: EraIndex) -> Self {
SLASH_DEFER_DURATION.with(|v| *v.borrow_mut() = eras);
self
}
pub fn invulnerables(mut self, invulnerables: Vec<AccountId>) -> Self {
self.invulnerables = invulnerables;
self
}
pub fn session_per_era(self, length: SessionIndex) -> Self {
SESSIONS_PER_ERA.with(|v| *v.borrow_mut() = length);
self
}
pub fn period(self, length: BlockNumber) -> Self {
PERIOD.with(|v| *v.borrow_mut() = length);
self
}
pub fn has_stakers(mut self, has: bool) -> Self {
self.has_stakers = has;
self
}
pub fn initialize_first_session(mut self, init: bool) -> Self {
self.initialize_first_session = init;
self
}
pub fn offset(self, offset: BlockNumber) -> Self {
OFFSET.with(|v| *v.borrow_mut() = offset);
self
}
pub fn min_nominator_bond(mut self, amount: Balance) -> Self {
self.min_nominator_bond = amount;
self
}
pub fn min_validator_bond(mut self, amount: Balance) -> Self {
self.min_validator_bond = amount;
self
}
pub fn set_status(mut self, who: AccountId, status: StakerStatus<AccountId>) -> Self {
self.status.insert(who, status);
self
}
pub fn set_stake(mut self, who: AccountId, stake: Balance) -> Self {
self.stakes.insert(who, stake);
self
}
pub fn add_staker(
mut self,
stash: AccountId,
ctrl: AccountId,
stake: Balance,
status: StakerStatus<AccountId>,
) -> Self {
self.stakers.push((stash, ctrl, stake, status));
self
}
pub fn balance_factor(mut self, factor: Balance) -> Self {
self.balance_factor = factor;
self
}
fn build(self) -> sp_io::TestExternalities {
sp_tracing::try_init_simple();
let mut storage = frame_system::GenesisConfig::default().build_storage::<Test>().unwrap();
let _ = pallet_balances::GenesisConfig::<Test> {
balances: vec![
(1, 10 * self.balance_factor),
(2, 20 * self.balance_factor),
(3, 300 * self.balance_factor),
(4, 400 * self.balance_factor),
// controllers
(10, self.balance_factor),
(20, self.balance_factor),
(30, self.balance_factor),
(40, self.balance_factor),
(50, self.balance_factor),
// stashes
(11, self.balance_factor * 1000),
(21, self.balance_factor * 2000),
(31, self.balance_factor * 2000),
(41, self.balance_factor * 2000),
(51, self.balance_factor * 2000),
// optional nominator
(100, self.balance_factor * 2000),
(101, self.balance_factor * 2000),
// aux accounts
(60, self.balance_factor),
(61, self.balance_factor * 2000),
(70, self.balance_factor),
(71, self.balance_factor * 2000),
(80, self.balance_factor),
(81, self.balance_factor * 2000),
// This allows us to have a total_payout different from 0.
(999, 1_000_000_000_000),
],
}
.assimilate_storage(&mut storage);
let mut stakers = vec![];
if self.has_stakers {
stakers = vec![
// (stash, ctrl, stake, status)
// these two will be elected in the default test where we elect 2.
(11, 10, self.balance_factor * 1000, StakerStatus::<AccountId>::Validator),
(21, 20, self.balance_factor * 1000, StakerStatus::<AccountId>::Validator),
// a loser validator
(31, 30, self.balance_factor * 500, StakerStatus::<AccountId>::Validator),
// an idle validator
(41, 40, self.balance_factor * 1000, StakerStatus::<AccountId>::Idle),
];
// optionally add a nominator
if self.nominate {
stakers.push((
101,
100,
self.balance_factor * 500,
StakerStatus::<AccountId>::Nominator(vec![11, 21]),
))
}
// replace any of the status if needed.
self.status.into_iter().for_each(|(stash, status)| {
let (_, _, _, ref mut prev_status) = stakers
.iter_mut()
.find(|s| s.0 == stash)
.expect("set_status staker should exist; qed");
*prev_status = status;
});
// replaced any of the stakes if needed.
self.stakes.into_iter().for_each(|(stash, stake)| {
let (_, _, ref mut prev_stake, _) = stakers
.iter_mut()
.find(|s| s.0 == stash)
.expect("set_stake staker should exits; qed.");
*prev_stake = stake;
});
// extend stakers if needed.
stakers.extend(self.stakers)
}
let _ = pallet_staking::GenesisConfig::<Test> {
stakers,
validator_count: self.validator_count,
minimum_validator_count: self.minimum_validator_count,
invulnerables: self.invulnerables,
slash_reward_fraction: Perbill::from_percent(10),
min_nominator_bond: self.min_nominator_bond,
min_validator_bond: self.min_validator_bond,
..Default::default()
}
.assimilate_storage(&mut storage);
let _ = pallet_session::GenesisConfig::<Test> {
keys: if self.has_stakers {
// genesis election will overwrite this, no worries.
Default::default()
} else {
// set some dummy validators in genesis.
(0..self.validator_count as u64)
.map(|x| (x, x, SessionKeys { other: UintAuthorityId(x as u64) }))
.collect()
},
}
.assimilate_storage(&mut storage);
let mut ext = sp_io::TestExternalities::from(storage);
ext.execute_with(|| {
let validators = Session::validators();
SESSION.with(|x| *x.borrow_mut() = (validators.clone(), HashSet::new()));
});
if self.initialize_first_session {
// We consider all test to start after timestamp is initialized This must be ensured by
// having `timestamp::on_initialize` called before `staking::on_initialize`. Also, if
// session length is 1, then it is already triggered.
ext.execute_with(|| {
System::set_block_number(1);
Session::on_initialize(1);
<Staking as Hooks<u64>>::on_initialize(1);
Timestamp::set_timestamp(INIT_TIMESTAMP);
});
}
ext
}
pub fn build_and_execute(self, test: impl FnOnce() -> ()) {
let mut ext = self.build();
ext.execute_with(test);
ext.execute_with(post_conditions);
}
}
fn post_conditions() {
check_nominators();
check_exposures();
check_ledgers();
check_count();
}
fn check_count() {
let nominator_count = Nominators::<Test>::iter().count() as u32;
let validator_count = Validators::<Test>::iter().count() as u32;
assert_eq!(nominator_count, CounterForNominators::<Test>::get());
assert_eq!(validator_count, CounterForValidators::<Test>::get());
// the voters that the `SortedListProvider` list is storing for us.
let external_voters = <Test as Config>::SortedListProvider::count();
assert_eq!(external_voters, nominator_count);
}
fn check_ledgers() {
// check the ledger of all stakers.
Bonded::<Test>::iter().for_each(|(_, ctrl)| assert_ledger_consistent(ctrl))
}
fn check_exposures() {
// a check per validator to ensure the exposure struct is always sane.
let era = active_era();
ErasStakers::<Test>::iter_prefix_values(era).for_each(|expo| {
assert_eq!(
expo.total as u128,
expo.own as u128 + expo.others.iter().map(|e| e.value as u128).sum::<u128>(),
"wrong total exposure.",
);
})
}
fn check_nominators() {
// a check per nominator to ensure their entire stake is correctly distributed. Will only kick-
// in if the nomination was submitted before the current era.
let era = active_era();
<Nominators<Test>>::iter()
.filter_map(
|(nominator, nomination)| {
if nomination.submitted_in > era {
Some(nominator)
} else {
None
}
},
)
.for_each(|nominator| {
// must be bonded.
assert_is_stash(nominator);
let mut sum = 0;
Session::validators()
.iter()
.map(|v| Staking::eras_stakers(era, v))
.for_each(|e| {
let individual =
e.others.iter().filter(|e| e.who == nominator).collect::<Vec<_>>();
let len = individual.len();
match len {
0 => { /* not supporting this validator at all. */ },
1 => sum += individual[0].value,
_ => panic!("nominator cannot back a validator more than once."),
};
});
let nominator_stake = Staking::slashable_balance_of(&nominator);
// a nominator cannot over-spend.
assert!(
nominator_stake >= sum,
"failed: Nominator({}) stake({}) >= sum divided({})",
nominator,
nominator_stake,
sum,
);
let diff = nominator_stake - sum;
assert!(diff < 100);
});
}
fn assert_is_stash(acc: AccountId) {
assert!(Staking::bonded(&acc).is_some(), "Not a stash.");
}
fn assert_ledger_consistent(ctrl: AccountId) {
// ensures ledger.total == ledger.active + sum(ledger.unlocking).
let ledger = Staking::ledger(ctrl).expect("Not a controller.");
let real_total: Balance = ledger.unlocking.iter().fold(ledger.active, |a, c| a + c.value);
assert_eq!(real_total, ledger.total);
assert!(
ledger.active >= Balances::minimum_balance() || ledger.active == 0,
"{}: active ledger amount ({}) must be greater than ED {}",
ctrl,
ledger.active,
Balances::minimum_balance()
);
}
pub(crate) fn active_era() -> EraIndex {
Staking::active_era().unwrap().index
}
pub(crate) fn current_era() -> EraIndex {
Staking::current_era().unwrap()
}
pub(crate) fn bond(stash: AccountId, ctrl: AccountId, val: Balance) {
let _ = Balances::make_free_balance_be(&stash, val);
let _ = Balances::make_free_balance_be(&ctrl, val);
assert_ok!(Staking::bond(Origin::signed(stash), ctrl, val, RewardDestination::Controller));
}
pub(crate) fn bond_validator(stash: AccountId, ctrl: AccountId, val: Balance) {
bond(stash, ctrl, val);
assert_ok!(Staking::validate(Origin::signed(ctrl), ValidatorPrefs::default()));
}
pub(crate) fn bond_nominator(
stash: AccountId,
ctrl: AccountId,
val: Balance,
target: Vec<AccountId>,
) {
bond(stash, ctrl, val);
assert_ok!(Staking::nominate(Origin::signed(ctrl), target));
}
/// Progress to the given block, triggering session and era changes as we progress.
///
/// This will finalize the previous block, initialize up to the given block, essentially simulating
/// a block import/propose process where we first initialize the block, then execute some stuff (not
/// in the function), and then finalize the block.
pub(crate) fn run_to_block(n: BlockNumber) {
Staking::on_finalize(System::block_number());
for b in (System::block_number() + 1)..=n {
System::set_block_number(b);
Session::on_initialize(b);
<Staking as Hooks<u64>>::on_initialize(b);
Timestamp::set_timestamp(System::block_number() * BLOCK_TIME + INIT_TIMESTAMP);
if b != n {
Staking::on_finalize(System::block_number());
}
}
}
/// Progresses from the current block number (whatever that may be) to the `P * session_index + 1`.
pub(crate) fn start_session(session_index: SessionIndex) {
let end: u64 = if Offset::get().is_zero() {
(session_index as u64) * Period::get()
} else {
Offset::get() + (session_index.saturating_sub(1) as u64) * Period::get()
};
run_to_block(end);
// session must have progressed properly.
assert_eq!(
Session::current_index(),
session_index,
"current session index = {}, expected = {}",
Session::current_index(),
session_index,
);
}
/// Go one session forward.
pub(crate) fn advance_session() {
let current_index = Session::current_index();
start_session(current_index + 1);
}
/// Progress until the given era.
pub(crate) fn start_active_era(era_index: EraIndex) {
start_session((era_index * <SessionsPerEra as Get<u32>>::get()).into());
assert_eq!(active_era(), era_index);
// One way or another, current_era must have changed before the active era, so they must match
// at this point.
assert_eq!(current_era(), active_era());
}
pub(crate) fn current_total_payout_for_duration(duration: u64) -> Balance {
let (payout, _rest) = <Test as Config>::EraPayout::era_payout(
Staking::eras_total_stake(active_era()),
Balances::total_issuance(),
duration,
);
assert!(payout > 0);
payout
}
pub(crate) fn maximum_payout_for_duration(duration: u64) -> Balance {
let (payout, rest) = <Test as Config>::EraPayout::era_payout(
Staking::eras_total_stake(active_era()),
Balances::total_issuance(),
duration,
);
payout + rest
}
/// Time it takes to finish a session.
///
/// Note, if you see `time_per_session() - BLOCK_TIME`, it is fine. This is because we set the
/// timestamp after on_initialize, so the timestamp is always one block old.
pub(crate) fn time_per_session() -> u64 {
Period::get() * BLOCK_TIME
}
/// Time it takes to finish an era.
///
/// Note, if you see `time_per_era() - BLOCK_TIME`, it is fine. This is because we set the
/// timestamp after on_initialize, so the timestamp is always one block old.
pub(crate) fn time_per_era() -> u64 {
time_per_session() * SessionsPerEra::get() as u64
}
/// Time that will be calculated for the reward per era.
pub(crate) fn reward_time_per_era() -> u64 {
time_per_era() - BLOCK_TIME
}
pub(crate) fn reward_all_elected() {
let rewards = <Test as Config>::SessionInterface::validators().into_iter().map(|v| (v, 1));
<Pallet<Test>>::reward_by_ids(rewards)
}
pub(crate) fn validator_controllers() -> Vec<AccountId> {
Session::validators()
.into_iter()
.map(|s| Staking::bonded(&s).expect("no controller for validator"))
.collect()
}
pub(crate) fn on_offence_in_era(
offenders: &[OffenceDetails<
AccountId,
pallet_session::historical::IdentificationTuple<Test>,
>],
slash_fraction: &[Perbill],
era: EraIndex,
) {
let bonded_eras = crate::BondedEras::<Test>::get();
for &(bonded_era, start_session) in bonded_eras.iter() {
if bonded_era == era {
let _ = Staking::on_offence(offenders, slash_fraction, start_session);
return
} else if bonded_era > era {
break
}
}
if Staking::active_era().unwrap().index == era {
let _ = Staking::on_offence(
offenders,
slash_fraction,
Staking::eras_start_session_index(era).unwrap(),
);
} else {
panic!("cannot slash in era {}", era);
}
}
pub(crate) fn on_offence_now(
offenders: &[OffenceDetails<
AccountId,
pallet_session::historical::IdentificationTuple<Test>,
>],
slash_fraction: &[Perbill],
) {
let now = Staking::active_era().unwrap().index;
on_offence_in_era(offenders, slash_fraction, now)
}
pub(crate) fn add_slash(who: &AccountId) {
on_offence_now(
&[OffenceDetails {
offender: (who.clone(), Staking::eras_stakers(active_era(), who.clone())),
reporters: vec![],
}],
&[Perbill::from_percent(10)],
);
}
/// Make all validator and nominator request their payment
pub(crate) fn make_all_reward_payment(era: EraIndex) {
let validators_with_reward = ErasRewardPoints::<Test>::get(era)
.individual
.keys()
.cloned()
.collect::<Vec<_>>();
// reward validators
for validator_controller in validators_with_reward.iter().filter_map(Staking::bonded) {
let ledger = <Ledger<Test>>::get(&validator_controller).unwrap();
assert_ok!(Staking::payout_stakers(Origin::signed(1337), ledger.stash, era));
}
}
#[macro_export]
macro_rules! assert_session_era {
($session:expr, $era:expr) => {
assert_eq!(
Session::current_index(),
$session,
"wrong session {} != {}",
Session::current_index(),
$session,
);
assert_eq!(
Staking::current_era().unwrap(),
$era,
"wrong current era {} != {}",
Staking::current_era().unwrap(),
$era,
);
};
}
pub(crate) fn staking_events() -> Vec<crate::Event<Test>> {
System::events()
.into_iter()
.map(|r| r.event)
.filter_map(|e| if let Event::Staking(inner) = e { Some(inner) } else { None })
.collect()
}
pub(crate) fn balances(who: &AccountId) -> (Balance, Balance) {
(Balances::free_balance(who), Balances::reserved_balance(who))
}