Files
pezkuwi-subxt/substrate/frame/fast-unstake/src/mock.rs
T
Ankan 00b85c51df [NPoS] Paging reward payouts in order to scale rewardable nominators (#1189)
helps https://github.com/paritytech/polkadot-sdk/issues/439.
closes https://github.com/paritytech/polkadot-sdk/issues/473.

PR link in the older substrate repository:
https://github.com/paritytech/substrate/pull/13498.

# Context
Rewards payout is processed today in a single block and limited to
`MaxNominatorRewardedPerValidator`. This number is currently 512 on both
Kusama and Polkadot.

This PR tries to scale the nominators payout to an unlimited count in a
multi-block fashion. Exposures are stored in pages, with each page
capped to a certain number (`MaxExposurePageSize`). Starting out, this
number would be the same as `MaxNominatorRewardedPerValidator`, but
eventually, this number can be lowered through new runtime upgrades to
limit the rewardeable nominators per dispatched call instruction.

The changes in the PR are backward compatible.

## How payouts would work like after this change
Staking exposes two calls, 1) the existing `payout_stakers` and 2)
`payout_stakers_by_page`.

### payout_stakers
This remains backward compatible with no signature change. If for a
given era a validator has multiple pages, they can call `payout_stakers`
multiple times. The pages are executed in an ascending sequence and the
runtime takes care of preventing double claims.

### payout_stakers_by_page
Very similar to `payout_stakers` but also accepts an extra param
`page_index`. An account can choose to payout rewards only for an
explicitly passed `page_index`.

**Lets look at an example scenario**
Given an active validator on Kusama had 1100 nominators,
`MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards
to all nominators, the caller would need to call `payout_stakers` 3
times.

- `payout_stakers(origin, stash, e)` => will pay the first 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the second set of 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the last set of 76
nominators.
...
- `payout_stakers(origin, stash, e)` => calling it the 4th time would
return an error `InvalidPage`.

The above calls can also be replaced by `payout_stakers_by_page` and
passing a `page_index` explicitly.

## Commission note
Validator commission is paid out in chunks across all the pages where
each commission chunk is proportional to the total stake of the current
page. This implies higher the total stake of a page, higher will be the
commission. If all the pages of a validator's single era are paid out,
the sum of commission paid to the validator across all pages should be
equal to what the commission would have been if we had a non-paged
exposure.

### Migration Note
Strictly speaking, we did not need to bump our storage version since
there is no migration of storage in this PR. But it is still useful to
mark a storage upgrade for the following reasons:

- New storage items are introduced in this PR while some older storage
items are deprecated.
- For the next `HistoryDepth` eras, the exposure would be incrementally
migrated to its corresponding paged storage item.
- Runtimes using staking pallet would strictly need to wait at least
`HistoryDepth` eras with current upgraded version (14) for the migration
to complete. At some era `E` such that `E >
era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to
version X which will remove the deprecated storage items.
In other words, it is a strict requirement that E<sub>x</sub> -
E<sub>14</sub> > `HistoryDepth`, where
E<sub>x</sub> = Era at which deprecated storages are removed from
runtime,
E<sub>14</sub> = Era at which runtime is upgraded to version 14.
- For Polkadot and Kusama, there is a [tracker
ticket](https://github.com/paritytech/polkadot-sdk/issues/433) to clean
up the deprecated storage items.

### Storage Changes

#### Added
- ErasStakersOverview
- ClaimedRewards
- ErasStakersPaged

#### Deprecated
The following can be cleaned up after 84 eras which is tracked
[here](https://github.com/paritytech/polkadot-sdk/issues/433).

- ErasStakers.
- ErasStakersClipped.
- StakingLedger.claimed_rewards, renamed to
StakingLedger.legacy_claimed_rewards.

### Config Changes
- Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize.

### TODO
- [x] Tracker ticket for cleaning up the old code after 84 eras.
- [x] Add companion.
- [x] Redo benchmarks before merge.
- [x] Add Changelog for pallet_staking.
- [x] Pallet should be configurable to enable/disable paged rewards.
- [x] Commission payouts are distributed across pages.
- [x] Review documentation thoroughly.
- [x] Rename `MaxNominatorRewardedPerValidator` ->
`MaxExposurePageSize`.
- [x] NMap for `ErasStakersPaged`.
- [x] Deprecate ErasStakers.
- [x] Integrity tests.

### Followup issues
[Runtime api for deprecated ErasStakers storage
item](https://github.com/paritytech/polkadot-sdk/issues/426)

---------

Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: command-bot <>
2023-11-01 15:21:44 +01:00

359 lines
10 KiB
Rust

// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::{self as fast_unstake};
use frame_support::{
assert_ok, derive_impl,
pallet_prelude::*,
parameter_types,
traits::{ConstU64, Currency},
weights::constants::WEIGHT_REF_TIME_PER_SECOND,
};
use sp_runtime::{
traits::{Convert, IdentityLookup},
BuildStorage,
};
use pallet_staking::{Exposure, IndividualExposure, StakerStatus};
use sp_std::prelude::*;
pub type AccountId = u128;
pub type BlockNumber = u64;
pub type Balance = u128;
pub type T = Runtime;
parameter_types! {
pub BlockWeights: frame_system::limits::BlockWeights =
frame_system::limits::BlockWeights::simple_max(
Weight::from_parts(2u64 * WEIGHT_REF_TIME_PER_SECOND, u64::MAX),
);
}
#[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::Config for Runtime {
type Block = Block;
type AccountData = pallet_balances::AccountData<Balance>;
// we use U128 account id in order to get a better iteration order out of a map.
type AccountId = AccountId;
type Lookup = IdentityLookup<Self::AccountId>;
}
impl pallet_timestamp::Config for Runtime {
type Moment = u64;
type OnTimestampSet = ();
type MinimumPeriod = ConstU64<5>;
type WeightInfo = ();
}
parameter_types! {
pub static ExistentialDeposit: Balance = 1;
}
impl pallet_balances::Config for Runtime {
type MaxLocks = ConstU32<128>;
type MaxReserves = ();
type ReserveIdentifier = [u8; 8];
type Balance = Balance;
type RuntimeEvent = RuntimeEvent;
type DustRemoval = ();
type ExistentialDeposit = ExistentialDeposit;
type AccountStore = System;
type WeightInfo = ();
type FreezeIdentifier = ();
type MaxFreezes = ();
type RuntimeHoldReason = ();
type RuntimeFreezeReason = ();
type MaxHolds = ();
}
pallet_staking_reward_curve::build! {
const I_NPOS: sp_runtime::curve::PiecewiseLinear<'static> = curve!(
min_inflation: 0_025_000,
max_inflation: 0_100_000,
ideal_stake: 0_500_000,
falloff: 0_050_000,
max_piece_count: 40,
test_precision: 0_005_000,
);
}
parameter_types! {
pub const RewardCurve: &'static sp_runtime::curve::PiecewiseLinear<'static> = &I_NPOS;
pub static BondingDuration: u32 = 3;
pub static CurrentEra: u32 = 0;
pub static Ongoing: bool = false;
pub static MaxWinners: u32 = 100;
}
pub struct MockElection;
impl frame_election_provider_support::ElectionProviderBase for MockElection {
type AccountId = AccountId;
type BlockNumber = BlockNumber;
type MaxWinners = MaxWinners;
type DataProvider = Staking;
type Error = ();
}
impl frame_election_provider_support::ElectionProvider for MockElection {
fn ongoing() -> bool {
Ongoing::get()
}
fn elect() -> Result<frame_election_provider_support::BoundedSupportsOf<Self>, Self::Error> {
Err(())
}
}
impl pallet_staking::Config for Runtime {
type Currency = Balances;
type CurrencyBalance = Balance;
type UnixTime = pallet_timestamp::Pallet<Self>;
type CurrencyToVote = ();
type RewardRemainder = ();
type RuntimeEvent = RuntimeEvent;
type Slash = ();
type Reward = ();
type SessionsPerEra = ();
type SlashDeferDuration = ();
type AdminOrigin = frame_system::EnsureRoot<Self::AccountId>;
type BondingDuration = BondingDuration;
type SessionInterface = ();
type EraPayout = pallet_staking::ConvertCurve<RewardCurve>;
type NextNewSession = ();
type HistoryDepth = ConstU32<84>;
type MaxExposurePageSize = ConstU32<64>;
type OffendingValidatorsThreshold = ();
type ElectionProvider = MockElection;
type GenesisElectionProvider = Self::ElectionProvider;
type VoterList = pallet_staking::UseNominatorsAndValidatorsMap<Self>;
type TargetList = pallet_staking::UseValidatorsMap<Self>;
type NominationsQuota = pallet_staking::FixedNominationsQuota<16>;
type MaxUnlockingChunks = ConstU32<32>;
type EventListeners = ();
type BenchmarkingConfig = pallet_staking::TestBenchmarkingConfig;
type WeightInfo = ();
}
pub struct BalanceToU256;
impl Convert<Balance, sp_core::U256> for BalanceToU256 {
fn convert(n: Balance) -> sp_core::U256 {
n.into()
}
}
pub struct U256ToBalance;
impl Convert<sp_core::U256, Balance> for U256ToBalance {
fn convert(n: sp_core::U256) -> Balance {
n.try_into().unwrap()
}
}
parameter_types! {
pub static Deposit: u128 = 7;
pub static BatchSize: u32 = 1;
}
impl fast_unstake::Config for Runtime {
type RuntimeEvent = RuntimeEvent;
type Deposit = Deposit;
type Currency = Balances;
type Staking = Staking;
type ControlOrigin = frame_system::EnsureRoot<Self::AccountId>;
type BatchSize = BatchSize;
type WeightInfo = ();
type MaxErasToCheckPerBlock = ConstU32<16>;
}
type Block = frame_system::mocking::MockBlock<Runtime>;
frame_support::construct_runtime!(
pub struct Runtime {
System: frame_system,
Timestamp: pallet_timestamp,
Balances: pallet_balances,
Staking: pallet_staking,
FastUnstake: fast_unstake,
}
);
parameter_types! {
static FastUnstakeEvents: u32 = 0;
}
pub(crate) fn fast_unstake_events_since_last_call() -> Vec<super::Event<Runtime>> {
let events = System::events()
.into_iter()
.map(|r| r.event)
.filter_map(|e| if let RuntimeEvent::FastUnstake(inner) = e { Some(inner) } else { None })
.collect::<Vec<_>>();
let already_seen = FastUnstakeEvents::get();
FastUnstakeEvents::set(events.len() as u32);
events.into_iter().skip(already_seen as usize).collect()
}
pub struct ExtBuilder {
unexposed: Vec<(AccountId, AccountId, Balance)>,
}
impl Default for ExtBuilder {
fn default() -> Self {
Self {
unexposed: vec![
(1, 1, 7 + 100),
(3, 3, 7 + 100),
(5, 5, 7 + 100),
(7, 7, 7 + 100),
(9, 9, 7 + 100),
],
}
}
}
pub(crate) const VALIDATORS_PER_ERA: AccountId = 32;
pub(crate) const VALIDATOR_PREFIX: AccountId = 100;
pub(crate) const NOMINATORS_PER_VALIDATOR_PER_ERA: AccountId = 4;
pub(crate) const NOMINATOR_PREFIX: AccountId = 1000;
impl ExtBuilder {
pub(crate) fn register_stakers_for_era(era: u32) {
// validators are prefixed with 100 and nominators with 1000 to prevent conflict. Make sure
// all the other accounts used in tests are below 100. Also ensure here that we don't
// overlap.
assert!(VALIDATOR_PREFIX + VALIDATORS_PER_ERA < NOMINATOR_PREFIX);
(VALIDATOR_PREFIX..VALIDATOR_PREFIX + VALIDATORS_PER_ERA)
.map(|v| {
// for the sake of sanity, let's register this taker as an actual validator.
let others = (NOMINATOR_PREFIX..
(NOMINATOR_PREFIX + NOMINATORS_PER_VALIDATOR_PER_ERA))
.map(|n| IndividualExposure { who: n, value: 0 as Balance })
.collect::<Vec<_>>();
(v, Exposure { total: 0, own: 0, others })
})
.for_each(|(validator, exposure)| {
pallet_staking::ErasStakers::<T>::insert(era, validator, exposure);
});
}
pub(crate) fn batch(self, size: u32) -> Self {
BatchSize::set(size);
self
}
pub(crate) fn build(self) -> sp_io::TestExternalities {
sp_tracing::try_init_simple();
let mut storage =
frame_system::GenesisConfig::<Runtime>::default().build_storage().unwrap();
let validators_range = VALIDATOR_PREFIX..VALIDATOR_PREFIX + VALIDATORS_PER_ERA;
let nominators_range =
NOMINATOR_PREFIX..NOMINATOR_PREFIX + NOMINATORS_PER_VALIDATOR_PER_ERA;
let _ = pallet_balances::GenesisConfig::<Runtime> {
balances: self
.unexposed
.clone()
.into_iter()
.map(|(stash, _, balance)| (stash, balance * 2))
.chain(validators_range.clone().map(|x| (x, 7 + 100)))
.chain(nominators_range.clone().map(|x| (x, 7 + 100)))
.collect::<Vec<_>>(),
}
.assimilate_storage(&mut storage);
let _ = pallet_staking::GenesisConfig::<Runtime> {
stakers: self
.unexposed
.into_iter()
.map(|(x, y, z)| (x, y, z, pallet_staking::StakerStatus::Nominator(vec![42])))
.chain(validators_range.map(|x| (x, x, 100, StakerStatus::Validator)))
.chain(nominators_range.map(|x| (x, x, 100, StakerStatus::Nominator(vec![x]))))
.collect::<Vec<_>>(),
..Default::default()
}
.assimilate_storage(&mut storage);
let mut ext = sp_io::TestExternalities::from(storage);
ext.execute_with(|| {
// for events to be deposited.
frame_system::Pallet::<Runtime>::set_block_number(1);
for era in 0..=(BondingDuration::get()) {
Self::register_stakers_for_era(era);
}
// because we read this value as a measure of how many validators we have.
pallet_staking::ValidatorCount::<Runtime>::put(VALIDATORS_PER_ERA as u32);
});
ext
}
pub fn build_and_execute(self, test: impl FnOnce() -> ()) {
self.build().execute_with(|| {
test();
})
}
}
pub(crate) fn run_to_block(n: u64, on_idle: bool) {
let current_block = System::block_number();
assert!(n > current_block);
while System::block_number() < n {
Balances::on_finalize(System::block_number());
Staking::on_finalize(System::block_number());
FastUnstake::on_finalize(System::block_number());
System::set_block_number(System::block_number() + 1);
Balances::on_initialize(System::block_number());
Staking::on_initialize(System::block_number());
FastUnstake::on_initialize(System::block_number());
if on_idle {
FastUnstake::on_idle(System::block_number(), BlockWeights::get().max_block);
}
}
}
pub(crate) fn next_block(on_idle: bool) {
let current = System::block_number();
run_to_block(current + 1, on_idle);
}
pub fn assert_unstaked(stash: &AccountId) {
assert!(!pallet_staking::Bonded::<T>::contains_key(stash));
assert!(!pallet_staking::Payee::<T>::contains_key(stash));
assert!(!pallet_staking::Validators::<T>::contains_key(stash));
assert!(!pallet_staking::Nominators::<T>::contains_key(stash));
}
pub fn create_exposed_nominator(exposed: AccountId, era: u32) {
// create an exposed nominator in era 1
pallet_staking::ErasStakers::<T>::mutate(era, VALIDATORS_PER_ERA, |expo| {
expo.others.push(IndividualExposure { who: exposed, value: 0 as Balance });
});
Balances::make_free_balance_be(&exposed, 100);
assert_ok!(Staking::bond(
RuntimeOrigin::signed(exposed),
10,
pallet_staking::RewardDestination::Staked
));
assert_ok!(Staking::nominate(RuntimeOrigin::signed(exposed), vec![exposed]));
// register the exposed one.
assert_ok!(FastUnstake::register_fast_unstake(RuntimeOrigin::signed(exposed)));
}