* Apply. * get rid of glob import * use meaningful generic type name * pjr_check operates on `Supports` struct used elsewhere * improve algorithmic complexity of `prepare_pjr_input` * fix rustdoc warnings * improve module docs * typo * simplify debug assertion * add test finding the phase-change threshold value for a constructed scenario * add more threshold scenarios to disambiguate plausible interpretations * add link to npos paper reference * docs: staked_assignment -> supports Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * add utility method for generating npos inputs * add a fuzzer which asserts that all unbalanced seq_phragmen are PJR Note that this currently fails. I hope that this can be rectified by calculating the threshold instead of choosing some arbitrary number. * assert in all cases, not just debug * leverage a native solution to choose candidates * use existing helper methods * add pjr-check and incorporate into the fuzzer We should probably have one of the W3F people look at this to ensure we're not misconstruing any definitions, but this seems like a fairly straightforward implementation. * fix compilation errors * Enable manually setting iteration parameters in single run. This gives us the ability to reproducably extract cases where honggfuzz has discovered a panic. For example: $ cargo run --release --bin phragmen_pjr -- --candidates 569 --voters 100 Tue 23 Feb 2021 11:23:39 AM CET Compiling bitflags v1.2.1 Compiling unicode-width v0.1.8 Compiling unicode-segmentation v1.7.1 Compiling ansi_term v0.11.0 Compiling strsim v0.8.0 Compiling vec_map v0.8.2 Compiling proc-macro-error-attr v1.0.4 Compiling proc-macro-error v1.0.4 Compiling textwrap v0.11.0 Compiling atty v0.2.14 Compiling heck v0.3.2 Compiling clap v2.33.3 Compiling structopt-derive v0.4.14 Compiling structopt v0.3.21 Compiling sp-npos-elections-fuzzer v2.0.0-alpha.5 (/home/coriolinus/Documents/Projects/paritytech/substrate/primitives/npos-elections/fuzzer) Finished release [optimized] target(s) in 6.15s Running `/home/coriolinus/Documents/Projects/paritytech/substrate/target/release/phragmen_pjr -c 569 -v 100` thread 'main' panicked at 'unbalanced sequential phragmen must satisfy PJR', primitives/npos-elections/fuzzer/src/phragmen_pjr.rs:133:5 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace This is still not adequate proof that seq_phragmen is broken; it could very well be that our PJR checker is doing the wrong thing, or we've somehow missed a parameter of interest. Still, it's concerning. * update comment verbiage for accuracy * it is valid in PJR for an elected candidate to have 0 support * Fix phragmen_pjr fuzzer It turns out that the fundamental problem causing previous implementations of the fuzzer to fail wasn't in `seq_phragmen` _or_ in `pjr_check`: it was in the rounding errors introduced in the various conversions between the internal data representation and the external one. Fixing the fuzzer is then simply an issue of using the internal representation and staying in that representation. However, that leaves the issue that `seq_phragmen` occasionally produces an output which is technically not PJR due to rounding errors. In the future we will need to add some kind of "close-enough" threshold. However, that is explicitly out of scope of this PR. * restart ci; it appears to be stalled * use necessary import for no-std * use a more realistic distribution of voters and candidates This isn't ideal; more realistic numbers would be about twice these. However, either case generation or voting has nonlinear execution time, and doubling these values brings iteration time from ~20s to ~180s. Fuzzing 6x as fast should make up for fuzzing cases half the size. * identify specifically which PJR check may fail * move candidate collection comment into correct place * standard_threshold: use a calculation method which cannot overflow * Apply suggestions from code review (update comments) Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * clarify the effectiveness bounds for t-pjr check * how to spell "committee" * reorganize: high -> low abstraction * ensure standard threshold calc cannot panic Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: kianenigma <kian.peymani@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Staking Module
The Staking module is used to manage funds at stake by network maintainers.
Overview
The Staking module is the means by which a set of network maintainers (known as authorities in some contexts and validators in others) are chosen based upon those who voluntarily place funds under deposit. Under deposit, those funds are rewarded under normal operation but are held at pain of slash (expropriation) should the staked maintainer be found not to be discharging its duties properly.
Terminology
- Staking: The process of locking up funds for some time, placing them at risk of slashing (loss) in order to become a rewarded maintainer of the network.
- Validating: The process of running a node to actively maintain the network, either by producing blocks or guaranteeing finality of the chain.
- Nominating: The process of placing staked funds behind one or more validators in order to share in any reward, and punishment, they take.
- Stash account: The account holding an owner's funds used for staking.
- Controller account: The account that controls an owner's funds for staking.
- Era: A (whole) number of sessions, which is the period that the validator set (and each validator's active nominator set) is recalculated and where rewards are paid out.
- Slash: The punishment of a staker by reducing its funds.
Goals
The staking system in Substrate NPoS is designed to make the following possible:
- Stake funds that are controlled by a cold wallet.
- Withdraw some, or deposit more, funds without interrupting the role of an entity.
- Switch between roles (nominator, validator, idle) with minimal overhead.
Scenarios
Staking
Almost any interaction with the Staking module requires a process of bonding (also known as being a staker). To become bonded, a fund-holding account known as the stash account, which holds some or all of the funds that become frozen in place as part of the staking process, is paired with an active controller account, which issues instructions on how they shall be used.
An account pair can become bonded using the bond call.
Stash accounts can change their associated controller using the
set_controller call.
There are three possible roles that any staked account pair can be in: Validator, Nominator
and Idle (defined in StakerStatus). There are three
corresponding instructions to change between roles, namely:
validate,
nominate, and chill.
Validating
A validator takes the role of either validating blocks or ensuring their finality, maintaining the veracity of the network. A validator should avoid both any sort of malicious misbehavior and going offline. Bonded accounts that state interest in being a validator do NOT get immediately chosen as a validator. Instead, they are declared as a candidate and they might get elected at the next era as a validator. The result of the election is determined by nominators and their votes.
An account can become a validator candidate via the
validate call.
Nomination
A nominator does not take any direct role in maintaining the network, instead, it votes on a set of validators to be elected. Once interest in nomination is stated by an account, it takes effect at the next election round. The funds in the nominator's stash account indicate the weight of its vote. Both the rewards and any punishment that a validator earns are shared between the validator and its nominators. This rule incentivizes the nominators to NOT vote for the misbehaving/offline validators as much as possible, simply because the nominators will also lose funds if they vote poorly.
An account can become a nominator via the nominate call.
Rewards and Slash
The reward and slashing procedure is the core of the Staking module, attempting to embrace valid behavior while punishing any misbehavior or lack of availability.
Rewards must be claimed for each era before it gets too old by $HISTORY_DEPTH using the
payout_stakers call. Any account can call payout_stakers, which pays the reward to the
validator as well as its nominators. Only the [Config::MaxNominatorRewardedPerValidator]
biggest stakers can claim their reward. This is to limit the i/o cost to mutate storage for each
nominator's account.
Slashing can occur at any point in time, once misbehavior is reported. Once slashing is determined, a value is deducted from the balance of the validator and all the nominators who voted for this validator (values are deducted from the stash account of the slashed entity).
Slashing logic is further described in the documentation of the slashing module.
Similar to slashing, rewards are also shared among a validator and its associated nominators. Yet, the reward funds are not always transferred to the stash account and can be configured. See Reward Calculation for more details.
Chilling
Finally, any of the roles above can choose to step back temporarily and just chill for a while. This means that if they are a nominator, they will not be considered as voters anymore and if they are validators, they will no longer be a candidate for the next election.
An account can step back via the chill call.
Session managing
The module implement the trait SessionManager. Which is the only API to query new validator
set and allowing these validator set to be rewarded once their era is ended.
Interface
Dispatchable Functions
The dispatchable functions of the Staking module enable the steps needed for entities to accept and change their role, alongside some helper functions to get/set the metadata of the module.
Public Functions
The Staking module contains many public storage items and (im)mutable functions.
Usage
Example: Rewarding a validator by id.
use frame_support::{decl_module, dispatch};
use frame_system::ensure_signed;
use pallet_staking::{self as staking};
pub trait Config: staking::Config {}
decl_module! {
pub struct Module<T: Config> for enum Call where origin: T::Origin {
/// Reward a validator.
#[weight = 0]
pub fn reward_myself(origin) -> dispatch::DispatchResult {
let reported = ensure_signed(origin)?;
<staking::Module<T>>::reward_by_ids(vec![(reported, 10)]);
Ok(())
}
}
}
Implementation Details
Era payout
The era payout is computed using yearly inflation curve defined at
T::RewardCurve as such:
staker_payout = yearly_inflation(npos_token_staked / total_tokens) * total_tokens / era_per_year
This payout is used to reward stakers as defined in next section
remaining_payout = max_yearly_inflation * total_tokens / era_per_year - staker_payout
The remaining reward is send to the configurable end-point
T::RewardRemainder.
Reward Calculation
Validators and nominators are rewarded at the end of each era. The total reward of an era is calculated using the era duration and the staking rate (the total amount of tokens staked by nominators and validators, divided by the total token supply). It aims to incentivize toward a defined staking rate. The full specification can be found here.
Total reward is split among validators and their nominators depending on the number of points
they received during the era. Points are added to a validator using
reward_by_ids or
reward_by_indices.
Module implements
pallet_authorship::EventHandler to add reward
points to block producer and block producer of referenced uncles.
The validator and its nominator split their reward as following:
The validator can declare an amount, named
commission, that does not get shared
with the nominators at each reward payout through its
ValidatorPrefs. This value gets deducted from the total reward
that is paid to the validator and its nominators. The remaining portion is split among the
validator and all of the nominators that nominated the validator, proportional to the value
staked behind this validator (i.e. dividing the
own or
others by
total in Exposure).
All entities who receive a reward have the option to choose their reward destination through the
Payee storage item (see
set_payee), to be one of the following:
- Controller account, (obviously) not increasing the staked value.
- Stash account, not increasing the staked value.
- Stash account, also increasing the staked value.
Additional Fund Management Operations
Any funds already placed into stash can be the target of the following operations:
The controller account can free a portion (or all) of the funds using the
unbond call. Note that the funds are not immediately
accessible. Instead, a duration denoted by BondingDuration
(in number of eras) must pass until the funds can actually be removed. Once the
BondingDuration is over, the withdraw_unbonded
call can be used to actually withdraw the funds.
Note that there is a limitation to the number of fund-chunks that can be scheduled to be
unlocked in the future via unbond. In case this maximum
(MAX_UNLOCKING_CHUNKS) is reached, the bonded account must first wait until a successful
call to withdraw_unbonded to remove some of the chunks.
Election Algorithm
The current election algorithm is implemented based on Phragmén. The reference implementation can be found here.
The election algorithm, aside from electing the validators with the most stake value and votes, tries to divide the nominator votes among candidates in an equal manner. To further assure this, an optional post-processing can be applied that iteratively normalizes the nominator staked values until the total difference among votes of a particular nominator are less than a threshold.
GenesisConfig
The Staking module depends on the GenesisConfig. The
GenesisConfig is optional and allow to set some initial stakers.
Related Modules
- Balances: Used to manage values at stake.
- Session: Used to manage sessions. Also, a list of new
validators is stored in the Session module's
Validatorsat the end of each era.
License: Apache-2.0