helps https://github.com/paritytech/polkadot-sdk/issues/439.
closes https://github.com/paritytech/polkadot-sdk/issues/473.
PR link in the older substrate repository:
https://github.com/paritytech/substrate/pull/13498.
# Context
Rewards payout is processed today in a single block and limited to
`MaxNominatorRewardedPerValidator`. This number is currently 512 on both
Kusama and Polkadot.
This PR tries to scale the nominators payout to an unlimited count in a
multi-block fashion. Exposures are stored in pages, with each page
capped to a certain number (`MaxExposurePageSize`). Starting out, this
number would be the same as `MaxNominatorRewardedPerValidator`, but
eventually, this number can be lowered through new runtime upgrades to
limit the rewardeable nominators per dispatched call instruction.
The changes in the PR are backward compatible.
## How payouts would work like after this change
Staking exposes two calls, 1) the existing `payout_stakers` and 2)
`payout_stakers_by_page`.
### payout_stakers
This remains backward compatible with no signature change. If for a
given era a validator has multiple pages, they can call `payout_stakers`
multiple times. The pages are executed in an ascending sequence and the
runtime takes care of preventing double claims.
### payout_stakers_by_page
Very similar to `payout_stakers` but also accepts an extra param
`page_index`. An account can choose to payout rewards only for an
explicitly passed `page_index`.
**Lets look at an example scenario**
Given an active validator on Kusama had 1100 nominators,
`MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards
to all nominators, the caller would need to call `payout_stakers` 3
times.
- `payout_stakers(origin, stash, e)` => will pay the first 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the second set of 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the last set of 76
nominators.
...
- `payout_stakers(origin, stash, e)` => calling it the 4th time would
return an error `InvalidPage`.
The above calls can also be replaced by `payout_stakers_by_page` and
passing a `page_index` explicitly.
## Commission note
Validator commission is paid out in chunks across all the pages where
each commission chunk is proportional to the total stake of the current
page. This implies higher the total stake of a page, higher will be the
commission. If all the pages of a validator's single era are paid out,
the sum of commission paid to the validator across all pages should be
equal to what the commission would have been if we had a non-paged
exposure.
### Migration Note
Strictly speaking, we did not need to bump our storage version since
there is no migration of storage in this PR. But it is still useful to
mark a storage upgrade for the following reasons:
- New storage items are introduced in this PR while some older storage
items are deprecated.
- For the next `HistoryDepth` eras, the exposure would be incrementally
migrated to its corresponding paged storage item.
- Runtimes using staking pallet would strictly need to wait at least
`HistoryDepth` eras with current upgraded version (14) for the migration
to complete. At some era `E` such that `E >
era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to
version X which will remove the deprecated storage items.
In other words, it is a strict requirement that E<sub>x</sub> -
E<sub>14</sub> > `HistoryDepth`, where
E<sub>x</sub> = Era at which deprecated storages are removed from
runtime,
E<sub>14</sub> = Era at which runtime is upgraded to version 14.
- For Polkadot and Kusama, there is a [tracker
ticket](https://github.com/paritytech/polkadot-sdk/issues/433) to clean
up the deprecated storage items.
### Storage Changes
#### Added
- ErasStakersOverview
- ClaimedRewards
- ErasStakersPaged
#### Deprecated
The following can be cleaned up after 84 eras which is tracked
[here](https://github.com/paritytech/polkadot-sdk/issues/433).
- ErasStakers.
- ErasStakersClipped.
- StakingLedger.claimed_rewards, renamed to
StakingLedger.legacy_claimed_rewards.
### Config Changes
- Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize.
### TODO
- [x] Tracker ticket for cleaning up the old code after 84 eras.
- [x] Add companion.
- [x] Redo benchmarks before merge.
- [x] Add Changelog for pallet_staking.
- [x] Pallet should be configurable to enable/disable paged rewards.
- [x] Commission payouts are distributed across pages.
- [x] Review documentation thoroughly.
- [x] Rename `MaxNominatorRewardedPerValidator` ->
`MaxExposurePageSize`.
- [x] NMap for `ErasStakersPaged`.
- [x] Deprecate ErasStakers.
- [x] Integrity tests.
### Followup issues
[Runtime api for deprecated ErasStakers storage
item](https://github.com/paritytech/polkadot-sdk/issues/426)
---------
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: command-bot <>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* reduce exec time of fast-unstake benchmarks
* fix test
* fmt
* fix patch the tests
* fix patch the tests
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* add batch size as well
* update some benches to be better
* fix one last test
* fix
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* reduce time even more
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* fix tests
* nit
* remove
* improve the weight calc further
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* fix benchmarks
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* update
* fix
* fix
* fmt
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* lots of changes again...
* smaller input
* update
* fmt
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* cleanup
* small simplification
* fmt
* reduce exec time a bit
* fix
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* test
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* increase again
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* review comments
* fmt
* fix
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
Co-authored-by: command-bot <>
* frame-system: explicit call index
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Use explicit call indices
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* pallet-template: explicit call index
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* DNM: Temporarily require call_index
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "DNM: Temporarily require call_index"
This reverts commit c4934e312e12af72ca05a8029d7da753a9c99346.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* implement a brand new batch with all tests passing.
* fix benchmarks as well
* make benchmarks more or less work
* fix migration
* add some testing
* Update frame/fast-unstake/src/benchmarking.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* review comments
* some fixes
* fix review comments
* fix build
* fmt
* fix benchmarks
* fmt
* update
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* [Enhancement] Remove optional Pool subscription from fast-unstake
* remove nomination-pools pallet dependency
* fixes
* more fixes
* more fixes
* more fixes
* add failing test for itamar
* an ugly example of fast unstake
* Revert "add failing test for itamar"
This reverts commit 16c4d8015698a0684c090c54fce8b470a2d2feb2.
* fast unstake wip
* clean it up a bit
* some comments
* on_idle logic
* fix
* comment
* new working version, checks all pass, looking good
* some notes
* add mock boilerplate
* more boilerplate
* simplify the weight stuff
* ExtBuilder for pools
* fmt
* rm bags-list, simplify setup_works
* mock + tests boilerplate
* make some benchmarks work
* mock boilerplate
* tests boilerplate
* run_to_block works
* add Error enums
* add test
* note
* make UnstakeRequest fields pub
* some tests
* fix origin
* fmt
* add fast_unstake_events_since_last_call
* text
* rewrite some benchmes and fix them -- the outcome is still strange
* Fix weights
* cleanup
* Update frame/election-provider-support/solution-type/src/single_page.rs
* fix build
* Fix pools tests
* iterate teset + mock
* test unfinished
* cleanup and add some tests
* add test successful_multi_queue
* comment
* rm Head check
* add TODO
* complete successful_multi_queue
* + test early_exit
* fix a lot of things above the beautiful atlantic ocean 🌊
* seemingly it is finished now
* Fix build
* ".git/.scripts/fmt.sh" 1
* Fix slashing amount as well
* better docs
* abstract types
* rm use
* import
* Update frame/nomination-pools/benchmarking/src/lib.rs
Co-authored-by: Nitwit <47109040+nitwit69@users.noreply.github.com>
* Update frame/fast-unstake/src/types.rs
Co-authored-by: Nitwit <47109040+nitwit69@users.noreply.github.com>
* Fix build
* fmt
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
* make bounded
* feedback from code review with Ankan
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/lib.rs
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
* Update frame/fast-unstake/src/mock.rs
* update to master
* some final review comments
* fmt
* fix clippy
* remove unused
* ".git/.scripts/fmt.sh" 1
* make it all build again
* fmt
* undo fishy change
Co-authored-by: Ross Bulat <ross@jkrbinvestments.com>
Co-authored-by: command-bot <>
Co-authored-by: Nitwit <47109040+nitwit69@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
Ankan