Nomination Pools (#10694)

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add admin roles and make some calls permissionless

* Destroy pool in withdraw unbonded

* Add docs on pool admin

* Fixup tests

* Test unbond_other permissionless scenarios

* Test withdraw unbonded permissionless

* Test only can join open pools

* Move unsafe set state to mock

* Test: nominate_works

* Add bounds: MinJoinBond, MinCreateBond, MaxPools

* Test MinCreateBond, MinJoinBond, MaxPools

* Add post checks to tests

* Remove some TODOs

* Setup weight infrastructure

* Benchmark claim_payout

* Benchmark create

* Benchmark nominate

* Benchmark join

* Benchmark unbond_other

* Refactor join benchmark to use scenario setup

* Clean up and address warnings

* Basic withdraw unbonded benchmarks

* Refactor nominate benchmark

* Refactor claim payout

* Add feature sp-staking/runtime-benchmarks

* Get node runtime to compile

* Get node to run

* Make claim_payout bench work with node

* Make pool_withdraw_unbonded bench work with node

* Make withdraw_unbonded_other work with node runtime'

* Make create benchmark work with node

* Make nominate benchmark work with node runtime

* WiP new benchmark crate

* Implement initial mock for benchmarks

* Establish benchmark setup logic

* Get claim payout and nominate benchmarks working

* Remove pool bench utils; make struct fields pub insteaad

* Get more benchmarks to work; trim interface trait

* Some more top level docs

* Finish tranistion benchmarks to crate

* Hook up benchmark pallet to node runtime

* Get benches to work with node runtime

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_nomination_pools --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/nomination-pools/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Benchmark withdraw_unbonded_other_kill

* Delete old benchmarking files

* Refunds for withdraw_unbonded

* Remove some TODOs

* 'Don't return an option for the current_era'

* Streamline extrinsic docs

* small docs tweaks

* Refactor ledger::slash

* Add on_slash impl for nomination pools

* slash refactor wip

* WIP slash working

* DRY Ledger::stash

* Fix slash saturation

* Remove unused param from slash

* Docs and warnings

* Test ledger::slash

* save progress

* Introduce counter for delegators

* Add tests for max delegator errors

* Reproducible account ids

* Adapt tests to new account id format

* Simplify create_accounts api

* Fix staking tests

* Save PerBill slash impl before removing

* Rever ledger slash test

* Get node runtime to work

* Organize sub pools by unbond era, not curren era

* staking: Proportional ledger slashing

* Some comment cleanup

* Add more test post checks

* Update frame/staking/src/pallet/mod.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Tests: account for storing unbond era

* Improve docs for staking interface

* Add events Created, Destroyed

* withdraw_unbonded: Remove useless withdraw dust check

* Test: withdraw_unbonded_other_handles_faulty_sub_pool_accounting

* Add extrinsics: set_state_other, set_metadata

* Test: set_state_other_works

* Test: set_metadata_works

* Add benchmarks for set_state_other, set_metadata

* Fix benchmarks

* Add weight info for new extrinsics

* Some feedback

* duo feedback

* Incorporate some more feedback

* integrate more kian feedback

* integrate more kian feedback

* More improvements

* Add destroying_mul

* Make do_reward_payout take refs

* Remove some TODOs

* Add test for saturating

* feedback

* Fix join test

* use `inner` for nested types in nomination pools (#11030)

* Use nested inner type for pool

* make tests and benchmarks work

* remove feat

* all tests work now

* fix node-runtime

* nomination-pools: update benches for new account format (#11033)

* Update benches to new account format

* More sensible seeds

* bring back rward account sanity check

* Comment

* Add extrinsic set_configs (#11038)

* Better sanity checks for nomination pools  (#11042)

* new sanity checks, few other changes

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* nomination-pools: Fix state event regression + benches (#11045)

* new sanity checks, few other changes

* Fix benches, improve sanity check

* Remove useless clear storage in benchmarking

* Set state

* Save

* Doc

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: kianenigma <kian@parity.io>

* FMT

* Try fill in all staking configs

* Fix build

* More changes to nomination pools (#11050)

* new sanity checks, few other changes

* some last touches as a whole

* Apply suggestions from code review

* Remove redundant event

* Improve unbond_other error handling

* Remove comment

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>

* Remove sanity module and some TODOs

* round of feedback and imp from kian

* Add TODO for ED QoL at reward pool creation

* Make sure reward pool never gets dusted

* Improve error type

* demonstrate per_thing usage

* Update sanity check & fix create_works

* Improve test ext pool creation & fix some more tests

* Try revert

* Revert "Try revert"

This reverts commit c044c94730e1a370eecd8f5b2c4f632835913063.

* Revert "Improve test ext pool creation & fix some more tests"

This reverts commit 1e862a64a7423479260c6e5ad1bd4c8c95651f3a.

* Revert "Update sanity check & fix create_works"

This reverts commit 568a7b727687e4d585e2796afc638df97b83c632.

Roll back reward account funding

* Revert "Improve error type"

This reverts commit 4b993ee601a037e7a44e4a49bbfd60cf45b38b78.

* Revert "Make sure reward pool never gets dusted"

This reverts commit e7a3eb45bdfd156d3f6d94d194e988032ebbc593.

revert

* Update some tests

* FMT

* Test that era offset works correctly

* Update mocks

* Remove unnescary docs

* Doc updates

* Update calculate_delegator_payout_works_with_a_pool_of_1

* Fix test: claim_payout_works

* do_reward_payout_correctly_sets_pool_state_to_destroying

* Remove test do_reward_payout_errors_correctly

* Fix test: do_reward_payout_works

* Fix test: create_errors_correctly

* Fix test: create works

* Fix test: unbond_other_of_3_works

* Ensure that ED is transferred into reward pool upon creation

* WIP pool lifecycle test

* Fix benchmarks

* Add sanity check for ED + reward pools

* `bond_extra` for nomination pools (#11100)

* bond_extra for nomination pools

* Update frame/nomination-pools/src/lib.rs

* Update frame/nomination-pools/src/lib.rs

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* add benchmarks

* remove the min logic of bond_extra

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* FMT

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* make it compile end to end

* Update some type viz

* Update kick terminology

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/nomination-pools/src/lib.rs

* Cache bonded account when creating pool

* Add bond extra weight stuff

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update docs for pool withdraw unbonded

* Update docs for unbond

* Improve Doc

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nomination-pools/Cargo.toml

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Improve Docs

* Some docs improvements

* fmt

* Remove unlock_era

* Fix accidental frame-support regression

* Fix issue with transactions in tests

* Fix doc links

* Make sure result in test is used

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix can toggle state

* Account for new_funds in ok to be open

* Update docs: ok_to_withdraw_unbonded_other_with

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove some staking comments

* Rename SubPoolsWithEra to UnbondingPoolsWithEra

* Use validators length for benchmarks

* Use metadata length for benchmarks

* Remove debug assert eq

* docs

* Fix test: withdraw_unbonded_other_errors_correctly

* Fix check for having enough balance to create the pool

* Bond event for pool creation

* Ok to be open

* FMT

* Remove _other postfix

* Update frame/staking/src/lib.rs

* Adjust tests to account for only remove when < ED

* Remove stale TODOs

* Remove dupe test

* Fix build

* Make sure to convert to u256 so we don't saturate

* Refund depositor with reward pool fee

* FMT

* Remove reachable defensive

* Use compact encoding for relevant extrinsics

* Remove unnescary make_free_be for cleaning reward account

* Add not to maintainers for reward account accounting

* Remove note to maintainers from public doc

* Make sure all configs have currency balance

* Avoid saturation in balance_to_unbond

* Partial Unbonding for Nomination Pools (#11212)

* first draft of partial unbonding for pools

* remove option

* Add some more tests and fix issues

* Fix all tests

* simplify some tests

* Update frame/nomination-pools/src/mock.rs

* remove clone

* rename to delegator_unbonding_eras

* Update frame/nomination-pools/src/tests.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update frame/nomination-pools/src/tests.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update frame/nomination-pools/src/tests.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* remove pub

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* undo

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* leftovers

* fix invariant

* Fix the depositor assumption

* round of self-review

* little bit more cleanup

* Update frame/nomination-pools/src/mock.rs

* Apply suggestions from code review

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Fix interpretation of MinCreateBond

* controvesial refactor

* rename

* make everything build

* add TODO about killing the reward account

* Update frame/nomination-pools/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update frame/nomination-pools/src/lib.rs

* last self-review

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Update Cargo.lock

* Rename Delegator to PoolMember

* fmt

* Get runtime to build with runtime-benchmarks feature

* Update Cargo.lock

* Fix asserts to work in more scenarios

* gte not gt

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_nomination_pools --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/nomination-pools/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update frame/staking/src/mock.rs

* Update frame/nomination-pools/src/lib.rs

* Update frame/staking/src/slashing.rs

* Apply suggestions from code review

* fmt

* Fix some tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

substrate/primitives/staking/src/lib.rs

@@ -19,6 +19,7 @@
 
 //! A crate which contains primitives that are useful for implementation that uses staking
 //! approaches in general. Definitions related to sessions, slashing, etc go here.
+use sp_runtime::{DispatchError, DispatchResult};
 use sp_std::collections::btree_map::BTreeMap;
 
 pub mod offence;
@@ -38,7 +39,7 @@ pub trait OnStakerSlash<AccountId, Balance> {
 	///
 	/// * `stash` - The stash of the staker whom the slash was applied to.
 	/// * `slashed_active` - The new bonded balance of the staker after the slash was applied.
-	/// * `slashed_unlocking` - a map of slashed eras, and the balance of that unlocking chunk after
+	/// * `slashed_unlocking` - A map of slashed eras, and the balance of that unlocking chunk after
 	///   the slash is applied. Any era not present in the map is not affected at all.
 	fn on_slash(
 		stash: &AccountId,
@@ -52,3 +53,80 @@ impl<AccountId, Balance> OnStakerSlash<AccountId, Balance> for () {
 		// Nothing to do here
 	}
 }
+
+/// Trait for communication with the staking pallet.
+pub trait StakingInterface {
+	/// Balance type used by the staking system.
+	type Balance;
+
+	/// AccountId type used by the staking system
+	type AccountId;
+
+	/// The minimum amount required to bond in order to be a nominator. This does not necessarily
+	/// mean the nomination will be counted in an election, but instead just enough to be stored as
+	/// a nominator. In other words, this is the minimum amount to register the intention to
+	/// nominate.
+	fn minimum_bond() -> Self::Balance;
+
+	/// Number of eras that staked funds must remain bonded for.
+	///
+	/// # Note
+	///
+	/// This must be strictly greater than the staking systems slash deffer duration.
+	fn bonding_duration() -> EraIndex;
+
+	/// The current era index.
+	///
+	/// This should be the latest planned era that the staking system knows about.
+	fn current_era() -> EraIndex;
+
+	/// The amount of active stake that `controller` has in the staking system.
+	fn active_stake(controller: &Self::AccountId) -> Option<Self::Balance>;
+
+	/// The total stake that `controller` has in the staking system. This includes the
+	/// [`Self::active_stake`], and any funds currently in the process of unbonding via
+	/// [`Self::unbond`].
+	///
+	/// # Note
+	///
+	/// This is only guaranteed to reflect the amount locked by the staking system. If there are
+	/// non-staking locks on the bonded pair's balance this may not be accurate.
+	fn total_stake(controller: &Self::AccountId) -> Option<Self::Balance>;
+
+	/// Bond (lock) `value` of `stash`'s balance. `controller` will be set as the account
+	/// controlling `stash`. This creates what is referred to as "bonded pair".
+	fn bond(
+		stash: Self::AccountId,
+		controller: Self::AccountId,
+		value: Self::Balance,
+		payee: Self::AccountId,
+	) -> DispatchResult;
+
+	/// Have `controller` nominate `validators`.
+	fn nominate(
+		controller: Self::AccountId,
+		validators: sp_std::vec::Vec<Self::AccountId>,
+	) -> DispatchResult;
+
+	/// Bond some extra amount in the _Stash_'s free balance against the active bonded balance of
+	/// the account. The amount extra actually bonded will never be more than the _Stash_'s free
+	/// balance.
+	fn bond_extra(controller: Self::AccountId, extra: Self::Balance) -> DispatchResult;
+
+	/// Schedule a portion of the active bonded balance to be unlocked at era
+	/// [Self::current_era] + [`Self::bonding_duration`].
+	///
+	/// Once the unlock era has been reached, [`Self::withdraw_unbonded`] can be called to unlock
+	/// the funds.
+	///
+	/// The amount of times this can be successfully called is limited based on how many distinct
+	/// eras funds are schedule to unlock in. Calling [`Self::withdraw_unbonded`] after some unlock
+	/// schedules have reached their unlocking era should allow more calls to this function.
+	fn unbond(controller: Self::AccountId, value: Self::Balance) -> DispatchResult;
+
+	/// Unlock any funds schedule to unlock before or at the current era.
+	fn withdraw_unbonded(
+		controller: Self::AccountId,
+		num_slashing_spans: u32,
+	) -> Result<u64, DispatchError>;
+}