* pallet-mmr: fix some typos
* pallet-mmr: make the MMR resilient to chain forks
* pallet-mmr: get hash for block that added node
* beefy-mmr: add debug logging
* add explanatory comment
* account for block offset of pallet activation
* add support for finding all nodes added by leaf
* minor improvements
* add helper to return all nodes added to mmr with a leaf append
* simplify leaf_node_index_to_leaf_index
summing the (shifted) differences in peak positions adds up to the (shifted) final position, so
don't need to fold over positions.
* dead fish: this also doesn't work
The idea was to keep a rolling window of `(parent_hash, pos)` leaf
entries in the offchain db, with the window matching the one
that provides `block_num -> block_hash` mappings in `frame_system`.
Once a leaf exits the window it would be "canonicalized" by switching
its offchain db key from `(parent_hash, pos)` to simple `pos`.
This doesn't work however because there's no way to get leaf contents
from offchain db while in runtime context.. so no way to get+clear+set
leaf to change its key in offchain db.
Ideas:
1. move the "canonicalization" logic to offchain worker
2. enhance IndexingApi with "offchain::move(old_key, new_key)"
This is weird, but correct, deterministic and safe AFAICT, so
it could be exposed to runtime.
* simplify rightmost_leaf_node_index_from_pos
* minor fix
* move leaf canonicalization to offchain worker
* move storage related code to storage.rs
* on offchain reads use canonic key for old leaves
* fix offchain worker write using canon key
* fix pallet-mmr tests
* add documentation and fix logging
* add offchain mmr canonicalization test
* test canon + generate + verify
* fix pallet-beefy-mmr tests
* implement review suggestions
* improve test
* pallet-mmr: add offchain pruning of forks
* pallet-mmr: improve offchain pruning
Instead of keeping pruning map as single blob in offchain db,
keep individual parent-hash lists with block-num identifier as part
of the offchain key.
Signed-off-by: acatangiu <adrian@parity.io>
* pallet-mmr: improve MMRStore<OffchainStorage>::get()
Do the math and retrieve node using correct (canon or non-canon)
offchain db key, instead of blindly looking in both canon and non-canon
offchain db locations for each node.
Still fallback on looking at both if for any reason it's not where
expected.
Signed-off-by: acatangiu <adrian@parity.io>
* pallet-mmr: storage: improve logs
* fix tests: correctly persist overlay
runtime indexing API works on overlay, whereas offchain context
bypasses overlay, so for loops > canon-window, canon would fail.
* pallet-mmr: fix numeric typo in test
* add comment around LeafData requirements
Signed-off-by: acatangiu <adrian@parity.io>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
* pallet-beefy: add Config::OnNewValidatorSet type
Add a hook to pallet-beefy for doing specific work when
BEEFY validator set changes.
For example, this can be used by pallet-beefy-mmr to cache
a lightweight MMR root over validators and make it available
to light clients.
* pallet-beefy-mmr: implement OnNewValidatorSet
Implement pallet-beefy::OnNewValidatorSet to be notified of BEEFY
validator set changes. Use the notifications to compute and cache
a light weight 'BEEFY authority set' which is an MMR root over
BEEFY validator set plus some extra info.
Previously, pallet-beefy-mmr was interogating pallet-beefy about
validator set id on every block to find out when it needs to recompute
the authority set.
By using the event-driven approach in this commit, we also save one
extra state interogation per block.
* pallet-beefy-mmr: add new authority_set() API
Expose current and next BEEFY authority sets through runtime API.
These can be directly used by light clients to avoid having them
compute them themselves based on BEEFY validator sets.
Signed-off-by: acatangiu <adrian@parity.io>
* rename BeefyMmr exposed runtime api
* Initial implementation of mms
* Some more attempts at `mms`
* Functioning `MMS` algorithm implementation.
Adding some tests too
* More tests and typos fixed.
* Adding fuzzer for `mms`
(but could not test it on Mac M1)
* Missing imports
* Fixing rustdoc
* More accurate implementation of `mms`
* Removing the fuzzer `mms` implementation
* Implementing `NposSolver` for `MMS`
had to add the `Clone` trait, maybe I could see if I can get rid of it.
* Fixing rust docs by adding () to resolve ambiguity
* Amending `unwrap` to `expect`
removing unneeded `Clone` trait
* Removing redundant `mms3.rs`
* Implementing `BalancingConfig` and rustdoc changes
* Implementing `weight` for `MMS`
* Implementing `weight` for `MMS`
* Fixing post merge
* Initial implementation of mms
* Some more attempts at `mms`
* Functioning `MMS` algorithm implementation.
Adding some tests too
* More tests and typos fixed.
* Adding fuzzer for `mms`
(but could not test it on Mac M1)
* Missing imports
* Fixing rustdoc
* More accurate implementation of `mms`
* Removing the fuzzer `mms` implementation
* Implementing `NposSolver` for `MMS`
had to add the `Clone` trait, maybe I could see if I can get rid of it.
* Amending `unwrap` to `expect`
removing unneeded `Clone` trait
* Fixing rust docs by adding () to resolve ambiguity
* Removing redundant `mms3.rs`
* Implementing `BalancingConfig` and rustdoc changes
* Implementing `weight` for `MMS`
* Implementing `weight` for `MMS`
* Fixing post merge
* Removing left over from rebase
* Fixing tests
* Removing unneeded import
* Removing unneeded functions
* Removing useless imports
Co-authored-by: kianenigma <kian@parity.io>
* grandpa: fix creation of justification ancestry
we would reject commits that have precommits targeting blocks lower than the
commit target. when there is an equivocation (or if it the commit is not
minimal) it is possible to have such precommits and we should assume that they
are the round base.
* grandpa: bump to 0.16.0
* grandpa: add test for justification with equivocation
* grandpa: fix failing test
* make pool roles optional
* undo lock file changes?
* add migration
* add the ability for pools to chill themselves
* boilerplate of tests
* somewhat stable, but I think I found another bug as well
* Fix it all
* Add more more sophisticated test + capture one more bug.
* Update frame/staking/src/lib.rs
* reduce the diff a little bit
* add some test for the slashing bug
* cleanup
* fix lock file?
* Fix
* fmt
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/mock.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix build
* fix some fishy tests..
* add one last integrity check for MinCreateBond
* remove bad assertion -- needs to be dealt with later
* nits
* fix tests and add benchmarks for chill
* remove stuff
* fix benchmarks
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_nomination_pools --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/nomination-pools/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* remove defensive
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Bot <admin@parity.io>
* Implement MaxEncodedLen on pallet-beefy
* Return Result in intialize_authorities
* Update docs
* Log error when authorities list gets truncated
* Update frame/beefy/src/lib.rs
Co-authored-by: Adrian Catangiu <adrian@parity.io>
* cargo fmt
Co-authored-by: Adrian Catangiu <adrian@parity.io>
* trie: Optimize `keys` function
Instead of iterating the entire state and collecting all keys that match the given prefix, we can
directly use the optimized prefix iterator.
* Add a test
* fix a few more things with nomination pools
* add bench
* use weight fn
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_nomination_pools --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/nomination-pools/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* allow real root to also set roles
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* move out of the closure
* fix a few more things
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* only check std
* add default-features=false
* Revert "add default-features=false"
This reverts commit bc9453757a1df670f418a2d57ee4ce203425ab1b.
* missing features only to non-optional
* updated mmr rpc api with functions for batch generation of proof
* update code comments
* fix build errors
* added tests to mmr-rpc
* add tests to pallet-mmr
* update comments
* minor comment fix
* remove unused variables
* fix rust doc errors
* refactor mmr runtime api
* fix tests
* minor fix
* minor fix
* fix node-runtime
* revert to initial api
* impl from proof fot batchproof
* minor fix
* minor fix
* use explicit functions to convert btw batch proof and single proof
* minor fix
* add new variant to mmr error
* fmt
* update conversion to single leaf proof
* fix style nit
Co-authored-by: Adrian Catangiu <adrian@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Add admin roles and make some calls permissionless
* Destroy pool in withdraw unbonded
* Add docs on pool admin
* Fixup tests
* Test unbond_other permissionless scenarios
* Test withdraw unbonded permissionless
* Test only can join open pools
* Move unsafe set state to mock
* Test: nominate_works
* Add bounds: MinJoinBond, MinCreateBond, MaxPools
* Test MinCreateBond, MinJoinBond, MaxPools
* Add post checks to tests
* Remove some TODOs
* Setup weight infrastructure
* Benchmark claim_payout
* Benchmark create
* Benchmark nominate
* Benchmark join
* Benchmark unbond_other
* Refactor join benchmark to use scenario setup
* Clean up and address warnings
* Basic withdraw unbonded benchmarks
* Refactor nominate benchmark
* Refactor claim payout
* Add feature sp-staking/runtime-benchmarks
* Get node runtime to compile
* Get node to run
* Make claim_payout bench work with node
* Make pool_withdraw_unbonded bench work with node
* Make withdraw_unbonded_other work with node runtime'
* Make create benchmark work with node
* Make nominate benchmark work with node runtime
* WiP new benchmark crate
* Implement initial mock for benchmarks
* Establish benchmark setup logic
* Get claim payout and nominate benchmarks working
* Remove pool bench utils; make struct fields pub insteaad
* Get more benchmarks to work; trim interface trait
* Some more top level docs
* Finish tranistion benchmarks to crate
* Hook up benchmark pallet to node runtime
* Get benches to work with node runtime
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_nomination_pools --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/nomination-pools/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Benchmark withdraw_unbonded_other_kill
* Delete old benchmarking files
* Refunds for withdraw_unbonded
* Remove some TODOs
* 'Don't return an option for the current_era'
* Streamline extrinsic docs
* small docs tweaks
* Refactor ledger::slash
* Add on_slash impl for nomination pools
* slash refactor wip
* WIP slash working
* DRY Ledger::stash
* Fix slash saturation
* Remove unused param from slash
* Docs and warnings
* Test ledger::slash
* save progress
* Introduce counter for delegators
* Add tests for max delegator errors
* Reproducible account ids
* Adapt tests to new account id format
* Simplify create_accounts api
* Fix staking tests
* Save PerBill slash impl before removing
* Rever ledger slash test
* Get node runtime to work
* Organize sub pools by unbond era, not curren era
* staking: Proportional ledger slashing
* Some comment cleanup
* Add more test post checks
* Update frame/staking/src/pallet/mod.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Tests: account for storing unbond era
* Improve docs for staking interface
* Add events Created, Destroyed
* withdraw_unbonded: Remove useless withdraw dust check
* Test: withdraw_unbonded_other_handles_faulty_sub_pool_accounting
* Add extrinsics: set_state_other, set_metadata
* Test: set_state_other_works
* Test: set_metadata_works
* Add benchmarks for set_state_other, set_metadata
* Fix benchmarks
* Add weight info for new extrinsics
* Some feedback
* duo feedback
* Incorporate some more feedback
* integrate more kian feedback
* integrate more kian feedback
* More improvements
* Add destroying_mul
* Make do_reward_payout take refs
* Remove some TODOs
* Add test for saturating
* feedback
* Fix join test
* use `inner` for nested types in nomination pools (#11030)
* Use nested inner type for pool
* make tests and benchmarks work
* remove feat
* all tests work now
* fix node-runtime
* nomination-pools: update benches for new account format (#11033)
* Update benches to new account format
* More sensible seeds
* bring back rward account sanity check
* Comment
* Add extrinsic set_configs (#11038)
* Better sanity checks for nomination pools (#11042)
* new sanity checks, few other changes
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* nomination-pools: Fix state event regression + benches (#11045)
* new sanity checks, few other changes
* Fix benches, improve sanity check
* Remove useless clear storage in benchmarking
* Set state
* Save
* Doc
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: kianenigma <kian@parity.io>
* FMT
* Try fill in all staking configs
* Fix build
* More changes to nomination pools (#11050)
* new sanity checks, few other changes
* some last touches as a whole
* Apply suggestions from code review
* Remove redundant event
* Improve unbond_other error handling
* Remove comment
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
* Remove sanity module and some TODOs
* round of feedback and imp from kian
* Add TODO for ED QoL at reward pool creation
* Make sure reward pool never gets dusted
* Improve error type
* demonstrate per_thing usage
* Update sanity check & fix create_works
* Improve test ext pool creation & fix some more tests
* Try revert
* Revert "Try revert"
This reverts commit c044c94730e1a370eecd8f5b2c4f632835913063.
* Revert "Improve test ext pool creation & fix some more tests"
This reverts commit 1e862a64a7423479260c6e5ad1bd4c8c95651f3a.
* Revert "Update sanity check & fix create_works"
This reverts commit 568a7b727687e4d585e2796afc638df97b83c632.
Roll back reward account funding
* Revert "Improve error type"
This reverts commit 4b993ee601a037e7a44e4a49bbfd60cf45b38b78.
* Revert "Make sure reward pool never gets dusted"
This reverts commit e7a3eb45bdfd156d3f6d94d194e988032ebbc593.
revert
* Update some tests
* FMT
* Test that era offset works correctly
* Update mocks
* Remove unnescary docs
* Doc updates
* Update calculate_delegator_payout_works_with_a_pool_of_1
* Fix test: claim_payout_works
* do_reward_payout_correctly_sets_pool_state_to_destroying
* Remove test do_reward_payout_errors_correctly
* Fix test: do_reward_payout_works
* Fix test: create_errors_correctly
* Fix test: create works
* Fix test: unbond_other_of_3_works
* Ensure that ED is transferred into reward pool upon creation
* WIP pool lifecycle test
* Fix benchmarks
* Add sanity check for ED + reward pools
* `bond_extra` for nomination pools (#11100)
* bond_extra for nomination pools
* Update frame/nomination-pools/src/lib.rs
* Update frame/nomination-pools/src/lib.rs
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* add benchmarks
* remove the min logic of bond_extra
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* FMT
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* make it compile end to end
* Update some type viz
* Update kick terminology
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/nomination-pools/src/lib.rs
* Cache bonded account when creating pool
* Add bond extra weight stuff
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update docs for pool withdraw unbonded
* Update docs for unbond
* Improve Doc
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/Cargo.toml
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Improve Docs
* Some docs improvements
* fmt
* Remove unlock_era
* Fix accidental frame-support regression
* Fix issue with transactions in tests
* Fix doc links
* Make sure result in test is used
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix can toggle state
* Account for new_funds in ok to be open
* Update docs: ok_to_withdraw_unbonded_other_with
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Remove some staking comments
* Rename SubPoolsWithEra to UnbondingPoolsWithEra
* Use validators length for benchmarks
* Use metadata length for benchmarks
* Remove debug assert eq
* docs
* Fix test: withdraw_unbonded_other_errors_correctly
* Fix check for having enough balance to create the pool
* Bond event for pool creation
* Ok to be open
* FMT
* Remove _other postfix
* Update frame/staking/src/lib.rs
* Adjust tests to account for only remove when < ED
* Remove stale TODOs
* Remove dupe test
* Fix build
* Make sure to convert to u256 so we don't saturate
* Refund depositor with reward pool fee
* FMT
* Remove reachable defensive
* Use compact encoding for relevant extrinsics
* Remove unnescary make_free_be for cleaning reward account
* Add not to maintainers for reward account accounting
* Remove note to maintainers from public doc
* Make sure all configs have currency balance
* Avoid saturation in balance_to_unbond
* Partial Unbonding for Nomination Pools (#11212)
* first draft of partial unbonding for pools
* remove option
* Add some more tests and fix issues
* Fix all tests
* simplify some tests
* Update frame/nomination-pools/src/mock.rs
* remove clone
* rename to delegator_unbonding_eras
* Update frame/nomination-pools/src/tests.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update frame/nomination-pools/src/tests.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update frame/nomination-pools/src/tests.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* remove pub
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* undo
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* leftovers
* fix invariant
* Fix the depositor assumption
* round of self-review
* little bit more cleanup
* Update frame/nomination-pools/src/mock.rs
* Apply suggestions from code review
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Fix interpretation of MinCreateBond
* controvesial refactor
* rename
* make everything build
* add TODO about killing the reward account
* Update frame/nomination-pools/src/lib.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update frame/nomination-pools/src/lib.rs
* last self-review
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
* Update Cargo.lock
* Rename Delegator to PoolMember
* fmt
* Get runtime to build with runtime-benchmarks feature
* Update Cargo.lock
* Fix asserts to work in more scenarios
* gte not gt
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_nomination_pools --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/nomination-pools/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Update frame/staking/src/mock.rs
* Update frame/nomination-pools/src/lib.rs
* Update frame/staking/src/slashing.rs
* Apply suggestions from code review
* fmt
* Fix some tests
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Adrian Catangiu