Combination of paritytech/polkadot#7005, its addon PR
paritytech/polkadot#7585 and its companion paritytech/cumulus#2433.
This PR introduces a new XcmFeesToAccount struct which implements the
`FeeManager` trait, and assigns this struct as the `FeeManager` in the
XCM config for all runtimes.
The struct simply deposits all fees handled by the XCM executor to a
specified account. In all runtimes, the specified account is configured
as the treasury account.
XCM __delivery__ fees are now being introduced (unless the root origin
is sending a message to a system parachain on behalf of the originating
chain).
# Note for reviewers
Most file changes are tests that had to be modified to account for the
new fees.
Main changes are in:
- cumulus/pallets/xcmp-queue/src/lib.rs <- To make it track the delivery
fees exponential factor
- polkadot/xcm/xcm-builder/src/fee_handling.rs <- Added. Has the
FeeManager implementation
- All runtime xcm_config files <- To add the FeeManager to the XCM
configuration
# Important note
After this change, instructions that create and send a new XCM (Query*,
Report*, ExportMessage, InitiateReserveWithdraw, InitiateTeleport,
DepositReserveAsset, TransferReserveAsset, LockAsset and RequestUnlock)
will require the corresponding origin account in the origin register to
pay for transport delivery fees, and the onward message will fail to be
sent if the origin account does not have the required amount. This
delivery fee is on top of what we already collect as tx fees in
pallet-xcm and XCM BuyExecution fees!
Wallet UIs that want to expose the new delivery fee can do so using the
formula:
```
delivery_fee_factor * (base_fee + encoded_msg_len * per_byte_fee)
```
where the delivery fee factor can be obtained from the corresponding
pallet based on which transport you are using (UMP, HRMP or bridges),
the base fee is a constant, the encoded message length from the message
itself and the per byte fee is the same as the configured per byte fee
for txs (i.e. `TransactionByteFee`).
---------
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Giles Cope <gilescope@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
- Async-backing related primitives are stable `primitives::v6`
- Async-backing API is now part of `api_version(7)`
- It's enabled on Rococo and Westend runtimes
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* Rename `polkadot-parachain` to `polkadot-parachain-primitives`
While doing this it also fixes some last `rustdoc` issues and fixes
another Cargo warning related to `pallet-paged-list`.
* Fix compilation
* ".git/.scripts/commands/fmt/fmt.sh"
* Fix XCM docs
---------
Co-authored-by: command-bot <>
* Rename squatted crates
This commit adds the staging- prefix to squatted crates so we can go forward and publish them to crates.io.
Using the staging- prefix is a temp fix until we decide on replacement names.
https://forum.parity.io/t/renaming-squated-crates-in-substrate-polkadot-cumulus/1964/6
* Fix test after crate renames
* Update Lockfile
* First baby steps
* Split scheduler into several modules
* Towards a more modular approach for scheduling
* move free_cores; IntoInterator -> BTreeMap
* Move clear()
* Move more functions out of scheduler
* Change weight composition
* More abstraction
* Further refactor
* clippy
* fmt
* fix test-runtime
* Add parathreads pallet to construct_runtime!
* Make all runtimes use (Parachains, Parathreads) scheduling
* Delete commented out code
* Remove parathreads scheduler from westend, rococo, and kusama
* fix rococo, westend, and kusama config
* Revert "fix rococo, westend, and kusama config"
This reverts commit 59e4de380d5c7d17eaaba5e2c2b81405de3465e3.
* Revert "Remove parathreads scheduler from westend, rococo, and kusama"
This reverts commit 4c44255296083ac5670560790ed77104917890a4.
* Remove CoreIndex from free_cores
* Remove unnecessary struct for parathreads
* parathreads provider take 1
* Comment out parathread tests
* Pop into lookahead
* fmt
* Fill lookahead with two entries for parachains
* fmt
* Current stage
* Towards ab parathreads
* no AB use
* Make tests typecheck
* quick hack to set scheduling lookahead to 1
* Fix scheduler tests
* fix paras_inherent tests
* misc
* Update more of a test
* cfg(test)
* some cleanup
* Undo paras_inherent changes
* Adjust paras inherent tests
* Undo changes to v2 primitives
* Undo v2 mod changes to tests
* minor
* Remove parathreads assigner and pallet
* minor
* minor
* more cleanup
* fmt
* minor
* minor
* minor
* Remove on_new_session from assignment provider
* Make adder collator integration test pass
* disable failing unit tests
* minor
* minor
* re-enable one unit test
* minor
* handle retries, add concluded para to pop interface
* comment out unused code
* Remove core_para from interface
* Remove first claimqueue element on clear if None instead removing all Nones
* Move claimqueue get out of loop
* Use VecDeque instead of Ved in ClaimQueue
* Make occupied() AB ready(?)
* handle freed disputed in clear_and_fill_claimqueue
* clear_and_fill_claimqueue returns scheduled Vec
* Rename and minor refactor
* return position of assignment taken from claimqueue
* minor
* Fix session boundary parachains number change + extended test
* Fix runtimes
* Fix polkadot runtime
* Remove polkadot pallet from benchmarks
* fix test runtime
* Add storage migration
* Minor refactor
* Minor
* migratin typechecks
* Add migration to runtimes
* Towards modular scheduling II (#6568)
* Add post migration check
* pebkac
* Disable migrations but mine
* Revert "Disable migrations but mine"
This reverts commit 4fa5c5a370c199944a7e0926f50b08626bfbad4c.
* Move scheduler migration
* Revert "Move scheduler migration"
This reverts commit a16b1659a907950bae048a9f7010f2aa76e02b6d.
* Fix migration
* cleanup
* Don't lose retries value anymore
* comment out test function
* Remove retries value from Assignment again
* minor
* Make collator for parathreads optional
* data type refactor
* update scheduler tests
* Change test function cfg
* comment out test function
* Try cfg(test) only
* fix cfg flags
* Add get_max_retries function to provider interface (#7047)
* Fix merge commit
* pebkac
* fix merge
* update cargo.lock
* fix merge
* fix merge
* Use btreemap instead of vec, fix scheduler calls.
* Use imported `ScheduledCore`
* Remove unused import in inclusion tests
* Use keys() instead of mapping over a BTreeMap
* Fix migrations for parachains scheduler
* Use BlockNumberFor<T> everywhere in scheduler
* Add on demand assignment provider pallet (#7110)
* Address some PR comments
* minor
* more cleanup
* find_map and timeout availability fixes
* Change default scheduling_lookahead to 1
* Add on demand assignment provider pallet
* Move test-runtime to new assignment provider
* Run cargo format on scheduler tests
* minor
* Mutate cores in single loop
* timeout predicate simplification
* claimqueue desired size fix
* Replace expect by ok_or
* More improvements
* Fix push back order and next_up_on_timeout
* minor
* session change docs
* Add pre_new_session call to hand pre session updates
* Remove sc_network dependency and PeerId from unnecessary data structures
* Remove unnecessary peer_ids
* Add OnDemandOrdering proxy (#7156)
* Add OnDemandBidding proxy
* Fix names
* OnDemandAssigner for rococo only
* Check PeerId in collator protocol before fetching collation
* On occupied, remove non occupied cores from the claimqueue front and refill
* Add missing docs
* Comment out unused field
* fix ScheduledCore in tests
* Fix the fix
* pebkac
* fmt
* Fix occupied dropping
* Remove double import
* ScheduledCore fixes
* Readd sc-network dep
* pebkac
* OpaquePeerId -> PeerId in can_collate interface
* Cargo.lock update for interface change
* Remove checks not needed anymore?
* Drop occupied core on session change if it would time out after the new session
* Add on demand assignment provider pallet
* Move test-runtime to new assignment provider
* Run cargo format on scheduler tests
* Add OnDemandOrdering proxy (#7156)
* Add OnDemandBidding proxy
* Fix names
* OnDemandAssigner for rococo only
* Remove unneeded config values
* Update comments
* Use and_then for queue position
* Return the max size of the spot queue on error
* Add comments to add_parathread_entry
* Add module comments
* Add log for when can_collate fails
* Change assigner queue type to `Assignment`
* Update assignment provider tests
* More logs
* Remove unused keyring import
* disable can_collate
* comment out can_collate
* Can collate first checks set if empty
* Move can_collate call to collation advertisement
* Fix backing test
* map to loop
* Remove obsolete check
* Move invalid collation test from backing to collator-protocol
* fix unused imports
* fix test
* fix Debug derivation
* Increase time limit on zombienet predicates
* Increase zombienet timeout
* Minor
* Address some PR comments
* Address PR comments
* Comment out failing assert due to on-demand assigner missing
* remove collator_restrictions info from backing
* Move can_collate to ActiveParas
* minor
* minor
* Update weight information for on demand config
* Add ttl to parasentry
* Fix tests missing parasentry ttl
* Adjust scheduler tests to use ttl default values
* Use match instead of if let for ttl drop
* Use RuntimeDebug trait for `ParasEntry` fields
* Add comments to on demand assignment pallet
* Fix spot traffic calculation
* Revert runtimedebug changes to primitives
* Remove runtimedebug derivation from `ParasEntry`
* Mention affinity in pallet level docs
* Use RuntimeDebug trait for ParasEntry child types
* Remove collator restrictions
* Fix primitive versioning and other merge issues
* Fix tests post merge
* Fix node side tests
* Edit parascheduler migration for clarity
* Move parascheduler migration up to next release
* Remove vestiges from merge
* Fix tests
* Refactor ttl handling
* Remove unused things from scheduler tests
* Move on demand assigner to own directory
* Update documentation
* Remove unused sc-network dependency in primitives
Was used for collator restrictions
* Remove unused import
* Reenable scheduler test
* Remove unused storage value
* Enable timeout predicate test and fix fn
Turns out that the issue with the compiler is fixed and we can now
use impl Trait in the manner used here.
* Remove unused imports
* Add benchmarking entry for perbill in config
* Correct typo
* Address review comments
* Log out errors when calculating spot traffic.
* Change parascheduler's log target name
* Update scheduler_common documentation
* Use mutate for affinity fns, add tests
* Add another on demand affinity test
* Unify parathreads and parachains in HostConfig (take 2) (#7452)
* Unify parathreads and parachains in HostConfig
* Fixed missed occurences
* Remove commented out lines
* `HostConfiguration v7`
* Fix version check
* Add `MigrateToV7` to `Unreleased`
* fmt
* fmt
* Fix compilation errors after the rebase
* Update runtime/parachains/src/scheduler/tests.rs
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
* Update runtime/parachains/src/scheduler/tests.rs
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
* fmt
* Fix migration test
* Fix tests
* Remove unneeded assert from tests
* parathread_cores -> on_demand_cores; parathread_retries -> on_demand_retries
* Fix a compilation error in tests
* Remove unused `use`
* update colander image version
---------
Co-authored-by: alexgparity <alex.gremm@parity.io>
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
Co-authored-by: Javier Viola <javier@parity.io>
* Fix branch after merge with master
* Refactor out duplicate checks into a helper fn
* Fix tests post merge
* Rename add_parathread_assignment, add test
* Update docs
* Remove unused on_finalize function
* Add weight info to on demand pallet
* Update runtime/parachains/src/configuration.rs
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
* Update runtime/parachains/src/scheduler_common/mod.rs
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
* Update runtime/parachains/src/assigner_on_demand/mod.rs
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
* Add benchmarking to on demand pallet
* Make place_order test check for success
* Add on demand benchmarks
* Add local test weights to rococo runtime
* Modify TTL drop behaviour to not skip claims
Previous behaviour would jump a new claim from the assignment provider
ahead in the claimqueue, assuming lookahead is larger than 1.
* Refactor ttl test to test claimqueue order
* Disable place_order ext. when no on_demand cores
* Use default genesis config for benchmark tests
* Refactor config builder param
* Move lifecycle test from scheduler to on demand
* Remove unneeded lifecycle test
Paras module via the parachain assignment provider doesn't provide
new assignments if a parachain loses it's lease. The on demand
assignment provider doesn't provide an assignment that is not a
parathread.
* Re enable validator shuffle test
* More realistic weights for place_order
* Remove redundant import
* Fix backwards compatibility (hopefully)
* ".git/.scripts/commands/bench/bench.sh" --subcommand=runtime --runtime=rococo --target_dir=polkadot --pallet=runtime_parachains::assigner_on_demand
* Fix tests.
* Fix off-by-one.
* Re enable claimqueue fills test
* Re enable schedule_rotates_groups test
* Fix fill_claimqueue_fills test
* Re enable next_up_on_timeout test, move fn
* Do not pop from assignment provider when retrying
* Fix tests missing collator in scheduledcore
* Add comment about timeout predicate.
* Rename parasentry retries to availability timeouts
* Re enable schedule_schedules... test
* Refactor prune retried test to new scheduler
* Have all scheduler tests use genesis_cfg fn
* Update docs
* Update copyright notices on new files
* Rename is_parachain_core to is_bulk_core
* Remove erroneous TODO
* Simplify import
* ".git/.scripts/commands/bench/bench.sh" --subcommand=runtime --runtime=rococo --target_dir=polkadot --pallet=runtime_parachains::configuration
* Revert AdvertiseCollation order shuffle
* Refactor place_order into keepalive and allowdeath
* Revert rename of hrmp max inbound channels
parachain encompasses both on demand and slot auction / bulk.
* Restore availability_timeout_predicate function
* Clean up leftover comments
* Update runtime/parachains/src/scheduler/tests.rs
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
* ".git/.scripts/commands/bench/bench.sh" --subcommand=runtime --runtime=westend --target_dir=polkadot --pallet=runtime_parachains::configuration
---------
Co-authored-by: alexgparity <alex.gremm@parity.io>
Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: command-bot <>
* On Demand - update weights and small nits (#7605)
* Remove collator restriction test in inclusion
On demand parachains won't have collator restrictions implemented in
this way but will instead use a preferred collator registered to a
`ParaId` in `paras_registrar`.
* Remove redundant config guard for test fns
* Update weights
* Update WeightInfo for on_demand assigner
* Unify assignment provider parameters into one call (#7606)
* Combine assignmentprovider params into one fn call
* Move scheduler_common to a module under scheduler
* Fix ttl handling in benchmark builder
* Run cargo format
* Remove obsolete test.
* Small improvement.
* Use same migration pattern as config module
* Remove old TODO
* Change log target name for assigner on demand
* Fix migration
* Fix clippy warnings
* Add HostConfiguration storage migration to V8
* Add `MigrateToV8` to unreleased migrations for all runtimes
* Fix storage version check for config v8
* Set `StorageVersion` to 8 in `MigrateToV8`
* Remove dups.
* Update primitives/src/v5/mod.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
---------
Co-authored-by: alexgparity <alex.gremm@parity.io>
Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com>
Co-authored-by: antonva <anton.asgeirsson@parity.io>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
* Companion for substrate#14471
* add missing MaxNominators constant
* missing MaxNominators in test-runtime
* missing MaxNominators in runtime/integration_tests
* Dont use deprecated functions
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fixup
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* update lockfile for {"substrate"}
* Update Substrate dep
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
* Move vstaging to production (and thus past session slashing).
WIP: test-runtime still needs to be fixed.
* Fix test-runtime.
---------
Co-authored-by: eskimor <eskimor@no-such-url.com>
* Move XCM query functionality to trait
* Fix tests
* Add PayOverXcm implementation
* fix the PayOverXcm trait to compile
* moved doc comment out of trait implmeentation and to the trait
* PayOverXCM documentation
* Change documentation a bit
* Added empty benchmark methods implementation and changed docs
* update PayOverXCM to convert AccountIds to MultiLocations
* Implement benchmarking method
* Change v3 to latest
* Descend origin to an asset sender (#6970)
* descend origin to an asset sender
* sender as tuple of dest and sender
* Add more variants to the QueryResponseStatus enum
* Change Beneficiary to Into<[u8; 32]>
* update PayOverXcm to return concrete errors and use AccountId as sender
* use polkadot-primitives for AccountId
* fix dependency to use polkadot-core-primitives
* force Unpaid instruction to the top of the instructions list
* modify report_outcome to accept interior argument
* use new_query directly for building final xcm query, instead of report_outcome
* fix usage of new_query to use the XcmQueryHandler
* fix usage of new_query to use the XcmQueryHandler
* tiny method calling fix
* xcm query handler (#7198)
* drop redundant query status
* rename ReportQueryStatus to OuterQueryStatus
* revert rename of QueryResponseStatus
* update mapping
* Update xcm/xcm-builder/src/pay.rs
Co-authored-by: Gavin Wood <gavin@parity.io>
* Updates
* Docs
* Fix benchmarking stuff
* Destination can be determined based on asset_kind
* Tweaking API to minimise clones
* Some repotting and docs
---------
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
* Properly set the pricing for the DMP router
* Publicize price types
* Use FixedU128 instead of Percent
* Add sp-arithmetic as a dependency for rococo runtime
* Add sp-arithmetic as a dependency to all runtimes
* Remove duplicate import
* Add missing import
* Fix tests
* Create an appropriate QueueDownwardMessageError variant
* Recalculate delivery fee factor based on past queue sizes
* Remove unused error variant
* Fixes
* Fixes
* Remove unused imports
* Rewrite fee factor update mechanism
* Remove unused imports
* Fixes
* Update runtime/parachains/src/dmp.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Make DeliveryFeeFactor be a StorageMap keyed on ParaIds
* Fixes
* introduce limit for fee increase on dmp queue
* add message_size based fee factor to increment_fee_factor
* change message_size fee rate to correct value
* fix div by 0 error
* bind limit to variable
* fix message_size_factor and add DeliveryFeeFactor test
* add test for ExponentialPrice implementation
* make test formula based
* make delivery fee factor test formula based
* add max value test for DeliveryFeeFactor and move limit to config
* change threshold back to dynamic value and fix tests
* fmt
* suggested changes and fmt
* small stylistic change
* fmt
* change to tokenlocation
* small fixes
* fmt
* remove sp_arithmetic dependency
* Update runtime/parachains/src/dmp.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
---------
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Just van Stam <just.van.stam@gmail.com>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Happy New Year!
* Remove year entierly
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Remove years from copyright notice in the entire repo
---------
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Ensure for a configurable origin in XCM (#6442), cherry picked from
5ae05e1a957857c449a43d8759a21292d03fd049
Add new associated type, AdminOrigin, bounded by EnsureOrigin trait in
XCM pallet. Replace ensure_root() with ensure_origin() from a
EnsureOrigin trait. Set AdminOrigin as EnsureRoot<AccountId> in xcm
configs.
* cargo fmt
* small stylistic change
---------
Co-authored-by: serkul <serkul@pop-os.localdomain>
* Some renames
* Fix
* Fix build for new APIs
* Remove diener
* Fixes
* Fixes
* Fix integration tests
* Fixes
* fix nis issuance
* Update Cargo.toml
* Polkadot doesn't have freezes/holds yet
* No networks use freezes/holds
* update lockfile for {"substrate"}
* Fix tests
There are more failing tests; just starting with the easy ones.
Reserved balance does not count towards ED anymore, therefore reducing
all the reserves by ED (1).
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fixes for Polkadot pallets
* Fix parachains benchmarks
* Update Substrate
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Keith Yeung