(imported from https://github.com/paritytech/cumulus/pull/2157)
## Changes
This MR refactores the XCMP, Parachains System and DMP pallets to use
the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
for delayed execution of incoming messages. The DMP pallet is entirely
replaced by the MQ and thereby removed. This allows for PoV-bounded
execution and resolves a number of issues that stem from the current
work-around.
All System Parachains adopt this change.
The most important changes are in `primitives/core/src/lib.rs`,
`parachains/common/src/process_xcm_message.rs`,
`pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
and the runtime configs.
### DMP Queue Pallet
The pallet got removed and its logic refactored into parachain-system.
Overweight message management can be done directly through the MQ
pallet.
Final undeployment migrations are provided by
`cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
can be configured with an aux config trait like:
```rust
parameter_types! {
pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
}
impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
type PalletName = DmpQueuePalletName;
type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
type DbWeight = <Runtime as frame_system::Config>::DbWeight;
}
// And adding them to your Migrations tuple:
pub type Migrations = (
...
cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
);
```
### XCMP Queue pallet
Removed all dispatch queue functionality. Incoming XCMP messages are now
either: Immediately handled if they are Signals, enqueued into the MQ
pallet otherwise.
New config items for the XCMP queue pallet:
```rust
/// The actual queue implementation that retains the messages for later processing.
type XcmpQueue: EnqueueMessage<ParaId>;
/// How a XCM over HRMP from a sibling parachain should be processed.
type XcmpProcessor: ProcessMessage<Origin = ParaId>;
/// The maximal number of suspended XCMP channels at the same time.
#[pallet::constant]
type MaxInboundSuspended: Get<u32>;
```
How to configure those:
```rust
// Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
// the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;
// Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
// with origin `Junction::Sibling(…)`.
type XcmpProcessor = ProcessFromSibling<
ProcessXcmMessage<
AggregateMessageOrigin,
xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
RuntimeCall,
>,
>;
// Not really important what to choose here. Just something larger than the maximal number of channels.
type MaxInboundSuspended = sp_core::ConstU32<1_000>;
```
The `InboundXcmpStatus` storage item was replaced by
`InboundXcmpSuspended` since it now only tracks inbound queue suspension
and no message indices anymore.
Now only sends the most recent channel `Signals`, as all prio ones are
out-dated anyway.
### Parachain System pallet
For `DMP` messages instead of forwarding them to the `DMP` pallet, it
now pushes them to the configured `DmpQueue`. The message processing
which was triggered in `set_validation_data` is now being done by the MQ
pallet `on_initialize`.
XCMP messages are still handed off to the `XcmpMessageHandler`
(XCMP-Queue pallet) - no change here.
New config items for the parachain system pallet:
```rust
/// Queues inbound downward messages for delayed processing.
///
/// Analogous to the `XcmpQueue` of the XCMP queue pallet.
type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
```
How to configure:
```rust
/// Use the MQ pallet to store DMP messages for delayed processing.
type DmpQueue = MessageQueue;
```
## Message Flow
The flow of messages on the parachain side. Messages come in from the
left via the `Validation Data` and finally end up at the `Xcm Executor`
on the right.
## Further changes
- Bumped the default suspension, drop and resume thresholds in
`QueueConfigData::default()`.
- `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
they would be a noop.
- Properly validate the `QueueConfigData` before setting it.
- Marked weight files as auto-generated so they wont auto-expand in the
MR files view.
- Move the `hypothetical` asserts to `frame_support` under the name
`experimental_hypothetically`
Questions:
- [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
the runtimes? Not sure how to best fix. Just having them like this makes
tests fail that rely on the real message processor when the feature is
enabled.
- [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
already takes 80% so I put it to 10% but that is quite low.
TODO:
- [x] Remove c&p code after
https://github.com/paritytech/polkadot/pull/6271
- [x] Use `HandleMessage` once it is public in Substrate
- [x] fix `runtime-benchmarks` feature
https://github.com/paritytech/polkadot/pull/6966
- [x] Benchmarks
- [x] Tests
- [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
- [x] Possibly cleanup Migrations (DMP+XCMP)
- [x] optional: create `TransformProcessMessageOrigin` in Substrate and
replace `ProcessFromSibling`
- [ ] Rerun weights on ref HW
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
This commit adds Rococo Asset Hub dedicated runtime so we can test new
features here, before merging them in Kusama Asset Hub.
Also adds one such feature: asset transfer over bridge (Rococo AssetHub
<> Wococo AssetHub)
- clone `asset-hub-kusama-runtime` -> `asset-hub-rococo-runtime`
- make it use Rococo primitives, names, assets, constants, etc
- add asset-transfer-over-bridge support to Rococo AssetHub <> Wococo
AssetHub
Fixes#1128
---------
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
* Rename `polkadot-parachain` to `polkadot-parachain-primitives`
While doing this it also fixes some last `rustdoc` issues and fixes
another Cargo warning related to `pallet-paged-list`.
* Fix compilation
* ".git/.scripts/commands/fmt/fmt.sh"
* Fix XCM docs
---------
Co-authored-by: command-bot <>
* Update substrate & polkadot
* min changes to make async backing compile
* (async backing) parachain-system: track limitations for unincluded blocks (#2438)
* unincluded segment draft
* read para head from storage proof
* read_para_head -> read_included_para_head
* Provide pub interface
* add errors
* fix unincluded segment update
* BlockTracker -> Ancestor
* add a dmp limit
* Read para head depending on the storage switch
* doc comments
* storage items docs
* add a sanity check on block initialize
* Check watermark
* append to the segment on block finalize
* Move segment update into set_validation_data
* Resolve para head todo
* option watermark
* fix comment
* Drop dmq check
* fix weight
* doc-comments on inherent invariant
* Remove TODO
* add todo
* primitives tests
* pallet tests
* doc comments
* refactor unincluded segment length into a ConsensusHook (#2501)
* refactor unincluded segment length into a ConsensusHook
* add docs
* refactor bandwidth_out calculation
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
* test for limits from impl
* fmt
* make tests compile
* update comment
* uncomment test
* fix collator test by adding parent to state proof
* patch HRMP watermark rules for unincluded segment
* get consensus-common tests to pass, using unincluded segment
* fix unincluded segment tests
* get all tests passing
* fmt
* rustdoc CI
* aura-ext: limit the number of authored blocks per slot (#2551)
* aura_ext consensus hook
* reverse dependency
* include weight into hook
* fix tests
* remove stray println
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
* fix test warning
* fix doc link
---------
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
* parachain-system: ignore go ahead signal once upgrade is processed (#2594)
* handle goahead signal for unincluded segment
* doc comment
* add test
* parachain-system: drop processed messages from inherent data (#2590)
* implement `drop_processed_messages`
* drop messages based on relay parent number
* adjust tests
* drop changes to mqc
* fix comment
* drop test
* drop more dead code
* clippy
* aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)
* aura-ext: check slot in consensus hook
* convert relay chain slot
* Make relay chain slot duration generic
* use fixed velocity hook for pallets with aura
* purge timestamp inherent
* fix warning
* adjust runtime tests
* fix slots in tests
* Make `xcm-emulator` test pass for new consensus hook (#2722)
* add pallets on_initialize
* tests pass
* add AuraExt on_init
* ".git/.scripts/commands/fmt/fmt.sh"
---------
Co-authored-by: command-bot <>
---------
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
* update polkadot git refs
* CollationGenerationConfig closure is now optional (#2772)
* CollationGenerationConfig closure is now optional
* fix test
* propagate network-protocol-staging feature (#2899)
* Feature Flagging Consensus Hook Type Parameter (#2911)
* First pass
* fmt
* Added as default feature in tomls
* Changed to direct dependency feature
* Dealing with clippy error
* Update pallets/parachain-system/src/lib.rs
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
---------
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
* fmt
* bump deps and remove warning
* parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)
* mostly address 2471 with a bug introduced
* adjust relevant messaging state after computing total
* fmt
* max -> min
* fix test implementation of xcmp source
* add test
* fix test message sending logic
* fix + test
* add more to unincluded segment test
* fmt
---------
Co-authored-by: Chris Sosnin <chris125_@live.com>
* Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)
* add a comment
* refactor client/service utilities
* deprecate start_collator
* update parachain-template
* update test-service in the same way
* update polkadot-parachain crate
* fmt
* wire up new SubmitCollation message
* some runtime utilities for implementing unincluded segment runtime APIs
* allow parachains to configure their level of sybil-resistance when starting the network
* make aura-ext compile
* update to specify sybil resistance levels
* fmt
* specify relay chain slot duration in milliseconds
* update Aura to explicitly produce Send futures
also, make relay_chain_slot_duration a Duration
* add authoring duration to basic collator and document params
* integrate new basic collator into parachain-template
* remove assert_send used for testing
* basic-aura: only author when parent included
* update polkadot-parachain-bin
* fmt
* some fixes
* fixes
* add a RelayNumberMonotonicallyIncreases
* add a utility function for initializing subsystems
* some logging for timestamp adjustment
* fmt
* some fixes for lookahead collator
* add a log
* update `find_potential_parents` to account for sessions
* bound the loop
* restore & deprecate old start_collator and start_full_node functions.
* remove unnecessary await calls
* fix warning
* clippy
* more clippy
* remove unneeded logic
* ci
* update comment
Co-authored-by: Marcin S. <marcin@bytedude.com>
* (async backing) restore `CheckInherents` for backwards-compatibility (#2977)
* bring back timestamp
* Restore CheckInherents
* revert to empty CheckInherents
* make CheckInherents optional
* attempt
* properly end system blocks
* add some more comments
* ignore failing system parachain tests
* update refs after main feature branch merge
* comment out the offending tests because CI runs ignored tests
* fix warnings
* fmt
* revert to polkadot master
* cargo update -p polkadot-primitives -p sp-io
---------
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>
Added some tests that aim to cover the runtime configuration
that is exercised when BH receives relayed complex message.
* checks correct importing of proofs for:
bridged chain finality, bridged para heads, bridged messages,
* checks relayer extension correctly configured to reward
submitting relayer,
* checks relayed message is successfully dispatched.
Also moved generic test-utils from
`asset-test-utils: parachains/runtimes/assets/test-utils`
one level up to new crate
`parachains-runtimes-test-utils: parachains/runtimes/test-utils`
to be reused by BridgeHubs.
Signed-off-by: acatangiu <adrian@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Oliver Tale-Yazdi