pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-14 01:41:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use Message Queue as DMP and XCMP dispatch queue (#1246)

Browse Source 
(imported from https://github.com/paritytech/cumulus/pull/2157)

## Changes

This MR refactores the XCMP, Parachains System and DMP pallets to use
the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
for delayed execution of incoming messages. The DMP pallet is entirely
replaced by the MQ and thereby removed. This allows for PoV-bounded
execution and resolves a number of issues that stem from the current
work-around.

All System Parachains adopt this change.  
The most important changes are in `primitives/core/src/lib.rs`,
`parachains/common/src/process_xcm_message.rs`,
`pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
and the runtime configs.

### DMP Queue Pallet

The pallet got removed and its logic refactored into parachain-system.
Overweight message management can be done directly through the MQ
pallet.

Final undeployment migrations are provided by
`cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
can be configured with an aux config trait like:

```rust
parameter_types! {
	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
}

impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
	type PalletName = DmpQueuePalletName;
	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
}

// And adding them to your Migrations tuple:
pub type Migrations = (
	...
	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
);
```

### XCMP Queue pallet

Removed all dispatch queue functionality. Incoming XCMP messages are now
either: Immediately handled if they are Signals, enqueued into the MQ
pallet otherwise.

New config items for the XCMP queue pallet:
```rust
/// The actual queue implementation that retains the messages for later processing.
type XcmpQueue: EnqueueMessage<ParaId>;

/// How a XCM over HRMP from a sibling parachain should be processed.
type XcmpProcessor: ProcessMessage<Origin = ParaId>;

/// The maximal number of suspended XCMP channels at the same time.
#[pallet::constant]
type MaxInboundSuspended: Get<u32>;
```

How to configure those:

```rust
// Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
// the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;

// Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
// with origin `Junction::Sibling(…)`.
type XcmpProcessor = ProcessFromSibling<
	ProcessXcmMessage<
		AggregateMessageOrigin,
		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
		RuntimeCall,
	>,
>;

// Not really important what to choose here. Just something larger than the maximal number of channels.
type MaxInboundSuspended = sp_core::ConstU32<1_000>;
```

The `InboundXcmpStatus` storage item was replaced by
`InboundXcmpSuspended` since it now only tracks inbound queue suspension
and no message indices anymore.

Now only sends the most recent channel `Signals`, as all prio ones are
out-dated anyway.

### Parachain System pallet

For `DMP` messages instead of forwarding them to the `DMP` pallet, it
now pushes them to the configured `DmpQueue`. The message processing
which was triggered in `set_validation_data` is now being done by the MQ
pallet `on_initialize`.

XCMP messages are still handed off to the `XcmpMessageHandler`
(XCMP-Queue pallet) - no change here.

New config items for the parachain system pallet:
```rust
/// Queues inbound downward messages for delayed processing. 
///
/// Analogous to the `XcmpQueue` of the XCMP queue pallet.
type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
``` 

How to configure:
```rust
/// Use the MQ pallet to store DMP messages for delayed processing.
type DmpQueue = MessageQueue;
``` 

## Message Flow

The flow of messages on the parachain side. Messages come in from the
left via the `Validation Data` and finally end up at the `Xcm Executor`
on the right.

![Untitled
(1)](https://github.com/paritytech/cumulus/assets/10380170/6cf8b377-88c9-4aed-96df-baace266e04d)

## Further changes

- Bumped the default suspension, drop and resume thresholds in
`QueueConfigData::default()`.
- `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
they would be a noop.
- Properly validate the `QueueConfigData` before setting it.
- Marked weight files as auto-generated so they wont auto-expand in the
MR files view.
- Move the `hypothetical` asserts to `frame_support` under the name
`experimental_hypothetically`

Questions:
- [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
the runtimes? Not sure how to best fix. Just having them like this makes
tests fail that rely on the real message processor when the feature is
enabled.
- [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
already takes 80% so I put it to 10% but that is quite low.

TODO:
- [x] Remove c&p code after
https://github.com/paritytech/polkadot/pull/6271
- [x] Use `HandleMessage` once it is public in Substrate
- [x] fix `runtime-benchmarks` feature
https://github.com/paritytech/polkadot/pull/6966
- [x] Benchmarks
- [x] Tests
- [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
- [x] Possibly cleanup Migrations (DMP+XCMP)
- [x] optional: create `TransformProcessMessageOrigin` in Substrate and
replace `ProcessFromSibling`
- [ ] Rerun weights on ref HW

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
This commit is contained in:
Oliver Tale-Yazdi
2023-11-02 15:31:38 +01:00
committed by GitHub
parent 7df0417bcd
commit e1c033ebe1
277 changed files with 11604 additions and 4733 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+33 -6
View File
@@ -760,6 +760,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-nft-fractionalization",
"pallet-nfts",
@@ -832,6 +833,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-nfts",
"pallet-nfts-runtime-api",
@@ -933,6 +935,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-nft-fractionalization",
"pallet-nfts",
@@ -993,6 +996,7 @@ dependencies = [
"pallet-asset-rate",
"pallet-assets",
"pallet-balances",
"pallet-message-queue",
"pallet-treasury",
"pallet-xcm",
"parachains-common",
@@ -1043,6 +1047,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-nft-fractionalization",
"pallet-nfts",
@@ -1091,7 +1096,6 @@ name = "asset-test-utils"
version = "1.0.0"
dependencies = [
"assets-common",
"cumulus-pallet-dmp-queue",
"cumulus-pallet-parachain-system",
"cumulus-pallet-xcmp-queue",
"cumulus-primitives-core",
@@ -2027,6 +2031,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-session",
"pallet-timestamp",
@@ -2090,6 +2095,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-session",
"pallet-timestamp",
@@ -2137,14 +2143,20 @@ dependencies = [
"cumulus-pallet-dmp-queue",
"cumulus-pallet-xcmp-queue",
"frame-support",
"frame-system",
"integration-tests-common",
"pallet-assets",
"pallet-balances",
"pallet-bridge-messages",
"pallet-message-queue",
"pallet-xcm",
"parachains-common",
"parity-scale-codec",
"polkadot-core-primitives",
"polkadot-parachain-primitives",
"polkadot-runtime-parachains",
"sp-core",
"sp-weights",
"staging-xcm",
"staging-xcm-executor",
"xcm-emulator",
@@ -2196,6 +2208,7 @@ dependencies = [
"pallet-bridge-parachains",
"pallet-bridge-relayers",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-session",
"pallet-timestamp",
@@ -2251,7 +2264,6 @@ dependencies = [
"bp-runtime",
"bp-test-utils",
"bridge-runtime-common",
"cumulus-pallet-dmp-queue",
"cumulus-pallet-parachain-system",
"cumulus-pallet-xcmp-queue",
"frame-benchmarking",
@@ -2325,6 +2337,7 @@ dependencies = [
"pallet-bridge-parachains",
"pallet-bridge-relayers",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-multisig",
"pallet-session",
"pallet-timestamp",
@@ -2871,6 +2884,7 @@ dependencies = [
"pallet-collective",
"pallet-collective-content",
"pallet-core-fellowship",
"pallet-message-queue",
"pallet-multisig",
"pallet-preimage",
"pallet-proxy",
@@ -3087,6 +3101,7 @@ dependencies = [
"pallet-contracts",
"pallet-contracts-primitives",
"pallet-insecure-randomness-collective-flip",
"pallet-message-queue",
"pallet-multisig",
"pallet-session",
"pallet-sudo",
@@ -3792,6 +3807,7 @@ name = "cumulus-pallet-dmp-queue"
version = "0.1.0"
dependencies = [
"cumulus-primitives-core",
"frame-benchmarking",
"frame-support",
"frame-system",
"log",
@@ -3801,7 +3817,7 @@ dependencies = [
"sp-io",
"sp-runtime",
"sp-std 8.0.0",
"sp-version",
"sp-tracing 10.0.0",
"staging-xcm",
]
@@ -3817,15 +3833,18 @@ dependencies = [
"cumulus-test-client",
"cumulus-test-relay-sproof-builder",
"environmental",
"frame-benchmarking",
"frame-support",
"frame-system",
"hex-literal",
"impl-trait-for-tuples",
"lazy_static",
"log",
"pallet-message-queue",
"parity-scale-codec",
"polkadot-parachain-primitives",
"polkadot-runtime-parachains",
"rand 0.8.5",
"sc-client-api",
"scale-info",
"sp-core",
@@ -3900,6 +3919,7 @@ dependencies = [
name = "cumulus-pallet-xcmp-queue"
version = "0.1.0"
dependencies = [
"bounded-collections",
"bp-xcm-bridge-hub-router",
"cumulus-pallet-parachain-system",
"cumulus-primitives-core",
@@ -3908,10 +3928,10 @@ dependencies = [
"frame-system",
"log",
"pallet-balances",
"pallet-message-queue",
"parity-scale-codec",
"polkadot-runtime-common",
"polkadot-runtime-parachains",
"rand_chacha 0.3.1",
"scale-info",
"sp-core",
"sp-io",
@@ -4191,6 +4211,7 @@ dependencies = [
"frame-system-rpc-runtime-api",
"pallet-balances",
"pallet-glutton",
"pallet-message-queue",
"pallet-sudo",
"pallet-timestamp",
"pallet-transaction-payment",
@@ -6121,6 +6142,7 @@ dependencies = [
"frame-try-runtime",
"pallet-aura",
"pallet-glutton",
"pallet-message-queue",
"pallet-sudo",
"pallet-timestamp",
"parachains-common",
@@ -6692,7 +6714,6 @@ dependencies = [
"bridge-hub-rococo-runtime",
"bridge-runtime-common",
"collectives-polkadot-runtime",
"cumulus-pallet-dmp-queue",
"cumulus-pallet-parachain-system",
"cumulus-pallet-xcmp-queue",
"cumulus-primitives-core",
@@ -11185,6 +11206,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-parachain-template",
"pallet-session",
"pallet-sudo",
@@ -11192,6 +11214,7 @@ dependencies = [
"pallet-transaction-payment",
"pallet-transaction-payment-rpc-runtime-api",
"pallet-xcm",
"parachains-common",
"parity-scale-codec",
"polkadot-parachain-primitives",
"polkadot-runtime-common",
@@ -11231,6 +11254,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"parity-scale-codec",
"polkadot-core-primitives",
"polkadot-primitives",
@@ -11254,7 +11278,6 @@ name = "parachains-runtimes-test-utils"
version = "1.0.0"
dependencies = [
"assets-common",
"cumulus-pallet-dmp-queue",
"cumulus-pallet-parachain-system",
"cumulus-pallet-xcmp-queue",
"cumulus-primitives-core",
@@ -11506,6 +11529,7 @@ dependencies = [
"pallet-authorship",
"pallet-balances",
"pallet-collator-selection",
"pallet-message-queue",
"pallet-session",
"pallet-sudo",
"pallet-timestamp",
@@ -14223,6 +14247,7 @@ dependencies = [
"pallet-assets",
"pallet-aura",
"pallet-balances",
"pallet-message-queue",
"pallet-sudo",
"pallet-timestamp",
"pallet-transaction-payment",
@@ -16563,6 +16588,7 @@ dependencies = [
"frame-system",
"frame-try-runtime",
"pallet-aura",
"pallet-message-queue",
"pallet-timestamp",
"parachains-common",
"parity-scale-codec",
@@ -21045,6 +21071,7 @@ name = "xcm-emulator"
version = "0.1.0"
dependencies = [
"cumulus-pallet-parachain-system",
"cumulus-pallet-xcmp-queue",
"cumulus-primitives-core",
"cumulus-primitives-parachain-inherent",
"cumulus-test-relay-sproof-builder",