Commit Graph

9 Commits

Author SHA1 Message Date
Oliver Tale-Yazdi e1c033ebe1 Use Message Queue as DMP and XCMP dispatch queue (#1246)
(imported from https://github.com/paritytech/cumulus/pull/2157)

## Changes

This MR refactores the XCMP, Parachains System and DMP pallets to use
the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
for delayed execution of incoming messages. The DMP pallet is entirely
replaced by the MQ and thereby removed. This allows for PoV-bounded
execution and resolves a number of issues that stem from the current
work-around.

All System Parachains adopt this change.  
The most important changes are in `primitives/core/src/lib.rs`,
`parachains/common/src/process_xcm_message.rs`,
`pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
and the runtime configs.

### DMP Queue Pallet

The pallet got removed and its logic refactored into parachain-system.
Overweight message management can be done directly through the MQ
pallet.

Final undeployment migrations are provided by
`cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
can be configured with an aux config trait like:

```rust
parameter_types! {
	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
}

impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
	type PalletName = DmpQueuePalletName;
	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
}

// And adding them to your Migrations tuple:
pub type Migrations = (
	...
	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
);
```

### XCMP Queue pallet

Removed all dispatch queue functionality. Incoming XCMP messages are now
either: Immediately handled if they are Signals, enqueued into the MQ
pallet otherwise.

New config items for the XCMP queue pallet:
```rust
/// The actual queue implementation that retains the messages for later processing.
type XcmpQueue: EnqueueMessage<ParaId>;

/// How a XCM over HRMP from a sibling parachain should be processed.
type XcmpProcessor: ProcessMessage<Origin = ParaId>;

/// The maximal number of suspended XCMP channels at the same time.
#[pallet::constant]
type MaxInboundSuspended: Get<u32>;
```

How to configure those:

```rust
// Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
// the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;

// Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
// with origin `Junction::Sibling(…)`.
type XcmpProcessor = ProcessFromSibling<
	ProcessXcmMessage<
		AggregateMessageOrigin,
		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
		RuntimeCall,
	>,
>;

// Not really important what to choose here. Just something larger than the maximal number of channels.
type MaxInboundSuspended = sp_core::ConstU32<1_000>;
```

The `InboundXcmpStatus` storage item was replaced by
`InboundXcmpSuspended` since it now only tracks inbound queue suspension
and no message indices anymore.

Now only sends the most recent channel `Signals`, as all prio ones are
out-dated anyway.

### Parachain System pallet

For `DMP` messages instead of forwarding them to the `DMP` pallet, it
now pushes them to the configured `DmpQueue`. The message processing
which was triggered in `set_validation_data` is now being done by the MQ
pallet `on_initialize`.

XCMP messages are still handed off to the `XcmpMessageHandler`
(XCMP-Queue pallet) - no change here.

New config items for the parachain system pallet:
```rust
/// Queues inbound downward messages for delayed processing. 
///
/// Analogous to the `XcmpQueue` of the XCMP queue pallet.
type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
``` 

How to configure:
```rust
/// Use the MQ pallet to store DMP messages for delayed processing.
type DmpQueue = MessageQueue;
``` 

## Message Flow

The flow of messages on the parachain side. Messages come in from the
left via the `Validation Data` and finally end up at the `Xcm Executor`
on the right.

![Untitled
(1)](https://github.com/paritytech/cumulus/assets/10380170/6cf8b377-88c9-4aed-96df-baace266e04d)

## Further changes

- Bumped the default suspension, drop and resume thresholds in
`QueueConfigData::default()`.
- `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
they would be a noop.
- Properly validate the `QueueConfigData` before setting it.
- Marked weight files as auto-generated so they wont auto-expand in the
MR files view.
- Move the `hypothetical` asserts to `frame_support` under the name
`experimental_hypothetically`

Questions:
- [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
the runtimes? Not sure how to best fix. Just having them like this makes
tests fail that rely on the real message processor when the feature is
enabled.
- [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
already takes 80% so I put it to 10% but that is quite low.

TODO:
- [x] Remove c&p code after
https://github.com/paritytech/polkadot/pull/6271
- [x] Use `HandleMessage` once it is public in Substrate
- [x] fix `runtime-benchmarks` feature
https://github.com/paritytech/polkadot/pull/6966
- [x] Benchmarks
- [x] Tests
- [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
- [x] Possibly cleanup Migrations (DMP+XCMP)
- [x] optional: create `TransformProcessMessageOrigin` in Substrate and
replace `ProcessFromSibling`
- [ ] Rerun weights on ref HW

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
2023-11-02 15:31:38 +01:00
Kian Paimani 35eb133baa Ensure correct variant count in Runtime[Hold/Freeze]Reason (#1900)
closes https://github.com/paritytech/polkadot-sdk/issues/1882

## Breaking Changes

This PR introduces a new item to `pallet_balances::Config`:

```diff
trait Config {
++    type RuntimeFreezeReasons;
}
```

This value is only used to check it against `type MaxFreeze`. A similar
check has been added for `MaxHolds` against `RuntimeHoldReasons`, which
is already given to `pallet_balances`.

In all contexts, you should pass the real `RuntimeFreezeReasons`
generated by `construct_runtime` to `type RuntimeFreezeReasons`. Passing
`()` would also work, but it would imply that the runtime uses no
freezes at all.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-24 12:01:04 +02:00
Bastian Köcher f3bf5c1acd xcm: Change TypeInfo::path to not include staging (#1948)
The `xcm` crate was renamed to `staging-xcm` to be able to publish it to
crates.io as someone as squatted `xcm`. The problem with this rename is
that the `TypeInfo` includes the crate name which ultimately lands in
the metadata. The metadata is consumed by downstream users like
`polkadot-js` or people building on top of `polkadot-js`. These people
are using the entire `path` to find the type in the type registry. Thus,
their code would break as the type path would now be [`staging_xcm`,
`VersionedXcm`] instead of [`xcm`, `VersionedXcm`]. This pull request
fixes this by renaming the path segment `staging_xcm` to `xcm`.

This requires: https://github.com/paritytech/scale-info/pull/197

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
2023-10-20 11:21:19 +02:00
Juan 68ab943c65 Remove deprecated pallet_balances's set_balance_deprecated and transfer dispatchables (#1226)
* remove deprecated dispatchables

* update test

* update tests

* update tests

* add prdocs

* add prdoc

* Update docs/prdoc/pr_1226.prdoc

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* move prdoc file

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
2023-09-06 16:06:47 +03:00
Gavin Wood bc6fca3c44 Ensure cumulus/bridges is ignored by formatter and run it (#1369) 2023-09-03 15:11:26 +02:00
Juan bdbe982970 Restructure dispatch macro related exports (#1162)
* restructure dispatch macro related exports

* moved Dispatchable to lib.rs

* fix .gitignore final newline

* ".git/.scripts/commands/fmt/fmt.sh"

* fix rustdocs

* wip

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: command-bot <>
Co-authored-by: ordian <write@reusable.software>
2023-08-31 14:08:44 +03:00
Oliver Tale-Yazdi dcda0e50f5 Fix build profiles (#1229)
* Fix build profiles

Closes https://github.com/paritytech/polkadot-sdk/issues/1155

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Manually set version to 1.0.0

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use workspace repo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* 'Authors and Edition from workspace

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-08-29 13:39:41 +02:00
alvicsam f441a5fc93 Diener workspacify
Signed-off-by: alvicsam <alvicsam@gmail.com>
2023-08-25 11:05:17 +02:00
Oliver Tale-Yazdi 3710edfedc [FRAME Core] New pallets: safe-mode and tx-pause (#12092)
* Add safe-mode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add to kitchensink-runtime

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Spelling

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename to tx-pause

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add SafeMode pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Automatically disable safe-mode in on_init…

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add permissionless enable+extend

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add repay+slash stake methods

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix stakes storage

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Genesis config for safe-mode pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Genesis config for safe-mode pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename ExtrinsicName to FunctionName

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Origin variable duration

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename FunctionName -> CallName

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename and docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Pallet safe mode tests (#12148)

* Add safe-mode mock runtime
* Add safe-mode tests
* Add ForceEnable- and ForceExtendOrigin
* Start dummy benchmarks
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Tests for `pallet-tx-pause` (#12259)

* mock added
* tests added
* dummy benchmarks started

* rename to active/inactive
tests broken, in progress

* Runtime types, fix tests

* WIP safe mode and tx pause {continued} (#12371)

* test coverage on safe mode and tx pause
* benchmarks & tests
* tests passing, use FullNameOf to check tx-pause unfilterables
* naming updates

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Set block number

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* dummy weights generated, safe mode

* add repay_reservation call with RepaymentDelay per #10033 feature requirements

* make call name optional to allow pausing pallets, handle `Contains` correctly for this throughout, doc comments started

* move to full_name notation for all interfaces, last commit introduced 1 more storage read.

* refactor is_paused

* safe math on safe mode

* Make stuff compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup & renames

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup TxPause pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Refactor to fungibles::* and rename stuf

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix node config

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Typos

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove CausalHoldReason

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Refactor benchmarks and runtime configs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add traits

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove old code

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup safe-mode benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/safe-mode/Cargo.toml

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/safe-mode/Cargo.toml

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove getters

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Apply suggestions from code review

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Use new as Origin benchmarking syntax

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix node

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tx-pause benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Renames

* Remove duplicate test

* Add docs

* docs

* Apply suggestions from code review

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>

* Cleanup tests

* docs

* Cleanup GenesisConfigs

* Doc traits

* Remove PauseTooLongNames

* docs

* Use V2 benchmarking

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use RuntimeHoldReason

* Fix kitchensink runtime

* Fix CI

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix CI

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename Stake to Deposit

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add Notify and test it

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix kitchensink

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/safe-mode/src/tests.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/safe-mode/src/tests.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/tx_pause.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/tx-pause/src/lib.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/tx-pause/src/lib.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/tx-pause/src/mock.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Simplify code

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update frame/support/src/traits/safe_mode.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Fixup merge

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make stuff compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make tx-pause compile again

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix more features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_safe_mode

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Dan Shields <35669742+NukeManDan@users.noreply.github.com>
Co-authored-by: Dan Shields <nukemandan@protonmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>
2023-08-25 00:56:00 +00:00