8 Commits

Author	SHA1	Message	Date
Dónal Murray	d68868f64a	Fix clippy lints behind feature gates and add new CI step all features (#2569) ... Many clippy lints usually enforced by `-Dcomplexity` and `-Dcorrectness` are not caught by CI as they are gated by `features`, like `runtime-benchmarks`, while the clippy CI job runs with only the default features for all targets. This PR also adds a CI step to run clippy with `--all-features` to ensure the code quality is maintained behind feature gates from now on. To improve local development, clippy lints are downgraded to warnings, but they still will result in an error at CI due to the `-Dwarnings` rustflag. --------- Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>	2023-12-20 16:30:10 +00:00
Oliver Tale-Yazdi	e1c033ebe1	Use Message Queue as DMP and XCMP dispatch queue (#1246) ... (imported from https://github.com/paritytech/cumulus/pull/2157) ## Changes This MR refactores the XCMP, Parachains System and DMP pallets to use the [MessageQueue](https://github.com/paritytech/substrate/pull/12485) for delayed execution of incoming messages. The DMP pallet is entirely replaced by the MQ and thereby removed. This allows for PoV-bounded execution and resolves a number of issues that stem from the current work-around. All System Parachains adopt this change. The most important changes are in `primitives/core/src/lib.rs`, `parachains/common/src/process_xcm_message.rs`, `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs` and the runtime configs. ### DMP Queue Pallet The pallet got removed and its logic refactored into parachain-system. Overweight message management can be done directly through the MQ pallet. Final undeployment migrations are provided by `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that can be configured with an aux config trait like: ```rust parameter_types! { pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME; pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent; } impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime { type PalletName = DmpQueuePalletName; type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>; type DbWeight = <Runtime as frame_system::Config>::DbWeight; } // And adding them to your Migrations tuple: pub type Migrations = ( ... cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>, cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>, ); ``` ### XCMP Queue pallet Removed all dispatch queue functionality. Incoming XCMP messages are now either: Immediately handled if they are Signals, enqueued into the MQ pallet otherwise. New config items for the XCMP queue pallet: ```rust /// The actual queue implementation that retains the messages for later processing. type XcmpQueue: EnqueueMessage<ParaId>; /// How a XCM over HRMP from a sibling parachain should be processed. type XcmpProcessor: ProcessMessage<Origin = ParaId>; /// The maximal number of suspended XCMP channels at the same time. #[pallet::constant] type MaxInboundSuspended: Get<u32>; ``` How to configure those: ```rust // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s. type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>; // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched // with origin `Junction::Sibling(…)`. type XcmpProcessor = ProcessFromSibling< ProcessXcmMessage< AggregateMessageOrigin, xcm_executor::XcmExecutor<xcm_config::XcmConfig>, RuntimeCall, >, >; // Not really important what to choose here. Just something larger than the maximal number of channels. type MaxInboundSuspended = sp_core::ConstU32<1_000>; ``` The `InboundXcmpStatus` storage item was replaced by `InboundXcmpSuspended` since it now only tracks inbound queue suspension and no message indices anymore. Now only sends the most recent channel `Signals`, as all prio ones are out-dated anyway. ### Parachain System pallet For `DMP` messages instead of forwarding them to the `DMP` pallet, it now pushes them to the configured `DmpQueue`. The message processing which was triggered in `set_validation_data` is now being done by the MQ pallet `on_initialize`. XCMP messages are still handed off to the `XcmpMessageHandler` (XCMP-Queue pallet) - no change here. New config items for the parachain system pallet: ```rust /// Queues inbound downward messages for delayed processing. /// /// Analogous to the `XcmpQueue` of the XCMP queue pallet. type DmpQueue: EnqueueMessage<AggregateMessageOrigin>; ``` How to configure: ```rust /// Use the MQ pallet to store DMP messages for delayed processing. type DmpQueue = MessageQueue; ``` ## Message Flow The flow of messages on the parachain side. Messages come in from the left via the `Validation Data` and finally end up at the `Xcm Executor` on the right.  ## Further changes - Bumped the default suspension, drop and resume thresholds in `QueueConfigData::default()`. - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when they would be a noop. - Properly validate the `QueueConfigData` before setting it. - Marked weight files as auto-generated so they wont auto-expand in the MR files view. - Move the `hypothetical` asserts to `frame_support` under the name `experimental_hypothetically` Questions: - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in the runtimes? Not sure how to best fix. Just having them like this makes tests fail that rely on the real message processor when the feature is enabled. - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler already takes 80% so I put it to 10% but that is quite low. TODO: - [x] Remove c&p code after https://github.com/paritytech/polkadot/pull/6271 - [x] Use `HandleMessage` once it is public in Substrate - [x] fix `runtime-benchmarks` feature https://github.com/paritytech/polkadot/pull/6966 - [x] Benchmarks - [x] Tests - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended` - [x] Possibly cleanup Migrations (DMP+XCMP) - [x] optional: create `TransformProcessMessageOrigin` in Substrate and replace `ProcessFromSibling` - [ ] Rerun weights on ref HW --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: command-bot <>	2023-11-02 15:31:38 +01:00
Przemek Rzad	bfb241d7f3	Add missing licenses and tune the scanning workflow (#1288) ... * Add missing Cumulus licenses * Typo * Add missing Substrate licenses * Single job checking the sub-repos in steps * Remove dates * Remove dates * Add missing (C) * Update FRAME UI tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update more UI tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>	2023-08-30 15:45:49 +03:00
Oliver Tale-Yazdi	70a0864e4c	Migrate to Weight::from_parts (#2245) ... * Migrate to from_parts Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix remaining files Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo update -p polkadot-primitives Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo update -p sp-io Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use Master Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo update -p polkadot-primitives Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo update -p sp-io Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>	2023-03-03 12:47:23 +00:00
Gavin Wood	3a459e8fcf	XCM v3 Companion (#697) ... * Fixes * Undiener * Undiener * Undiener * Lockfile * Changes for send returning hash * Include message ID as params to execute_xcm * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Companion fixes * Formatting * Fixes * Formatting * Bump * Bump * Fixes * Formatting * Make the price of UMP/XCMP message sending configurable * cargo fmt * Remove InvertLocation * Formatting * Use ConstantPrice from polkadot-runtime-common * Fix naming * cargo fmt * Fixes * Fixes * Fixes * Add CallDispatcher * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Remove unused import * Remove unused import * XCMv3 fixes (#1710) * Fixes XCMv3 related Fixes XCMv3 (removed query_holding) Fixes XCMv3 - should use _depositable_count? Fixes XCMv3 - removed TrustedReserve Fixes - missing weights for statemine/statemint/westmint [DO-NOT-CHERRY-PICK] tmp return query_holding to aviod conficts to master Fixes - missing functions for pallet_xcm_benchmarks::generic::Config Fixes for XCMv3 benchmarking Fix xcm - removed query_holding * ".git/.scripts/bench-bot.sh" xcm statemine assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" xcm statemint assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" xcm westmint assets pallet_xcm_benchmarks::generic * Fix imports * Avoid consuming XCM message for NotApplicable scenario (#1787) * Avoid consuming message for NotApplicable scenario * Avoid consuming message for NotApplicable scenario tests * Add 10 message processing limit to DMP queue * Add 10 message limit to XCMP queue * Always increment the message_processed count whenever a message is processed * Fix formatting * Set an upper limit to the overweight message DMP queue * Add upper limit to XCMP overweight message queue * Fix for missing weight for `fn unpaid_execution()` * Fix - usage of `messages_processed` * Fixes * Fixes * Fixes * cargo fmt * Fixes * Fixes * Fixes * Fixes * Remove unused import * Fixes for gav-xcm-v3 (#1835) * Fix for FungiblesAdapter - trait changes: Contains -> AssetChecking * Fix for missing weight for `fn unpaid_execution()` * Used NonLocalMint for all NonZeroIssuance * Fix * Fixes * Fixes * Fixes * Fixes * Fixes * Fix tests * Fixes * Add SafeCallFilter * Add missing config items * Add TODO * Use () as the PriceForParentDelivery * Fixes * Fixes * Fixes * Fixes * Update transact_origin to transact_origin_and_runtime_call * Add ReachableDest config item to XCM pallet * Update SafeCallFilter to allow remark_with_event in runtime benchmarks * cargo fmt * Update substrate * Fix worst_case_holding * Fix DMQ queue unit tests * Remove unused label * cargo fmt * Actually process incoming XCMs * Fixes * Fixes * Fixes * Fixes - return back Weightless * Added measured benchmarks for `pallet_xcm` (#1968) * Fix Fix Fix * Fix * Fixes for transact benchmark * Fixes add pallet_xcm to benchmarks * Revert remark_with_event * ".git/.scripts/bench-bot.sh" xcm statemine assets pallet_xcm_benchmarks::generic * Fixes * TMP * Fix for reserve_asset_deposited * ".git/.scripts/bench-bot.sh" pallet statemine assets pallet_xcm * Fix * ".git/.scripts/bench-bot.sh" pallet statemint assets pallet_xcm * Fix * ".git/.scripts/bench-bot.sh" pallet westmint assets pallet_xcm * Fix westmint * ".git/.scripts/bench-bot.sh" xcm statemine assets pallet_xcm_benchmarks::generic * Fix * ".git/.scripts/bench-bot.sh" xcm westmint assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" xcm statemint assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" pallet collectives-polkadot collectives pallet_xcm * Fix for collectives * ".git/.scripts/bench-bot.sh" pallet bridge-hub-kusama bridge-hubs pallet_xcm * ".git/.scripts/bench-bot.sh" pallet bridge-hub-rococo bridge-hubs pallet_xcm * Fixes for bridge-hubs * Fixes - return back Weightless * Fix - removed MigrateToTrackInactive for contracts-rococo Co-authored-by: command-bot <> * cargo fmt * Fix benchmarks * Bko gav xcm v3 (#1993) * Fix * ".git/.scripts/bench-bot.sh" xcm statemint assets pallet_xcm_benchmarks::fungible * ".git/.scripts/bench-bot.sh" xcm statemine assets pallet_xcm_benchmarks::fungible * ".git/.scripts/bench-bot.sh" xcm westmint assets pallet_xcm_benchmarks::fungible * ".git/.scripts/bench-bot.sh" xcm statemine assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" xcm statemint assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" xcm westmint assets pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" pallet statemine assets pallet_xcm * ".git/.scripts/bench-bot.sh" pallet westmint assets pallet_xcm * ".git/.scripts/bench-bot.sh" pallet statemint assets pallet_xcm * ".git/.scripts/bench-bot.sh" pallet collectives-polkadot collectives pallet_xcm * ".git/.scripts/bench-bot.sh" pallet bridge-hub-kusama bridge-hubs pallet_xcm * ".git/.scripts/bench-bot.sh" pallet bridge-hub-rococo bridge-hubs pallet_xcm Co-authored-by: command-bot <> * Change AllowUnpaidExecutionFrom to be explicit * xcm-v3 benchmarks, weights, fixes for bridge-hubs (#2035) * Dumy weights to get compile * Change UniversalLocation according to https://github.com/paritytech/polkadot/pull/4097 (Location Inversion Removed) * Fix bridge-hubs weights * ".git/.scripts/bench-bot.sh" pallet statemine assets pallet_xcm * ".git/.scripts/bench-bot.sh" pallet statemint assets pallet_xcm * ".git/.scripts/bench-bot.sh" pallet collectives-polkadot collectives pallet_xcm * ".git/.scripts/bench-bot.sh" pallet westmint assets pallet_xcm * ".git/.scripts/bench-bot.sh" xcm bridge-hub-kusama bridge-hubs pallet_xcm_benchmarks::generic * ".git/.scripts/bench-bot.sh" xcm bridge-hub-kusama bridge-hubs pallet_xcm_benchmarks::fungible * ".git/.scripts/bench-bot.sh" pallet bridge-hub-kusama bridge-hubs pallet_xcm * ".git/.scripts/bench-bot.sh" pallet bridge-hub-rococo bridge-hubs pallet_xcm * ".git/.scripts/bench-bot.sh" xcm bridge-hub-rococo bridge-hubs pallet_xcm_benchmarks::fungible * ".git/.scripts/bench-bot.sh" xcm bridge-hub-rococo bridge-hubs pallet_xcm_benchmarks::generic * Change NetworkId to Option<NetworkId> Co-authored-by: command-bot <> Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> * Add event for showing the hash of an UMP sent message (#1228) * Add UpwardMessageSent event in parachain-system * additional fixes * Message Id * Fix errors from merge * fmt * more fmt * Remove todo * more formatting * Fixes * Fixes * Fixes * Fixes * Allow explicit unpaid executions from the relay chains for system parachains (#2060) * Allow explicit unpaid executions from the relay chains for system parachains * Put origin-filtering barriers into WithComputedOrigin * Use ConstU32<8> * Small nits * formatting * cargo fmt * Allow receiving XCMs from any relay chain plurality * Fixes * update lockfile for {"polkadot", "substrate"} * Update polkadot * Add runtime-benchmarks feature Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: girazoki <gorka.irazoki@gmail.com> Co-authored-by: parity-processbot <>	2023-01-17 09:26:34 +00:00
Keith Yeung	5cbb6b4c43	Migrate Weights properly to v2 (#1722) ... * Migrate Weights properly to v2 * Add missing on_runtime_upgrade implementation * Fix benchmarks * Apply suggestions from code review Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo fmt Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>	2022-10-11 16:36:03 +00:00
Shawn Tabrizi	48d4f1c57d	Companion for Weight v1.5 (#1581) ... * cargo test -p cumulus-primitives-utility * cargo test -p cumulus-pallet-xcmp-queue * cargo test -p cumulus-pallet-xcm * cargo test -p cumulus-pallet-dmp-queue * cargo test -p pallet-template * cargo test -p cumulus-test-runtime * fix weights * fix more weights * cargo test -p parachains-common * cargo test -p parachain-template-runtime * fix weights import * cargo test -p collectives-polkadot-runtime * cargo test -p contracts-rococo-runtime * more * unused * fixes * Update benchmarking.rs * Update lib.rs * Update lib.rs * fix * fix bug in conversion * update lockfile for {"polkadot", "substrate"} Co-authored-by: parity-processbot <>	2022-08-31 12:24:42 +00:00
Doordashcon	8d96b8673f	Add benchmarks for Xcmp QueueConfigData setters (#982) ... * add benchmarks for xcmp queue config data setters * add new benchmarks * cargo fmt * added newline * Additional weights for dmp queue for westmint * include new weights * Adding WeightInfo trait and friends * WeightInfo should be on xcmp rather than dmp pallet * cargo fmt * update scripts * mock weightinfo * cargo fmt * canvas kusama is substrate weight * weights from bm2 * expanding to other similar config functions * updated weights from bm2 * Revert "updated weights from bm2" This reverts commit b1702780982c278b44f572c2089b1d7ddc564d76. * Consolidation to one benchmark * reran weights * Update pallets/xcmp-queue/src/lib.rs Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com> * integrating review feedback * rerun weights * Add DispatchClass::Operational Co-authored-by: Squirrel <gilescope@gmail.com> Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>	2022-03-07 11:47:24 +00:00