Commit Graph

7 Commits

Author SHA1 Message Date
Pavel Orlov 3c972fc19e XCM Fee Payment Runtime API (#3607)
The PR provides API for obtaining:
- the weight required to execute an XCM message,
- a list of acceptable `AssetId`s for message execution payment,
- the cost of the weight in the specified acceptable `AssetId`.

It is meant to address an issue where one has to guess how much fee to
pay for execution. Also, at the moment, a client has to guess which
assets are acceptable for fee execution payment.
See the related issue
https://github.com/paritytech/polkadot-sdk/issues/690.
With this API, a client is supposed to query the list of the supported
asset IDs (in the XCM version format the client understands), weigh the
XCM program the client wants to execute and convert the weight into one
of the acceptable assets. Note that the client is supposed to know what
program will be executed on what chains. However, having a small
companion JS library for the pallet-xcm and xtokens should be enough to
determine what XCM programs will be executed and where (since these
pallets compose a known small set of programs).
```Rust
pub trait XcmPaymentApi<Call>
	where
		Call: Codec,
	{
		/// Returns a list of acceptable payment assets.
		///
		/// # Arguments
		///
		/// * `xcm_version`: Version.
		fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec<VersionedAssetId>, Error>;
		/// Returns a weight needed to execute a XCM.
		///
		/// # Arguments
		///
		/// * `message`: `VersionedXcm`.
		fn query_xcm_weight(message: VersionedXcm<Call>) -> Result<Weight, Error>;
		/// Converts a weight into a fee for the specified `AssetId`.
		///
		/// # Arguments
		///
		/// * `weight`: convertible `Weight`.
		/// * `asset`: `VersionedAssetId`.
		fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
		/// Get delivery fees for sending a specific `message` to a `destination`.
		/// These always come in a specific asset, defined by the chain.
		///
		/// # Arguments
		/// * `message`: The message that'll be sent, necessary because most delivery fees are based on the
		///   size of the message.
		/// * `destination`: The destination to send the message to. Different destinations may use
		///   different senders that charge different fees.
		fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
	}
```
An
[example](https://gist.github.com/PraetorP/4bc323ff85401abe253897ba990ec29d)
of a client side code.

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Daniel Shiposha <mrshiposha@gmail.com>
2024-03-26 18:34:28 +00:00
Oliver Tale-Yazdi eefd5fe449 Multi-Block-Migrations, poll hook and new System callbacks (#1781)
This MR is the merge of
https://github.com/paritytech/substrate/pull/14414 and
https://github.com/paritytech/substrate/pull/14275. It implements
[RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes
https://github.com/paritytech/polkadot-sdk/issues/198.

----- 

This Merge request introduces three major topicals:

1. Multi-Block-Migrations
1. New pallet `poll` hook for periodic service work
1. Replacement hooks for `on_initialize` and `on_finalize` in cases
where `poll` cannot be used

and some more general changes to FRAME.  
The changes for each topical span over multiple crates. They are listed
in topical order below.

# 1.) Multi-Block-Migrations

Multi-Block-Migrations are facilitated by creating `pallet_migrations`
and configuring `System::Config::MultiBlockMigrator` to point to it.
Executive picks this up and triggers one step of the migrations pallet
per block.
The chain is in lockdown mode for as long as an MBM is ongoing.
Executive does this by polling `MultiBlockMigrator::ongoing` and not
allowing any transaction in a block, if true.

A MBM is defined through trait `SteppedMigration`. A condensed version
looks like this:
```rust
/// A migration that can proceed in multiple steps.
pub trait SteppedMigration {
	type Cursor: FullCodec + MaxEncodedLen;
	type Identifier: FullCodec + MaxEncodedLen;

	fn id() -> Self::Identifier;

	fn max_steps() -> Option<u32>;

	fn step(
		cursor: Option<Self::Cursor>,
		meter: &mut WeightMeter,
	) -> Result<Option<Self::Cursor>, SteppedMigrationError>;
}
```

`pallet_migrations` can be configured with an aggregated tuple of these
migrations. It then starts to migrate them one-by-one on the next
runtime upgrade.
Two things are important here:
- 1. Doing another runtime upgrade while MBMs are ongoing is not a good
idea and can lead to messed up state.
- 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`,
otherwise it is not used.**

The pallet supports an `UpgradeStatusHandler` that can be used to notify
external logic of upgrade start/finish (for example to pause XCM
dispatch).

Error recovery is very limited in the case that a migration errors or
times out (exceeds its `max_steps`). Currently the runtime dev can
decide in `FailedMigrationHandler::failed` how to handle this. One
follow-up would be to pair this with the `SafeMode` pallet and enact
safe mode when an upgrade fails, to allow governance to rescue the
chain. This is currently not possible, since governance is not
`Mandatory`.

## Runtime API

- `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to
inform the Block Author whether they can push transactions.

### Integration

Add it to your runtime implementation of `Core` and `BlockBuilder`:
```patch
diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs
@@ impl_runtime_apis! {
	impl sp_block_builder::Core<Block> for Runtime {
-		fn initialize_block(header: &<Block as BlockT>::Header) {
+		fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode {
			Executive::initialize_block(header)
		}

		...
	}
```

# 2.) `poll` hook

A new pallet hook is introduced: `poll`. `Poll` is intended to replace
mostly all usage of `on_initialize`.
The reason for this is that any code that can be called from
`on_initialize` cannot be migrated through an MBM. Currently there is no
way to statically check this; the implication is to use `on_initialize`
as rarely as possible.
Failing to do so can result in broken storage invariants.

The implementation of the poll hook depends on the `Runtime API` changes
that are explained above.

# 3.) Hard-Deadline callbacks

Three new callbacks are introduced and configured on `System::Config`:
`PreInherents`, `PostInherents` and `PostTransactions`.
These hooks are meant as replacement for `on_initialize` and
`on_finalize` in cases where the code that runs cannot be moved to
`poll`.
The reason for this is to make the usage of HD-code (hard deadline) more
explicit - again to prevent broken invariants by MBMs.

# 4.) FRAME (general changes)

## `frame_system` pallet

A new memorize storage item `InherentsApplied` is added. It is used by
executive to track whether inherents have already been applied.
Executive and can then execute the MBMs directly between inherents and
transactions.

The `Config` gets five new items:
- `SingleBlockMigrations` this is the new way of configuring migrations
that run in a single block. Previously they were defined as last generic
argument of `Executive`. This shift is brings all central configuration
about migrations closer into view of the developer (migrations that are
configured in `Executive` will still work for now but is deprecated).
- `MultiBlockMigrator` this can be configured to an engine that drives
MBMs. One example would be the `pallet_migrations`. Note that this is
only the engine; the exact MBMs are injected into the engine.
- `PreInherents` a callback that executes after `on_initialize` but
before inherents.
- `PostInherents` a callback that executes after all inherents ran
(including MBMs and `poll`).
- `PostTransactions` in symmetry to `PreInherents`, this one is called
before `on_finalize` but after all transactions.

A sane default is to set all of these to `()`. Example diff suitable for
any chain:
```patch
@@ impl frame_system::Config for Test {
 	type MaxConsumers = ConstU32<16>;
+	type SingleBlockMigrations = ();
+	type MultiBlockMigrator = ();
+	type PreInherents = ();
+	type PostInherents = ();
+	type PostTransactions = ();
 }
```

An overview of how the block execution now looks like is here. The same
graph is also in the rust doc.

<details><summary>Block Execution Flow</summary>
<p>

![Screenshot 2023-12-04 at 19 11
29](https://github.com/paritytech/polkadot-sdk/assets/10380170/e88a80c4-ef11-4faa-8df5-8b33a724c054)

</p>
</details> 

## Inherent Order

Moved to https://github.com/paritytech/polkadot-sdk/pull/2154

---------------


## TODO

- [ ] Check that `try-runtime` still works
- [ ] Ensure backwards compatibility with old Runtime APIs
- [x] Consume weight correctly
- [x] Cleanup

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2024-02-28 19:49:00 +00:00
Bastian Köcher d3d301fa42 ParachainHost: No need to be generic over the block or hash type (#2537)
The `BlockNumber` and `Hash` are fixed types any way.
2023-11-29 15:31:51 +01:00
Oliver Tale-Yazdi 342d720573 Use same fmt and clippy configs as in Substrate (#7611)
* Use same rustfmt.toml as Substrate

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* format format file

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Format with new config

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add Substrate Clippy config

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Print Clippy version in CI

Otherwise its difficult to reproduce locally.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make fmt happy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update node/core/pvf/src/error.rs

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

* Update node/core/pvf/src/error.rs

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
2023-08-14 14:29:29 +00:00
drskalman 2bbc7a73a3 Companion for Substrate#14373 (#7572)
* rename BEEFY `crypto` →`ecdsa_crypto`

* - bump up `BeefyApi` to version 3
- deal with `PeerId` error.

* update BEEFY dependency names for `fake-runtime` and `chain_spec`
revert Cargo.toml

* cargo fmt

* Use master Cargo.lock

* update lockfile for {"substrate"}

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: parity-processbot <>
2023-08-02 15:25:12 +00:00
Francisco Aguirre cc9f8129af Change Fixed to WeightInfoBounds for Polkadot (#7077)
* Add polkadot XCM benchmarks

* Add temp

* ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::generic

* Add weights to XCM on Polkadot

* Make CI fail on old files

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update template

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add reserve_asset_deposited benchmark

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Change initiate_reserve_deposit in runtime weights

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove trusted reserves from runtimes

* Fix pallet-xcm-benchmarks mock

* Fix test

* Change pallet xcm weigher in kusama

* Fix

* Remove merge conflict artifact

* Remove initiate_reserve_withdraw from generic benchmarks

* Add missing implementation to XCM benchmark

* Fix failing karura test

* Remove dbg!

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Fix fmt

* Revert "Fix fmt"

This reverts commit 676f2d8db07d7427750c79f95494d4988d06fda5.

* Fix fmt

* Remove duplicated template code

* Add back part of the template

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* Don't skip reserve asset deposited benchmark

* Remove call to non-generated benchmark yet

* Underscore unused parameter

* Skip not supported benchmarks and hardcode value

* Remove ReserveAssetDeposited benchmark

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* Add back ReserveAssetDeposited

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* Use default benchmark for ReserveAssetDeposited

* Add missing parameter

* Revert reserve asset deposited benchmark

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm westend pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible

* Add 'real' benchmarks

* Add TrustedReserve to actual XcmConfig

* Add TrustedReserve to actual XcmConfig (fix)

* Whitelist from benchmarking XCM storage keys read each block (#6871)

* Whitelist from benchmarking XCM storage keys read each block

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm

* Remove XcmPallet SupportedVersion from the benchmark whitelist

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime kusama pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm

* WIP

* Add necessary traits, remove unnecessary whitelisted keys

* Fix tests

* Remove unused file

* Remove unused import

---------

Co-authored-by: command-bot <>

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::fungible

* Fix spellchecker issues

* Remove unused migration code

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2023-07-19 14:18:03 +00:00
Bastian Köcher e53d15aa20 Do not depend on native runtimes for RuntimeApi (#7451)
* Implement runtime apis for fake runtime

These runtime api implementations are only used to make the compiler
think that we have implemented all required runtime apis. They will not
be called as we switch the executor to `WasmExecutor`. In the near
future we will not require these fake implementations anymore after
Substrate has shifted away from this compile time requirement.

This brings us the advantage that the `polkadot-service` doesn't need to
depend on the runtimes for getting the `RuntimeApi` type.

It also removes around 1min of build time on my machine ;)

* Fix warning

* FMT

* ".git/.scripts/commands/fmt/fmt.sh"

* Use more descriptive id

* Fix warnings

* Adapt path

* Fix 🙈

---------

Co-authored-by: command-bot <>
2023-07-04 10:09:14 +02:00