Revert "pallet-xcm: Deprecate `execute` and `send` in favor of
`execute_blob` and `send_blob` (#3749)"
This reverts commit feee773d15.
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>
This PR:
- adds `EnsureDecodableXcm` (testing) router that attempts to *encode*
and *decode* passed XCM `message` to ensure that the receiving side will
be able to decode, at least with the same XCM version.
- fixes `pallet_xcm` / `pallet_xcm_benchmarks` assets data generation
Relates to investigation of
https://substrate.stackexchange.com/questions/11288 and missing fix
https://github.com/paritytech/polkadot-sdk/pull/2129 which did not get
into the fellows 1.1.X release.
## Questions/TODOs
- [x] fix XCM benchmarks, which produces undecodable data - new router
catched at least two cases
- `BoundedVec exceeds its limit`
- `Fungible asset of zero amount is not allowed`
- [x] do we need to add `sort` to the `prepend_with` as we did for
reanchor [here](https://github.com/paritytech/polkadot-sdk/pull/2129)?
@serban300 (**created separate/follow-up PR**:
https://github.com/paritytech/polkadot-sdk/pull/4235)
- [x] We added decoding check to `XcmpQueue` -> `validate_xcm_nesting`,
why not to added to the `ParentAsUmp` or `ChildParachainRouter`?
@franciscoaguirre (**created separate/follow-up PR**:
https://github.com/paritytech/polkadot-sdk/pull/4236)
- [ ] `SendController::send_blob` replace `VersionedXcm::<()>::decode(`
with `VersionedXcm::<()>::decode_with_depth_limit(MAX_XCM_DECODE_DEPTH,
data)` ?
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
This should be configured on the runtime level and not somewhere inside
the pallet.
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Basically combines all the recommended calls into one
`build_using_defaults()` call or `init_with_defaults()` when there are
some custom changes required.
Preparation for https://github.com/paritytech/polkadot-sdk/pull/3935
Changes:
- Add some `default-features = false` for the case that a crate and that
dependency both support nostd builds.
- Shuffle files around of some benchmarking-only crates. These
conditionally disabled the `cfg_attr` for nostd and pulled in libstd.
Example [here](https://github.com/ggwpez/zepter/pull/95). The actual
logic is moved into a `inner.rs` to preserve nostd capability of the
crate in case the benchmarking feature is disabled.
- Add some `use sp_std::vec` where needed.
- Remove some `optional = true` in cases where it was not optional.
- Removed one superfluous `cfg_attr(not(feature = "std"), no_std..`.
All in all this should be logical no-op.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Introduce types to define 1:1 balance conversion for different relative
asset ids/locations of native asset.
Examples:
native asset on Asset Hub presented as `VersionedLocatableAsset` type in
the context of Relay Chain is
```
{
`location`: (0, Parachain(1000)),
`asset_id`: (1, Here),
}
```
and it's balance should be converted 1:1 by implementations of
`ConversionToAssetBalance` trait.
---------
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
This PR introduces `BlockHashProvider` into `pallet_mmr::Config`
This type is used to get `block_hash` for a given `block_number` rather
than directly using `frame_system::Pallet::block_hash`
The `DefaultBlockHashProvider` uses `frame_system::Pallet::block_hash`
to get the `block_hash`
Closes: #4062
This MR contains two major changes and some maintenance cleanup.
## 1. Free Standing Pallet Benchmark Runner
Closes https://github.com/paritytech/polkadot-sdk/issues/3045, depends
on your runtime exposing the `GenesisBuilderApi` (like
https://github.com/paritytech/polkadot-sdk/pull/1492).
Introduces a new binary crate: `frame-omni-bencher`.
It allows to directly benchmark a WASM blob - without needing a node or
chain spec.
This makes it much easier to generate pallet weights and should allow us
to remove bloaty code from the node.
It should work for all FRAME runtimes that dont use 3rd party host calls
or non `BlakeTwo256` block hashing (basically all polkadot parachains
should work).
It is 100% backwards compatible with the old CLI args, when the `v1`
compatibility command is used. This is done to allow for forwards
compatible addition of new commands.
### Example (full example in the Rust docs)
Installing the CLI:
```sh
cargo install --locked --path substrate/utils/frame/omni-bencher
frame-omni-bencher --help
```
Building the Westend runtime:
```sh
cargo build -p westend-runtime --release --features runtime-benchmarks
```
Benchmarking the runtime:
```sh
frame-omni-bencher v1 benchmark pallet --runtime target/release/wbuild/westend-runtime/westend_runtime.compact.compressed.wasm --all
```
## 2. Building the Benchmark Genesis State in the Runtime
Closes https://github.com/paritytech/polkadot-sdk/issues/2664
This adds `--runtime` and `--genesis-builder=none|runtime|spec`
arguments to the `benchmark pallet` command to make it possible to
generate the genesis storage by the runtime. This can be used with both
the node and the freestanding benchmark runners. It utilizes the new
`GenesisBuilder` RA and depends on having
https://github.com/paritytech/polkadot-sdk/pull/3412 deployed.
## 3. Simpler args for `PalletCmd::run`
You can do three things here to integrate the changes into your node:
- nothing: old code keeps working as before but emits a deprecated
warning
- delete: remove the pallet benchmarking code from your node and use the
omni-bencher instead
- patch: apply the patch below and keep using as currently. This emits a
deprecated warning at runtime, since it uses the old way to generate a
genesis state, but is the smallest change.
```patch
runner.sync_run(|config| cmd
- .run::<HashingFor<Block>, ReclaimHostFunctions>(config)
+ .run_with_spec::<HashingFor<Block>, ReclaimHostFunctions>(Some(config.chain_spec))
)
```
## 4. Maintenance Change
- `pallet-nis` get a `BenchmarkSetup` config item to prepare its
counterparty asset.
- Add percent progress print when running benchmarks.
- Dont immediately exit on benchmark error but try to run as many as
possible and print errors last.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Part of https://github.com/paritytech/polkadot-sdk/issues/226
Related https://github.com/paritytech/polkadot-sdk/issues/1833
- Deprecate `CurrencyAdapter` and introduce `FungibleAdapter`
- Deprecate `ToStakingPot` and replace usage with `ResolveTo`
- Required creating a new `StakingPotAccountId` struct that implements
`TypedGet` for the staking pot account ID
- Update parachain common utils `DealWithFees`, `ToAuthor` and
`AssetsToBlockAuthor` implementations to use `fungible`
- Update runtime XCM Weight Traders to use `ResolveTo` instead of
`ToStakingPot`
- Update runtime Transaction Payment pallets to use `FungibleAdapter`
instead of `CurrencyAdapter`
- [x] Blocked by https://github.com/paritytech/polkadot-sdk/pull/1296,
needs the `Unbalanced::decrease_balance` fix
closes#1324
#### Problem
Currently, it is possible to accidentally use inner unversioned
migration instead of `VersionedMigration` since both implement
`OnRuntimeUpgrade`.
#### Solution
With this change, we make it clear that value of `Inner` is not intended
to be used directly. It is achieved by bounding `Inner` to new trait
`UncheckedOnRuntimeUpgrade`, which has the same interface (except
`unchecked_` prefix) as `OnRuntimeUpgrade`.
#### `try-runtime` functions
Since developers can implement `try-runtime` for `Inner` value in
`VersionedMigration` and have custom logic for it, I added the same
`try-runtime` functions to `UncheckedOnRuntimeUpgrade`. I looked for a
ways to not duplicate functions, but couldn't find anything that doesn't
significantly change the codebase. So I would appreciate If you have any
suggestions to improve this
cc @liamaharon @xlc
polkadot address: 16FqwPZ8GRC5U5D4Fu7W33nA55ZXzXGWHwmbnE1eT6pxuqcT
---------
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Working towards migrating the `parity-bridges-common` repo inside
`polkadot-sdk`. This PR upgrades some dependencies in order to align
them with the versions used in `parity-bridges-common`
Related to
https://github.com/paritytech/parity-bridges-common/issues/2538
When using `schedule_code_upgrade` to change the code of a parachain in
the relay chain runtime, we had already fixed to not set the `GoAhead`
signal. This was done to not brick any parachain after the upgrade,
because they were seeing the signal without having any upgrade prepared.
The remaining problem is that the parachain code is only upgraded after
a parachain header was enacted, aka the parachain made some progress.
However, this is quite complicated if the parachain is bricked (which is
the most common scenario why to manually schedule a code upgrade). Thus,
this pull request replaces `SetGoAhead` with `UpgradeStrategy` to signal
to the logic kind of strategy want to use. The strategies are either
`SetGoAheadSignal` or `ApplyAtExpectedBlock`. `SetGoAheadSignal` sets
the go ahead signal as before and awaits a parachain block.
`ApplyAtExpectedBlock` schedules the upgrade and applies it directly at
the `expected_block` without waiting for the parachain to make any kind
of progress.
Fix "double-weights" for extrinsics, use only the ones benchmarked in
the runtime.
Deprecate extrinsics that don't specify WeightLimit, remove their usage
across the repo.
---------
Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: command-bot <>
Rejoice! Rejoice! The story is nearly over.
This PR removes stale migrations, auxiliary structures, and package
dependencies, thus making Rococo and Westend totally free from any
`im-online`-related stuff.
`im-online` still stays a part of the Substrate node and its runtime:
https://github.com/paritytech/polkadot-sdk/blob/0d9324847391e902bb42f84f0e76096b1f764efe/substrate/bin/node/runtime/src/lib.rs#L2276-L2277
I'm not sure if it makes sense to remove it from there considering that
we're not removing `im-online` from FRAME. Please share your opinion.
Runtime release 1.2 includes bumping of the ParachainHost APIs up to
v10, so let's move all the released APIs out of vstaging folder, this PR
does not include any logic changes only renaming of the modules and some
moving around.
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
This PR adds a new extrinsic `Call::restore_ledger ` gated by
`StakingAdmin` origin that restores a corrupted staking ledger. This
extrinsic will be used to recover ledgers that were affected by the
issue discussed in
https://github.com/paritytech/polkadot-sdk/issues/3245.
The extrinsic will re-write the storage items associated with a stash
account provided as input parameter. The data used to reset the ledger
can be either i) fetched on-chain or ii) partially/totally set by the
input parameters of the call.
In order to use on-chain data to restore the staking locks, we need a
way to read the current lock in the balances pallet. This PR adds a
`InspectLockableCurrency` trait and implements it in the pallet
balances. An alternative would be to tightly couple staking with the
pallet balances but that's inelegant (an example of how it would look
like in [this
branch](https://github.com/paritytech/polkadot-sdk/tree/gpestana/ledger-badstate-clean_tightly)).
More details on the type of corruptions and corresponding fixes
https://hackmd.io/DLb5jEYWSmmvqXC9ae4yRg?view#/
We verified that the `Call::restore_ledger` does fix all current
corrupted ledgers in Polkadot and Kusama. You can verify it here
https://hackmd.io/v-XNrEoGRpe7APR-EZGhOA.
**Changes introduced**
- Adds `Call::restore_ledger ` extrinsic to recover a corrupted ledger;
- Adds trait `frame_support::traits::currency::InspectLockableCurrency`
to allow external pallets to read current locks given an account and
lock ID;
- Implements the `InspectLockableCurrency` in the pallet-balances.
- Adds staking locks try-runtime checks
(https://github.com/paritytech/polkadot-sdk/issues/3751)
**Todo**
- [x] benchmark `Call::restore_ledger`
- [x] throughout testing of all ledger recovering cases
- [x] consider adding the staking locks try-runtime checks to this PR
(https://github.com/paritytech/polkadot-sdk/issues/3751)
- [x] simulate restoring all ledgers
(https://hackmd.io/Dsa2tvhISNSs7zcqriTaxQ?view) in Polkadot and Kusama
using chopsticks -- https://hackmd.io/v-XNrEoGRpe7APR-EZGhOA
Related to https://github.com/paritytech/polkadot-sdk/issues/3245
Closes https://github.com/paritytech/polkadot-sdk/issues/3751
---------
Co-authored-by: command-bot <>
This works only for collators that implement the `collator_fn` allowing
`collation-generation` subsystem to pull collations triggered on new
heads.
Also enables
`request_v2::CollationFetchingResponse::CollationWithParentHeadData` for
test adder/undying collators.
TODO:
- [x] fix tests
- [x] new tests
- [x] PR doc
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
The PR provides API for obtaining:
- the weight required to execute an XCM message,
- a list of acceptable `AssetId`s for message execution payment,
- the cost of the weight in the specified acceptable `AssetId`.
It is meant to address an issue where one has to guess how much fee to
pay for execution. Also, at the moment, a client has to guess which
assets are acceptable for fee execution payment.
See the related issue
https://github.com/paritytech/polkadot-sdk/issues/690.
With this API, a client is supposed to query the list of the supported
asset IDs (in the XCM version format the client understands), weigh the
XCM program the client wants to execute and convert the weight into one
of the acceptable assets. Note that the client is supposed to know what
program will be executed on what chains. However, having a small
companion JS library for the pallet-xcm and xtokens should be enough to
determine what XCM programs will be executed and where (since these
pallets compose a known small set of programs).
```Rust
pub trait XcmPaymentApi<Call>
where
Call: Codec,
{
/// Returns a list of acceptable payment assets.
///
/// # Arguments
///
/// * `xcm_version`: Version.
fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec<VersionedAssetId>, Error>;
/// Returns a weight needed to execute a XCM.
///
/// # Arguments
///
/// * `message`: `VersionedXcm`.
fn query_xcm_weight(message: VersionedXcm<Call>) -> Result<Weight, Error>;
/// Converts a weight into a fee for the specified `AssetId`.
///
/// # Arguments
///
/// * `weight`: convertible `Weight`.
/// * `asset`: `VersionedAssetId`.
fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
/// Get delivery fees for sending a specific `message` to a `destination`.
/// These always come in a specific asset, defined by the chain.
///
/// # Arguments
/// * `message`: The message that'll be sent, necessary because most delivery fees are based on the
/// size of the message.
/// * `destination`: The destination to send the message to. Different destinations may use
/// different senders that charge different fees.
fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
}
```
An
[example](https://gist.github.com/PraetorP/4bc323ff85401abe253897ba990ec29d)
of a client side code.
---------
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Daniel Shiposha <mrshiposha@gmail.com>
**Update:** Pushed additional changes based on the review comments.
**This pull request fixes various spelling mistakes in this
repository.**
Most of the changes are contained in the first **3** commits:
- `Fix spelling mistakes in comments and docs`
- `Fix spelling mistakes in test names`
- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`
Other source code spelling mistakes are separated into individual
commits for easier reviewing:
- `Fix the spelling of 'authority'`
- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`
- `Fix the spelling of 'prev_enqueud_messages'`
- `Fix the spelling of 'endpoint'`
- `Fix the spelling of 'children'`
- `Fix the spelling of 'PenpalSiblingSovereignAccount'`
- `Fix the spelling of 'PenpalSudoAccount'`
- `Fix the spelling of 'insufficient'`
- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`
- `Fix the spelling of 'subtracted'`
- `Fix the spelling of 'CandidatePendingAvailability'`
- `Fix the spelling of 'exclusive'`
- `Fix the spelling of 'until'`
- `Fix the spelling of 'discriminator'`
- `Fix the spelling of 'nonexistent'`
- `Fix the spelling of 'subsystem'`
- `Fix the spelling of 'indices'`
- `Fix the spelling of 'committed'`
- `Fix the spelling of 'topology'`
- `Fix the spelling of 'response'`
- `Fix the spelling of 'beneficiary'`
- `Fix the spelling of 'formatted'`
- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`
- `Fix the spelling of 'succeeded'`
- `Fix the spelling of 'reopened'`
- `Fix the spelling of 'proposer'`
- `Fix the spelling of 'InstantiationNonce'`
- `Fix the spelling of 'depositor'`
- `Fix the spelling of 'expiration'`
- `Fix the spelling of 'phantom'`
- `Fix the spelling of 'AggregatedKeyValue'`
- `Fix the spelling of 'randomness'`
- `Fix the spelling of 'defendant'`
- `Fix the spelling of 'AquaticMammal'`
- `Fix the spelling of 'transactions'`
- `Fix the spelling of 'PassingTracingSubscriber'`
- `Fix the spelling of 'TxSignaturePayload'`
- `Fix the spelling of 'versioning'`
- `Fix the spelling of 'descendant'`
- `Fix the spelling of 'overridden'`
- `Fix the spelling of 'network'`
Let me know if this structure is adequate.
**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.
~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~
~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~
~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~
**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.
~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~
Based on issue
[#2512](https://github.com/paritytech/polkadot-sdk/issues/2512), it
seems that some ecosystem teams are using these networks to set up their
staging environments and test certain use cases, some of them involving
sending XCMs from the relay with origins not allowed in the current
configuration.
This change reverts the configuration of `SendXcmOrigin`.
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
We witnessed really poor performance on Rococo, where we ended up with
50 on-demand cores. This was due to the fact that for each core the full
queue was processed. With this change full queue processing will happen
way less often (most of the time complexity is O(1) or O(log(n))) and if
it happens then only for one core (in expectation).
Also spot price is now updated before each order to ensure economic back
pressure.
TODO:
- [x] Implement
- [x] Basic tests
- [x] Add more tests (see todos)
- [x] Run benchmark to confirm better performance, first results suggest
> 100x faster.
- [x] Write migrations
- [x] Bump scale-info version and remove patch in Cargo.toml
- [x] Write PR docs: on-demand performance improved, more on-demand
cores are now non problematic anymore. If need by also the max queue
size can be increased again. (Maybe not to 10k)
Optional: Performance can be improved even more, if we called
`pop_assignment_for_core()`, before calling `report_processed` (Avoid
needless affinity drops). The effect gets smaller the larger the claim
queue and I would only go for it, if it does not add complexity to the
scheduler.
---------
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: antonva <anton.asgeirsson@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
Co-authored-by: ordian <write@reusable.software>
The PR adds two things:
1. Runtime API exposing the whole claim queue
2. Consumes the API in `collation-generation` to fetch the next
scheduled `ParaEntry` for an occupied core.
Related to https://github.com/paritytech/polkadot-sdk/issues/1797
Introduces `CryptoBytes` type defined as:
```rust
pub struct CryptoBytes<const N: usize, Tag = ()>(pub [u8; N], PhantomData<fn() -> Tag>);
```
The type implements a bunch of methods and traits which are typically
expected from a byte array newtype
(NOTE: some of the methods and trait implementations IMO are a bit
redundant, but I decided to maintain them all to not change too much
stuff in this PR)
It also introduces two (generic) typical consumers of `CryptoBytes`:
`PublicBytes` and `SignatureBytes`.
```rust
pub struct PublicTag;
pub PublicBytes<const N: usize, CryptoTag> = CryptoBytes<N, (PublicTag, CryptoTag)>;
pub struct SignatureTag;
pub SignatureBytes<const N: usize, CryptoTag> = CryptoBytes<N, (SignatureTag, CryptoTag)>;
```
Both of them use a tag to differentiate the two types at a higher level.
Downstream specializations will further specialize using a dedicated
crypto tag. For example in ECDSA:
```rust
pub struct EcdsaTag;
pub type Public = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
pub type Signature = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
```
Overall we have a cleaner and most importantly **consistent** code for
all the types involved
All these details are opaque to the end user which can use `Public` and
`Signature` for the cryptos as before
Part of #3326
cc @kianenigma @ggwpez @liamaharon
polkadot address: 12poSUQPtcF1HUPQGY3zZu2P8emuW9YnsPduA4XG3oCEfJVp
---------
Signed-off-by: Matteo Muraca <mmuraca247@gmail.com>
Currently the xcm-executor returns an `Unimplemented` error if it
receives any HRMP-related instruction.
What I propose here, which is what we are currently doing in our forked
executor at polimec, is to introduce a trait implemented by the executor
which will handle those instructions.
This way, if parachains want to keep the default behavior, they just use
`()` and it will return unimplemented, but they can also implement their
own logic to establish HRMP channels with other chains in an automated
fashion, without requiring to go through governance.
Our implementation is mentioned in the [polkadot HRMP
docs](https://arc.net/l/quote/hduiivbu), and it was suggested to us to
submit a PR to add these changes to polkadot-sdk.
---------
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: command-bot <>
Crowdloan account should burn all funds after a crowd loan got dissolved
to ensure that the account is reaped correctly.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Francisco Aguirre