This is a refactoring, no changes to the logic are included (if you find
some, report :D).
## Change Overview
In https://github.com/paritytech/polkadot-sdk/issues/2455, dependency on
actual runtimes was removed for the system parachains. This means that
the trait bounds we have on various `start_node_xy` do not check
anything anymore, since they all point to the same runtime. Exception is
asset-hub-polkadot which uses a different key type.
This PR unifies the different nodes as much as possible.
`start_node_impl` is doing the heavy lifting and has been made a bit
more flexible to support the rpc extension instantiation that was there
before.
The generics for `Runtime` and `AuraId` have been removed where
possible. The fake runtime is imported as `FakeRuntime` to make it very
clear to readers that it is not the generic parameter.
Multiple nodes where using the same import queue/start_consensus
closure, they have been unified.
---------
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Andrei Eres <eresav@me.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Alin Dima <alin@parity.io>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Dónal Murray <donalm@seadanda.dev>
Deprecate the `xcm::body::TREASURER_INDEX` constant and use the standard
Treasury variant from the `xcm::BodyId` type instead.
To align with the production runtimes:
https://github.com/polkadot-fellows/runtimes/pull/149
Currently, pool member funds cannot be unbonded if the depositor's stake
is less than `MinNominatorBond`. This usually happens after
`T::MinNominatorBond` is increased.
To fix this, the above mentioned condition is added as a case for
permissionless dispatch of `chill`. After pool is chilled, pool members
can unbond their funds since pool won't be nominating anymore.
Consequently, same check is added to `nominate` call, i.e pool can not
start nominating if it's depositor does not have `MinNominatorBond`
cc @Ank4n @kianenigma
closes#2350
Polkadot address: 16FqwPZ8GRC5U5D4Fu7W33nA55ZXzXGWHwmbnE1eT6pxuqcT
---------
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>
Changes the way we perform the random selection of backed candidates
when there isn't enough room for all of them. Instead of picking
individual backed candidates `apply_weight` now operates on chains of
candidates. This is fully backwards compatible and relies on the node
side (provisioner/prospective parachains) doing the heavy lifting and
providing the candidates in the order they form a chain.
The same approach can be implemented for bitfields random selection once
https://github.com/paritytech/polkadot-sdk/pull/3479 is merged.
The approach taken in this PR aims for reduced additional complexity at
the cost of being less fair wrt how many backed candidates from each
chain are picked. It favors elastic scaling parachains vs parachains not
using elastic scaling, but from my perspective it should be fine as this
should happen under exceptional circumstances like dispute storms.
Note: to make things more fair we can consider specializing `random_sel`
such that it will try to pick candidates one by one in the order
provided by the provisioner such that non elastic scaling parachains
have the same chance of getting a candidate backed.
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
In order to prepare for PolkaVM support I removed the wat support from
our test fixture crate.
- Removed redundant tests (invalid module checks are already inside the
prepare module where they belong
- Converted the gas_sync tests to Rust
- Moved the start function test to the `wasm` module
The current `penpal` runtime utilizes the `EthereumLocation` parameter,
which is employed for XCM emulated integration tests concerning the
Rococo <> ETH bridge. It includes a hard-coded chainId for the Ethereum
testnet utilized in Rococo. The `EthereumLocation` serves the purpose of
aligning with the `TrustedReserves`. However, due to this hard-coded
configuration, reusing `penpal` for testing various environments such as
Kusama/Polkadot versus Ethereum bridge becomes unfeasible.
This PR introduces the capability to easily customize the asset location
for `TrustedReserves` without needing to know anything about Ethereum.
## TODO
- [x] fix integration tests with
`System::set_storage(CustomizableAssetFromSystemAssetHub::key(),
<whatever-location-is-needed>)` @claravanstaden
- [ ] ~~maybe add some helper function/macro to support `set_storage`
for other runtimes (that we could reuse)~~
- [ ] Release patch for: `penpal-runtime` + emulated crate with
`set_storage` support (if needed)
- [ ] backport to 1.7.0
- [ ] backport to 1.8.0
---------
Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
This PR extends the Initialized event of the chainHead_follow
subscription.
Now, the event provides multiple finalized block hashes. This
information allows clients that are disconnected, and that want to
reconnect, to not lose information about the state of the chain.
At the moment, the spec encourages servers to provide at least 1 minute
of finalized blocks (~10 blocks). The users are responsible for
unpinning these blocks at a later time. This PR tries to report at least
1 finalized block and at most 16 blocks, if they are available.
Closes: https://github.com/paritytech/polkadot-sdk/issues/3432
cc @paritytech/subxt-team
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Changes:
- `QueueFootprint` gets a new field; `ready_pages` that contains the
non-overweight and not yet processed pages.
- `XCMP` queue pallet is change to use the `ready_pages` instead of
`pages` to calculate the channel suspension thresholds.
This should give the XCMP queue pallet a more correct view of when to
suspend channels.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
# Description
Removed deprecated type `GenesisConfig` from the codebase.
Closes https://github.com/paritytech/polkadot-sdk/issues/175
# Checklist
- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [x] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
required)
- [x] I have made corresponding changes to the documentation (if
applicable)
---------
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
The first step towards
https://github.com/paritytech/polkadot-sdk/issues/3155
Brings all templates under the following structure
```
templates
| parachain
| | polkadot-launch
| | runtime --> parachain-template-runtime
| | pallets --> pallet-parachain-template
| | node --> parachain-template-node
| minimal
| | runtime --> minimal-template-runtime
| | pallets --> pallet-minimal-template
| | node --> minimal-template-node
| solochain
| | runtime --> solochain-template-runtime
| | pallets --> pallet-template (the naming is not consistent here)
| | node --> solochain-template-node
```
The only note-worthy changes in this PR are:
- More `Cargo.toml` fields are forwarded to use the one from the
workspace.
- parachain template now has weights and benchmarks
- adds a shell pallet to the minimal template
- remove a few unused deps
A list of possible follow-ups:
- [ ] Unify READMEs, create a parent README for all
- [ ] remove references to `docs.substrate.io` in templates
- [ ] make all templates use `#[derive_impl]`
- [ ] update and unify all licenses
- [ ] Remove polkadot launch, use
https://github.com/paritytech/polkadot-sdk/blob/35349df993ea2e7c4769914ef5d199e787b23d4c/cumulus/zombienet/examples/small_network.toml
instead.
# Description
*Deletes `testing.md` file in accordance with the discussion on issue
#2527.* Old references to Gurke or simnet have been removed.
Fixes#2527
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
After some discussion with @kogeler after the we added the rate-limit
middleware it may slow down
the rpc call timings metrics significantly because it works as follows:
1. The rate limit guard is checked when the call comes and if a slot is
available -> process the call
2. If no free spot is available then the call will be sleeping
`jitter_delay + min_time_rate_guard` then woken up and checked at most
ten times
3. If no spot is available after 10 iterations -> the call is rejected
(this may take tens of seconds)
Thus, this PR adds a label "is_rate_limited" to filter those out on the
metrics "substrate_rpc_calls_time" and "substrate_rpc_calls_finished".
I had to merge two middleware layers Metrics and RateLimit to avoid
shared state in a hacky way.
---------
Co-authored-by: James Wilson <james@jsdw.me>
Closes#2160
First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)
Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.
Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)
Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
- Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
- Signed transactions (with a hardcoded signature).
- General transactions (without a hardcoded signature).
`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.
There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).
Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.
For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.
## Code Migration
### NOW: Getting it to build
Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:
```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
/* snip */
MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```
After:
```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
/* snip */
AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```
You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:
```rust
fn construct_extrinsic(
/* snip */
) -> UncheckedExtrinsic {
let extra: SignedExtra = (
/* snip */
MySpecialSignedExtension::new(/* snip */),
);
let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
let signature = payload.using_encoded(|e| sender.sign(e));
UncheckedExtrinsic::new_signed(
/* snip */
Signature::Sr25519(signature),
extra,
)
}
```
After:
```rust
fn construct_extrinsic(
/* snip */
) -> UncheckedExtrinsic {
let tx_ext: TxExtension = (
/* snip */
MySpecialSignedExtension::new(/* snip */).into(),
);
let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
let signature = payload.using_encoded(|e| sender.sign(e));
UncheckedExtrinsic::new_signed(
/* snip */
Signature::Sr25519(signature),
tx_ext,
)
}
```
### SOON: Migrating to `TransactionExtension`
Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.
- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.
#### `TransactionExtensionBase`
This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.
#### `TransactionExtension`
Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.
Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.
Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)
Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.
If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.
## TODO
- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
- [x] `ChargeTransactionPayment`
- [x] `DummyExtension`
- [x] `ChargeAssetTxPayment` (asset-tx-payment)
- [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
- [x] `CheckWeight`
- [x] `CheckTxVersion`
- [x] `CheckSpecVersion`
- [x] `CheckNonce`
- [x] `CheckNonZeroSender`
- [x] `CheckMortality`
- [x] `CheckGenesis`
- [x] `CheckOnlySudoAccount`
- [x] `WatchDummy`
- [x] `PrevalidateAttests`
- [x] `GenericSignedExtension`
- [x] `SignedExtension` (chain-polkadot-bulletin)
- [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
- [ ]
- [x] Fix any broken tests.
---------
Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Marcin S. <marcin@realemail.net>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Julian Eager <eagr@tutanota.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: yjh <yjh465402634@gmail.com>
Co-authored-by: Tom Mi <tommi@niemi.lol>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
Co-authored-by: PG Herveou <pgherveou@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: bader y <ibnbassem@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Parth <desaiparth08@gmail.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Jonathan Udd <jonathan@dwellir.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Evgeny Snitko <evgeny@parity.io>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro>
Co-authored-by: zhiqiangxu <652732310@qq.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Anwesh <anweshknayak@gmail.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com>
Co-authored-by: Muharem <ismailov.m.h@gmail.com>
Co-authored-by: joepetrowski <joe@parity.io>
Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com>
Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: command-bot <>
Step in https://github.com/paritytech/polkadot-sdk/issues/171
This PR removes the need to specify `as [disambiguation_path]` for cases
where the trait definition resides within the same scope as default impl
path.
For example, in the following macro invocation
```rust
#[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::Config for Runtime {
...
}
```
the trait `DefaultConfig` lies within the `frame_system` scope and
`TestDefaultConfig` impls the `DefaultConfig` trait. Using this
information, we can compute the disambiguation path internally, thus
removing the need of an explicit specification.
In cases where the trait lies outside this scope, we would still need to
specify it explicitly, but this should take care of most (if not all)
uses of `derive_impl` within FRAME's context.
https://github.com/paritytech/polkadot-sdk/issues/3130
builds on top of https://github.com/paritytech/polkadot-sdk/pull/3160
Processes the availability cores and builds a record of how many
candidates it should request from prospective-parachains and their
predecessors.
Tries to supply as many candidates as the runtime can back. Note that
the runtime changes to back multiple candidates per para are not yet
done, but this paves the way for it.
The following backing/inclusion policy is assumed:
1. the runtime will never back candidates of the same para which don't
form a chain with the already backed candidates. Even if the others are
still pending availability. We're optimistic that they won't time out
and we don't want to back parachain forks (as the complexity would be
huge).
2. if a candidate is timed out of the core before being included, all of
its successors occupying a core will be evicted.
3. only the candidates which are made available and form a chain
starting from the on-chain para head may be included/enacted and cleared
from the cores. In other words, if para head is at A and the cores are
occupied by B->C->D, and B and D are made available, only B will be
included and its core cleared. C and D will remain on the cores awaiting
for C to be made available or timed out. As point (2) above already
says, if C is timed out, D will also be dropped.
4. The runtime will deduplicate candidates which form a cycle. For
example if the provisioner supplies candidates A->B->A, the runtime will
only back A (as the state output will be the same)
Note that if a candidate is timed out, we don't guarantee that in the
next relay chain block the block author will be able to fill all of the
timed out cores of the para. That increases complexity by a lot.
Instead, the provisioner will supply N candidates where N is the number
of candidates timed out, but doesn't include their successors which will
be also deleted by the runtime. This'll be backfilled in the next relay
chain block.
Adjacent changes:
- Also fixes: https://github.com/paritytech/polkadot-sdk/issues/3141
- For non prospective-parachains, don't supply multiple candidates per
para (we can't have elastic scaling without prospective parachains
enabled). paras_inherent should already sanitise this input but it's
more efficient this way.
Note: all of these changes are backwards-compatible with the
non-elastic-scaling scenario (one core per para).
### What's been done
- `subsystem-bench` has been split into two parts: a cli benchmark
runner and a library.
- The cli runner is quite simple. It just allows us to run `.yaml` based
test sequences. Now it should only be used to run benchmarks during
development.
- The library is used in the cli runner and in regression tests. Some
code is changed to make the library independent of the runner.
- Added first regression tests for availability read and write that
replicate existing test sequences.
### How we run regression tests
- Regression tests are simply rust integration tests without the
harnesses.
- They should only be compiled under the `subsystem-benchmarks` feature
to prevent them from running with other tests.
- This doesn't work when running tests with `nextest` in CI, so
additional filters have been added to the `nextest` runs.
- Each benchmark run takes a different time in the beginning, so we
"warm up" the tests until their CPU usage differs by only 1%.
- After the warm-up, we run the benchmarks a few more times and compare
the average with the exception using a precision.
### What is still wrong?
- I haven't managed to set up approval voting tests. The spread of their
results is too large and can't be narrowed down in a reasonable amount
of time in the warm-up phase.
- The tests start an unconfigurable prometheus endpoint inside, which
causes errors because they use the same 9999 port. I disable it with a
flag, but I think it's better to extract the endpoint launching outside
the test, as we already do with `valgrind` and `pyroscope`. But we still
use `prometheus` inside the tests.
### Future work
* https://github.com/paritytech/polkadot-sdk/issues/3528
* https://github.com/paritytech/polkadot-sdk/issues/3529
* https://github.com/paritytech/polkadot-sdk/issues/3530
* https://github.com/paritytech/polkadot-sdk/issues/3531
---------
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
If an XCM execution fails or ends with leftover assets, these will be
trapped.
In order to claim them, a custom XCM has to be executed, with the
`ClaimAsset` instruction.
However, arbitrary XCM execution is not allowed everywhere yet and XCM
itself is still not easy enough to use for users out there with trapped
assets.
This new extrinsic in `pallet-xcm` will allow these users to easily
claim their assets, without concerning themselves with writing arbitrary
XCMs.
Part of fixing https://github.com/paritytech/polkadot-sdk/issues/3495
---------
Co-authored-by: command-bot <>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
This fixes an issue introduced in
https://github.com/paritytech/substrate/pull/14101, in which I removed
the `Call` enum's documentation and replaced it with a link to the
`Pallet` struct, but this also removed any docs related to call from the
metadata.
I tried to add a regression test for this, but it seems to me that this
is not possible, given that using `type-info` we only assert in type-ids
for `Call`, `Event` and `Error`. I removed some doc comments from a test
setup in `frame-support-test` to demonstrate the issue there. @jsdw do
you have any comments on this?
I also fixed a small issue in the custom html/css of `polkadot-sdk-doc`
crate, making sure it does not affect the rust-doc page of all other
crates.
- [x] Investigate a regression test
- [x] prdoc
If approval was in progress we didn't actually restart it, so we end up
in a situation where we distribute our assignment, but we don't
distribute any approval.
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Fixing:
```
Verification failed for block 0x07bbf1e04121d70a4bdb21cc055132b53ac2390fa95c4d05497fc91b1e8bf7f5 received from (12D3KooWJzLd8skcAgA24EcJey7aJAhYctfUxWGjSP5Usk9wbpPZ): "Header 0x07bbf1e04121d70a4bdb21cc055132b53ac2390fa95c4d05497fc91b1e8bf7f5 rejected: too far in the future"
```
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
This MR is the merge of
https://github.com/paritytech/substrate/pull/14414 and
https://github.com/paritytech/substrate/pull/14275. It implements
[RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes
https://github.com/paritytech/polkadot-sdk/issues/198.
-----
This Merge request introduces three major topicals:
1. Multi-Block-Migrations
1. New pallet `poll` hook for periodic service work
1. Replacement hooks for `on_initialize` and `on_finalize` in cases
where `poll` cannot be used
and some more general changes to FRAME.
The changes for each topical span over multiple crates. They are listed
in topical order below.
# 1.) Multi-Block-Migrations
Multi-Block-Migrations are facilitated by creating `pallet_migrations`
and configuring `System::Config::MultiBlockMigrator` to point to it.
Executive picks this up and triggers one step of the migrations pallet
per block.
The chain is in lockdown mode for as long as an MBM is ongoing.
Executive does this by polling `MultiBlockMigrator::ongoing` and not
allowing any transaction in a block, if true.
A MBM is defined through trait `SteppedMigration`. A condensed version
looks like this:
```rust
/// A migration that can proceed in multiple steps.
pub trait SteppedMigration {
type Cursor: FullCodec + MaxEncodedLen;
type Identifier: FullCodec + MaxEncodedLen;
fn id() -> Self::Identifier;
fn max_steps() -> Option<u32>;
fn step(
cursor: Option<Self::Cursor>,
meter: &mut WeightMeter,
) -> Result<Option<Self::Cursor>, SteppedMigrationError>;
}
```
`pallet_migrations` can be configured with an aggregated tuple of these
migrations. It then starts to migrate them one-by-one on the next
runtime upgrade.
Two things are important here:
- 1. Doing another runtime upgrade while MBMs are ongoing is not a good
idea and can lead to messed up state.
- 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`,
otherwise it is not used.**
The pallet supports an `UpgradeStatusHandler` that can be used to notify
external logic of upgrade start/finish (for example to pause XCM
dispatch).
Error recovery is very limited in the case that a migration errors or
times out (exceeds its `max_steps`). Currently the runtime dev can
decide in `FailedMigrationHandler::failed` how to handle this. One
follow-up would be to pair this with the `SafeMode` pallet and enact
safe mode when an upgrade fails, to allow governance to rescue the
chain. This is currently not possible, since governance is not
`Mandatory`.
## Runtime API
- `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to
inform the Block Author whether they can push transactions.
### Integration
Add it to your runtime implementation of `Core` and `BlockBuilder`:
```patch
diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs
@@ impl_runtime_apis! {
impl sp_block_builder::Core<Block> for Runtime {
- fn initialize_block(header: &<Block as BlockT>::Header) {
+ fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode {
Executive::initialize_block(header)
}
...
}
```
# 2.) `poll` hook
A new pallet hook is introduced: `poll`. `Poll` is intended to replace
mostly all usage of `on_initialize`.
The reason for this is that any code that can be called from
`on_initialize` cannot be migrated through an MBM. Currently there is no
way to statically check this; the implication is to use `on_initialize`
as rarely as possible.
Failing to do so can result in broken storage invariants.
The implementation of the poll hook depends on the `Runtime API` changes
that are explained above.
# 3.) Hard-Deadline callbacks
Three new callbacks are introduced and configured on `System::Config`:
`PreInherents`, `PostInherents` and `PostTransactions`.
These hooks are meant as replacement for `on_initialize` and
`on_finalize` in cases where the code that runs cannot be moved to
`poll`.
The reason for this is to make the usage of HD-code (hard deadline) more
explicit - again to prevent broken invariants by MBMs.
# 4.) FRAME (general changes)
## `frame_system` pallet
A new memorize storage item `InherentsApplied` is added. It is used by
executive to track whether inherents have already been applied.
Executive and can then execute the MBMs directly between inherents and
transactions.
The `Config` gets five new items:
- `SingleBlockMigrations` this is the new way of configuring migrations
that run in a single block. Previously they were defined as last generic
argument of `Executive`. This shift is brings all central configuration
about migrations closer into view of the developer (migrations that are
configured in `Executive` will still work for now but is deprecated).
- `MultiBlockMigrator` this can be configured to an engine that drives
MBMs. One example would be the `pallet_migrations`. Note that this is
only the engine; the exact MBMs are injected into the engine.
- `PreInherents` a callback that executes after `on_initialize` but
before inherents.
- `PostInherents` a callback that executes after all inherents ran
(including MBMs and `poll`).
- `PostTransactions` in symmetry to `PreInherents`, this one is called
before `on_finalize` but after all transactions.
A sane default is to set all of these to `()`. Example diff suitable for
any chain:
```patch
@@ impl frame_system::Config for Test {
type MaxConsumers = ConstU32<16>;
+ type SingleBlockMigrations = ();
+ type MultiBlockMigrator = ();
+ type PreInherents = ();
+ type PostInherents = ();
+ type PostTransactions = ();
}
```
An overview of how the block execution now looks like is here. The same
graph is also in the rust doc.
<details><summary>Block Execution Flow</summary>
<p>
</p>
</details>
## Inherent Order
Moved to https://github.com/paritytech/polkadot-sdk/pull/2154
---------------
## TODO
- [ ] Check that `try-runtime` still works
- [ ] Ensure backwards compatibility with old Runtime APIs
- [x] Consume weight correctly
- [x] Cleanup
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
While adding runtime tests to
https://github.com/polkadot-fellows/runtimes/pull/130, I noticed the
Ethereum chain ID was hardcoded. For Kusama + Polkadot, the Ethereum
chain ID should 1 (Mainnet), whereas on Rococo it is 11155111 (Sepolia).
This PR also updates the Snowbridge crates versions to the current
versions on crates.io.
---------
Co-authored-by: claravanstaden <Cats 4 life!>
Changes:
- Add an optional `bump` field to the crates in a prdoc.
- Explain the cargo semver interpretation for <1 versions in the release
doc.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Sebastian Kunert