Using taplo, fixes all our broken and inconsistent toml formatting and
adds CI to keep them tidy.
If people want we can customise the format rules as described here
https://taplo.tamasfe.dev/configuration/formatter-options.html
@ggwpez, I suggest zepter is used only for checking features are
propagated, and leave formatting for taplo to avoid duplicate work and
conflicts.
TODO
- [x] Use `exclude = [...]` syntax in taplo file to ignore zombienet
tests instead of deleting the dir
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
This PR provides the infrastructure for the pov-reclaim mechanism
discussed in #209. The goal is to provide the current proof size to the
runtime so it can be used to reclaim storage weight.
## New Host Function
- A new host function is provided
[here](https://github.com/skunert/polkadot-sdk/blob/5b317fda3be205f4136f10d4490387ccd4f9765d/cumulus/primitives/pov-reclaim/src/lib.rs#L23).
It returns the size of the current proof size to the runtime. If
recording is not enabled, it returns 0.
## Implementation Overview
- Implement option to enable proof recording during import in the
client. This is currently enabled for `polkadot-parachain`,
`parachain-template` and the cumulus test node.
- Make the proof recorder ready for no-std. It was previously only
enabled for std environments, but we need to record the proof size in
`validate_block` too.
- Provide a recorder implementation that only the records the size of
incoming nodes and does not store the nodes itself.
- Fix benchmarks that were broken by async backing changes
- Provide new externalities extension that is registered by default if
proof recording is enabled.
- I think we should discuss the naming, pov-reclaim was more intuitive
to me, but we could also go with clawback like in the issue.
## Impact of proof recording during import
With proof recording: 6.3058 Kelem/s
Without proof recording: 6.3427 Kelem/s
The measured impact on the importing performance is quite low on my
machine using the block import benchmark. With proof recording I am
seeing a performance hit of 0.585%.
---------
Co-authored-by: command-bot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
see #2189
This PR does the following:
- Bring the user api functions into a new pallet-contracts-uapi (They
are currently defined in ink!
[here])(https://github.com/paritytech/ink/blob/master/crates/env/src/engine/on_chain/ext.rs)
- Add older api versions and unstable to the user api trait.
- Remove pallet-contracts-primitives and bring the types it defined in
uapi / pallet-contracts
- Add the infrastructure to build fixtures from Rust files and test it
works by replacing `dummy.wat` and `call.wat`
- Move all the doc from wasm/runtime.rs to pallet-contracts-uapi.
This will be done in a follow up:
- convert the rest of the test from .wat to rust
- bring risc-v uapi up to date with wasm
- finalize the uapi host fns, making sure everything is codegen from the
source host fns in pallet-contracts
---------
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Remove deprecated `AllPalletsXY` types.
They have been deprecated for nearly 1.5 years now, I think its fine to
remove them.
If anyone feels like we should first put a date on the deprecation as
stated in the deprecation guideline, feel free to speak up. To me it
looks like this has been forgotten and can be directly removed.
This commit introduces a new concept called `NotificationService` which
allows Polkadot protocols to communicate with the underlying
notification protocol implementation directly, without routing events
through `NetworkWorker`. This implies that each protocol has its own
service which it uses to communicate with remote peers and that each
`NotificationService` is unique with respect to the underlying
notification protocol, meaning `NotificationService` for the transaction
protocol can only be used to send and receive transaction-related
notifications.
The `NotificationService` concept introduces two additional benefits:
* allow protocols to start using custom handshakes
* allow protocols to accept/reject inbound peers
Previously the validation of inbound connections was solely the
responsibility of `ProtocolController`. This caused issues with light
peers and `SyncingEngine` as `ProtocolController` would accept more
peers than `SyncingEngine` could accept which caused peers to have
differing views of their own states. `SyncingEngine` would reject excess
peers but these rejections were not properly communicated to those peers
causing them to assume that they were accepted.
With `NotificationService`, the local handshake is not sent to remote
peer if peer is rejected which allows it to detect that it was rejected.
This commit also deprecates the use of `NetworkEventStream` for all
notification-related events and going forward only DHT events are
provided through `NetworkEventStream`. If protocols wish to follow each
other's events, they must introduce additional abtractions, as is done
for GRANDPA and transactions protocols by following the syncing protocol
through `SyncEventStream`.
Fixes https://github.com/paritytech/polkadot-sdk/issues/512
Fixes https://github.com/paritytech/polkadot-sdk/issues/514
Fixes https://github.com/paritytech/polkadot-sdk/issues/515
Fixes https://github.com/paritytech/polkadot-sdk/issues/554
Fixes https://github.com/paritytech/polkadot-sdk/issues/556
---
These changes are transferred from
https://github.com/paritytech/substrate/pull/14197 but there are no
functional changes compared to that PR
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Sets `frame_system::LastRuntimeUpgrade` after running try-runtime
migrations to better emulate real behavior.
This fixes an issue where migrations using the spec version to determine
whether to execute can incorrectly fail idempotency checks.
@s0me0ne-unkn0wn noticed this issue with the session key migration
introduced in https://github.com/paritytech/polkadot-sdk/pull/2265.
Addresses #409.
This request has been raised by multiple community members - the ability
for the nomination pool root role to configure permissionless commission
claiming:
> Would it be possible to have a claim_commission_other extrinsic for
claiming commission of nomination pools permissionless?
This PR does not quite introduce this additional call, but amends
`do_claim_commission` to check a new `claim_permission` field in the
`Commission` struct, configured by an enum:
```
enum CommissionClaimPermission {
Permissionless,
Account(AccountId),
}
```
This can be optionally set in a bonded pool's
`commission.claim_permission` field:
```
struct BondedPool {
commission: {
<snip>
claim_permission: Option<CommissionClaimPermission<T::AccountId>>,
},
<snip>
}
```
This is a new field and requires a migration to add it to existing
pools. This will be `None` on pool creation, falling back to the `root`
role having sole access to claim commission if it is not set; this is
the behaviour as it is today. Once set, the field _can_ be set to `None`
again.
#### Changes
- [x] Add `commision.claim_permission` field.
- [x] Add `can_claim_commission` and amend `do_claim_commission`.
- [x] Add `set_commission_claim_permission` call.
- [x] Test to cover new configs and call.
- [x] Add and amend benchmarks.
- [x] Generate new weights + slot into call
`set_commission_claim_permission`.
- [x] Add migration to introduce `commission.claim_permission`, bump
storage version.
- [x] Update Westend weights.
- [x] Migration working.
---------
Co-authored-by: command-bot <>
The `chill_other` call is the only staking call that explicitly requires
`controller` in its signature. This PR changes the controller arg to be
the stash instead, with `StakingLedger` then fetching the controller
from storage.
This is not a breaking change per se - the call types do not change, but
is noteworthy as UIs will now want to pass the stash account into
`chill_other` calls, & metadata will reflect this.
Note: This is very low impact. `chill_other` has [hardly ever been
used](https://polkadot.subscan.io/extrinsic?address=&module=staking&call=chill_other&result=all&signedChecked=signed%20only&startDate=&endDate=&startBlock=&timeType=date&version=9431&endBlock=)
on Polkadot - notwithstanding the one called 11 days ago at block
18177457 that was a part of test I did, the last call was made 493 days
ago. Only 2 calls have ever been successful.
Addresses controller deprecation #2500
---------
Co-authored-by: command-bot <>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
The FastUnstake pallet tests were previously directly modifying storage
items in the pallet-staking to alter exposure. However, due to the
introduction of the [new paged exposure
feature](https://github.com/paritytech/polkadot-sdk/pull/1189) these
tests were not testing against correct storage items. This issue
resulted in a bug that I didn't catch, which has been addressed in [this
fix](https://github.com/paritytech/polkadot-sdk/pull/2369).
This PR introduces a modification to how the pallet-fast-unstake handles
exposure. It now utilizes `pallet-staking::EraInfo` to set or mutate
Exposures.
Amends some staking pallet docs, and deprecation comment, to adjust to
the latest controller deprecation state.
Note, do we need the `README.md` file, which is a duplicate of the
pallet docs? Docs would be easier to maintain, and less ambiguity for
devs to refer to, if we had one source of truth in the generated pallet
docs.
This moves the macro related re-exports to `__private` to make it more
obvious for downstream users that they are using an internal api.
---------
Co-authored-by: command-bot <>
Westend Asset Hub currently failing because the spec name is being
changed next runtime upgrade
(https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4413125).
This also fixes an idempotency issue with a staking pallet migration.
Similar issues will be caught automatically now that we've also updated
to try-runtime-cli v0.5.0 which checks for idempotency issues.
This also enables try-state checks running in the CI.
The `SkipCheckifFeeless::IDENTIFIER` became transparent (ie was whatever
the inner signed ext was). This PR just makes the `TypeInfo` transparent
too, so that libraries that use said info to decode the data (ie subxt)
can behave identically whether or not the `SkipCheckifFeeless` wrapper
is used or not.
---------
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Fully-qualified path was not being used in benchmark macro for one of
the cases. This PR fixes this and removes the unnecessary import in a
couple of files, which I believe was done to fix this issue.
Deprecates `RewardDestination::Controller` variant.
- [x] `RewardDestination::Controller` annotated with `#[deprecated]`.
- [x] `Controller` variant is now handled the same way as `Stash` in
`payout_stakers`.
- [x] `set_payee` errors if `RewardDestination::Controller` is provided.
- [x] Added `update_payee` call to lazily migrate
`RewardDestination::Controller` `Payee` storage entries to
`RewardDestination::Account(controller)` .
- [x] `payout_stakers_dead_controller` has been removed from benches &
weights - was not used.
- [x] Tests no longer use `RewardDestination::Controller`.
---------
Co-authored-by: command-bot <>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
The `lazy_static` package does not work well in `no-std`: it requires
`spin_no_std` feature, which also will propagate into `std` if enabled.
This is not what we want.
This PR provides simple address uri parser which allows to get rid of
_regex_ which was used to parse the address uri, what in turns allows to
remove lazy_static.
Three regular expressions
(`SS58_REGEX`,`SECRET_PHRASE_REGEX`,`JUNCTION_REGEX`) were replaced with
the parser which unifies all of them.
The new parser does not support Unicode, it is ASCII only.
Related to: #2044
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: command-bot <>
Addresses a bug caused by
https://github.com/paritytech/polkadot-sdk/pull/1189. The changes are
still not released yet, so would like to push the fix soon so it can go
together with the release of the above PR.
`fast_unstake` checks if a staker is exposed in an era. However, this fn
is still returning whether the staker is exposed based on the old
storage item. This PR fixes that by looking in both old and new exposure
storages.
Also adds some integrity tests for paged exposures.
Adds a function for querying the last runtime upgrade spec version. This
can be useful for when writing runtime level migrations to ensure that
they are not executed multiple times. An example would be a session key
migration.
---------
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
The goal of this PR is to migrate Identity deposits from the Relay Chain
to a system parachain.
The problem I want to solve is that `IdentityOf` and `SubsOf` both store
an amount that's held in reserve as a storage deposit. When migrating to
a parachain, we can take a snapshot of the actual `IdentityInfo` and
sub-account mappings, but should migrate (off chain) the `deposit`s to
zero, since the chain (and by extension, accounts) won't have any funds
at genesis.
The good news is that we expect parachain deposits to be significantly
lower (possibly 100x) on the parachain. That is, a deposit of 21 DOT on
the Relay Chain would need 0.21 DOT on a parachain. This PR proposes to
migrate the deposits in the following way:
1. Introduces a new pallet with two extrinsics:
- `reap_identity`: Has a configurable `ReapOrigin`, which would be set
to `EnsureSigned` on the Relay Chain (i.e. callable by anyone) and
`EnsureRoot` on the parachain (we don't want identities reaped from
there).
- `poke_deposit`: Checks what deposit the pallet holds (at genesis,
zero) and attempts to update the amount based on the calculated deposit
for storage data.
2. `reap_identity` clears all storage data for a `target` account and
unreserves their deposit.
3. A `ReapIdentityHandler` teleports the necessary DOT to the parachain
and calls `poke_deposit`. Since the parachain deposit is much lower, and
was just unreserved, we know we have enough.
One awkwardness I ran into was that the XCMv3 instruction set does not
provide a way for the system to teleport assets without a fee being
deducted on reception. Users shouldn't have to pay a fee for the system
to migrate their info to a more efficient location. So I wrote my own
program and did the `InitiateTeleport` accounting on my own to send a
program with `UnpaidExecution`. Have discussed an
`InitiateUnpaidTeleport` instruction with @franciscoaguirre . Obviously
any chain executing this would have to pass a `Barrier` for free
execution.
TODO:
- [x] Confirm People Chain ParaId
- [x] Confirm People Chain deposit rates (determined in
https://github.com/paritytech/polkadot-sdk/pull/2281)
- [x] Add pallet to Westend
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
This PR introduces:
- XCM host functions `xcm_send`, `xcm_execute`
- An Xcm trait into the config. that proxy these functions to to
`pallet_xcm`, or disable their usage by using `()`.
- A mock_network and xcm_test files to test the newly added xcm-related
functions.
---------
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Fixes https://github.com/paritytech/polkadot-sdk/issues/1725
This PR adds the following changes:
1. An attribute `pallet::feeless_if` that can be optionally attached to
a call like so:
```rust
#[pallet::feeless_if(|_origin: &OriginFor<T>, something: &u32| -> bool {
*something == 0
})]
pub fn do_something(origin: OriginFor<T>, something: u32) -> DispatchResult {
....
}
```
The closure passed accepts references to arguments as specified in the
call fn. It returns a boolean that denotes the conditions required for
this call to be "feeless".
2. A signed extension `SkipCheckIfFeeless<T: SignedExtension>` that
wraps a transaction payment processor such as
`pallet_transaction_payment::ChargeTransactionPayment`. It checks for
all calls annotated with `pallet::feeless_if` to see if the conditions
are met. If so, the wrapped signed extension is not called, essentially
making the call feeless.
In order to use this, you can simply replace your existing signed
extension that manages transaction payment like so:
```diff
- pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
+ pallet_skip_feeless_payment::SkipCheckIfFeeless<
+ Runtime,
+ pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
+ >,
```
### Todo
- [x] Tests
- [x] Docs
- [x] Prdoc
---------
Co-authored-by: Nikhil Gupta <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Remove the `GRANDPA_AUTHORITIES_KEY` key and its usage. Apparently this
was used in the early days to communicate the grandpa authorities to the
node. However, we have now a runtime api that does this for us. So, this
pull request is moving from the custom managed storage item to a FRAME
managed storage item.
This pr also includes a migration for doing the switch on a running
chain.
---------
Co-authored-by: Davide Galassi <davxy@datawok.net>
These changes are required so that the bridgehub system runtimes can
more easily be configured with multiple message processors
Example usage:
```rust
use frame_support::traits::QueuePausedQuery;
impl pallet_message_queue::Config for Runtime {
type QueuePausedQuery = (A, B, C)
}
Changes:
- Adds a new call `remove_key` to the sudo pallet to permanently remove
the sudo key.
- Remove some clones and general maintenance
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Original PR https://github.com/paritytech/substrate/pull/14641
---
Closes https://github.com/paritytech/polkadot-sdk/issues/109
### Problem
Quoting from the above issue:
> When adding a pallet to chain after genesis we currently don't set the
StorageVersion. So, when calling on_chain_storage_version it returns 0
while the pallet is maybe already at storage version 9 when it was added
to the chain. This could lead to issues when running migrations.
### Solution
- Create a new trait `BeforeAllRuntimeMigrations` with a single method
`fn before_all_runtime_migrations() -> Weight` trait with a noop default
implementation
- Modify `Executive` to call
`BeforeAllRuntimeMigrations::before_all_runtime_migrations` for all
pallets before running any other hooks
- Implement `BeforeAllRuntimeMigrations` in the pallet proc macro to
initialize the on-chain version to the current pallet version if the
pallet has no storage set (indicating it has been recently added to the
runtime and needs to have its version initialised).
### Other changes in this PR
- Abstracted repeated boilerplate to access the `pallet_name` in the
pallet expand proc macro.
### FAQ
#### Why create a new hook instead of adding this logic to the pallet
`pre_upgrade`?
`Executive` currently runs `COnRuntimeUpgrade` (custom migrations)
before `AllPalletsWithSystem` migrations. We need versions to be
initialized before the `COnRuntimeUpgrade` migrations are run, because
`COnRuntimeUpgrade` migrations may use the on-chain version for critical
logic. e.g. `VersionedRuntimeUpgrade` uses it to decide whether or not
to execute.
We cannot reorder `COnRuntimeUpgrade` and `AllPalletsWithSystem` so
`AllPalletsWithSystem` runs first, because `AllPalletsWithSystem` have
some logic in their `post_upgrade` hooks to verify that the on-chain
version and current pallet version match. A common use case of
`COnRuntimeUpgrade` migrations is to perform a migration which will
result in the versions matching, so if they were reordered these
`post_upgrade` checks would fail.
#### Why init the on-chain version for pallets without a current storage
version?
We must init the on-chain version for pallets even if they don't have a
defined storage version so if there is a future version bump, the
on-chain version is not automatically set to that new version without a
proper migration.
e.g. bad scenario:
1. A pallet with no 'current version' is added to the runtime
2. Later, the pallet is upgraded with the 'current version' getting set
to 1 and a migration is added to Executive Migrations to migrate the
storage from 0 to 1
a. Runtime upgrade occurs
b. `before_all` hook initializes the on-chain version to 1
c. `on_runtime_upgrade` of the migration executes, and sees the on-chain
version is already 1 therefore think storage is already migrated and
does not execute the storage migration
Now, on-chain version is 1 but storage is still at version 0.
By always initializing the on-chain version when the pallet is added to
the runtime we avoid that scenario.
---------
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
### This PR is a port of this [PR for
substrate](https://github.com/paritytech/substrate/pull/13013) by
@kianenigma
Add infrastructure needed to have a Pallet::decode_entire_state(), which
makes sure all "typed" storage items defined in the pallet are
decode-able.
This is not enforced in any way at the moment. Teams who wish to
integrate/use this in the try-runtime feature flag should add
frame_support::storage::migration::EnsureStateDecodes as the LAST ITEM
of the runtime's custom migrations, and pass it to frame-executive. This
will make it usable in try-runtime on-runtime-upgrade.
This now catches cases like
https://github.com/paritytech/polkadot-sdk/pull/1969:
```pre
ERROR runtime::executive] failed to decode the value at key: Failed to decode value at key: 0x94eadf0156a8ad5156507773d0471e4ab8ebad86f546c7e0b135a4212aace339. Storage info StorageInfo { pallet_name: Ok("ParaScheduler"), storage_name: Ok("AvailabilityCores"), prefix: Err(Utf8Error { valid_up_to: 0, error_len: Some(1) }), max_values: Some(1), max_size: None }. Raw value: Some("0x0c010101010101")
```
... or:
Closes#241
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Andrew Jones