Commit Graph

8373 Commits

Author SHA1 Message Date
Gonçalo Pestana 8ee4042c3b Refactor staking ledger (#1484)
This PR refactors the staking ledger logic to encapsulate all reads and
mutations of `Ledger`, `Bonded`, `Payee` and stake locks within the
`StakingLedger` struct implementation.

With these changes, all the reads and mutations to the `Ledger`, `Payee`
and `Bonded` storage map should be done through the methods exposed by
StakingLedger to ensure the data and lock consistency of the operations.
The new introduced methods that mutate and read Ledger are:

- `ledger.update()`: inserts/updates a staking ledger in storage;
updates staking locks accordingly (and ledger.bond(), which is synthatic
sugar for ledger.update())
- `ledger.kill()`: removes all Bonded and StakingLedger related data for
a given ledger; updates staking locks accordingly;
`StakingLedger::get(account)`: queries both the `Bonded` and `Ledger`
storages and returns a `Option<StakingLedger>`. The pallet impl exposes
fn ledger(account) as synthatic sugar for `StakingLedger::get(account)`.

Retrieving a ledger with `StakingLedger::get()` can be done by providing
either a stash or controller account. The input must be wrapped in a
`StakingAccount` variant (Stash or Controller) which is treated
accordingly. This simplifies the caller API but will eventually be
deprecated once we completely get rid of the controller account in
staking. However, this refactor will help with the work necessary when
completely removing the controller.

Other goals:

- No logical changes have been introduced in this PR;
- No breaking changes or updates in wallets required;
- No new storage items or need to perform storage migrations;
- Centralise the changes to bonds and ledger updates to simplify the
OnStakingUpdate updates to the target list (related to
https://github.com/paritytech/polkadot-sdk/issues/443)

Note: it would be great to prevent or at least raise a warning if
`Ledger<T>`, `Payee<T>` and `Bonded<T>` storage types are accessed
outside the `StakingLedger` implementation. This PR should not get
blocked by that feature, but there's a tracking issue here
https://github.com/paritytech/polkadot-sdk/issues/149

Related and step towards
https://github.com/paritytech/polkadot-sdk/issues/443
2023-10-15 22:50:07 +02:00
drskalman 1b34571c0c Paired-key Crypto Scheme (#1705)
BEEFY needs two cryptographic keys at the same time. Validators should
sign BEEFY payload using both ECDSA and BLS key. The network will gossip
a payload which contains a valid ECDSA key. The prover nodes aggregate
the BLS keys if aggregation fails to verifies the validator which
provided a valid ECDSA signature but an invalid BLS signature is subject
to slashing.

As such BEEFY session should be initiated with both key. Currently there
is no straight forward way of doing so, beside having a session with
RuntimeApp corresponding to a crypto scheme contains both keys.

This pull request implement a generic paired_crypto scheme as well as
implementing it for (ECDSA, BLS) pair.

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2023-10-15 11:42:40 +02:00
S E R A Y A c9b51cd49c add link to rfc-0001 in broker README (#1862)
# Description
- What does this PR do?
  - link added
- Why are these changes needed?
  - improve docs
- How were these changes implemented and what do they affect?
  - only concerns docs
2023-10-15 10:00:58 +02:00
Julian Eager 9f7656df15 Discard Executor (#1855)
closes #622 

Pros:
* simpler interface, just functions:
`create_runtime_from_artifact_bytes()` and `execute_artifact()`

Cons:
* extra overhead of constructing executor semantics each time

I could make it a combination of
* `create_runtime_config(params)` (such that we could clone the
constructed semantics)
* `create_runtime(blob, config)`
* `execute_artifact(blob, config, params)`

Not sure if it's worth it though.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-10-14 23:06:00 +02:00
Juan 7c87d61f5a Macros to use path instead of ident (#1474) 2023-10-14 08:26:19 +02:00
0xmovses 24840290af Refactor alliance benchmarks to v2 (#1868)
- This PR refactors `alliance/src/benchmarkings.rs` to use benchmarking
v2. These changes are needed to improve the readability and
maintainability of the benchmarking code.

- No known issue to backlink.

## Local Testing 
1. `cargo build --features runtime-benchmarks` 
2. `cargo run --locked --release -p node-cli --bin substrate-node
--features runtime-benchmarks -- benchmark pallet --execution wasm
--wasm-execution compiled --chain dev --pallet "*" --extrinsic "*"
--steps 2 --repeat 1`
2023-10-13 21:54:45 +01:00
Adrian Catangiu 82bfe28424 frame: use derive-impl for beefy and mmr pallets (#1867)
Part of #171
2023-10-13 11:37:48 +03:00
Adrian Catangiu 832060000b sc-consensus-beefy: improve gossip logic (#1852)
- Remove cached messages used for deduplication in `GossipValidator`
since they're already deduplicated in upper layer `NetworkGossip`.
- Add cache for "justified rounds" to quickly discard any further (even
if potentially different) justifications at the gossip level, once a
valid one (for a respective round) is submitted to the worker.
- Add short-circuit in worker `finalize()` method to not attempt to
finalize same block multiple times (for example when we get
justifications for same block from multiple components like
block-import, gossip or on-demand).
- Change a test which had A LOT of latency in syncing blocks for some
weird reason and would only run after ~150seconds. It now runs
instantly.

Fixes https://github.com/paritytech/polkadot-sdk/issues/1728
2023-10-13 09:12:30 +02:00
gupnik 6b27dad359 Adds instance support for composite enums (#1857)
Fixes https://github.com/paritytech/polkadot-sdk/issues/1839

Currently, `composite_enum`s do not support pallet instances. This PR
allows the following:
```rust
	#[pallet::composite_enum]
	pub enum HoldReason<I: 'static = ()> {
		SomeHoldReason
	}
```

### Todo

- [x]  UI Test
2023-10-13 07:48:51 +02:00
Kevin Krone f0e6d2ad52 Adding try_state hook for Treasury pallet (#1820)
This PR adds a `try_state` hook for the `Treasury` pallet.
Part of #239.
2023-10-12 14:53:01 +02:00
Mira Ressel 447e753323 ci: bump ci image to rust 1.73.0 (#1830)
Co-authored-by: command-bot <>
2023-10-11 19:49:59 +02:00
0xmovses 1d9ec57276 Refactor Identity to benchmark v2 (#1838)
This PR refactors `identity/benchmarkings.rs` to use benchmarking v2.
These changes are needed to improve the readability and maintainability
of the benchmarking code. Changes were implemented using
[this](https://github.com/paritytech/polkadot-sdk/commit/9ec80090f5065ce0e3234886c43bc623e8e60d77)
commit as a guide. The logic of the benchmarks remains the same.

No known issue to backlink.

## Local Testing
To test the new benchmarks:
1. `cargo build --features runtime-benchmarks`
2. `./target/debug/polkadot benchmark pallet --steps=5 --repeat=2
--pallet=pallet_identity --extrinsic='*'`

---------

Co-authored-by: Richard Melkonian <movses@richards-mbp.home>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-11 17:21:01 +02:00
gupnik 294e99831d Fixes path issue in derive-impl (#1823)
Needs https://github.com/sam0x17/macro_magic/pull/13

The associated PR allows the export of tokens from macro_magic at the
specified path. This fixes the path issue in derive-impl. Now, we can
import the default config using the standard rust syntax:

```rust
use frame_system::config_preludes::TestDefaultConfig;

[derive_impl(TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::DefaultConfig for Test {
   //....
}
```
2023-10-11 07:26:13 +02:00
Sam Johnson 5adcb3e106 upgrade to macro_magic 0.4.3 (#1832)
# Description

Upgrades `macro_magic` to 0.4.3, which introduces the ability to have
`export_tokens` use the same name as the underlying item for its
auto-generated macro name. Ultimately this will allow for better dev ux
in our derive_impl feature.
2023-10-10 19:48:50 +02:00
Liam Aharon 55f354429c remote-ext: fix state download stall on slow connections and reduce memory usage (#1295)
Original PR https://github.com/paritytech/substrate/pull/14746

---

## Fixing stall

### Introduction
I experienced an apparent stall downloading state from
`https://rococo-try-runtime-node.parity-chains.parity.io:443` which was
having networking difficulties only responding to my JSONRPC requests
with 50-200KB/s of bandwidth.

This PR fixes the issue causing the stall, and generally improves
performance remote-ext when it downloads state by greatly reducing the
chances of a timeout occuring.

### Description
Introduces a new `REQUEST_DURATION_TARGET` constant and modifies
`get_storage_data_dynamic_batch_size` to

- Increase or decrease the batch size of the next request depending on
whether the elapsed time of the last request was gt or lt the target
- Reset the batch size to 1 if the request times out

This fixes an issue on slow connections that can otherwise cause
multiple timeouts and a stalled download when:

1. The batch size increases rapidly as remote-ext downloads keys with
small associated storage values
2. remote-ext tries to process a large series of subsequent keys all
with extremely large associated storage values (Rococo has a series of
keys 1-5MB large)
3. The huge storage values download for 5 minutes until the request
times out
4. The partially downloaded keys are thrown out and remote-ext tries
again with a smaller batch size, but the batch size is still far too
large and takes 5 minutes to be reduced again
5. The download will be essentially stalled for many hours while the
above step cycles


After this PR, the request size will

- Not grow as large to begin with, as it is regulated downwards as the
request duration exceeds the target
- Drop immediately to 1 if the request times out. A timeout indicates
the keys next in line to download have extremely large storage values
compared to previously downloaded keys, and we need to reset the batch
size to figure out what our new ideal batch size is. By not resetting
down to 1, we risk the next request timing out again.

## Reducing memory

As suggested by @bkchr, I adjusted `get_storage_data_dynamic_batch_size`
from being recursive to a loop which allows removing a bunch of clones
that were chewing through a lot of memory. I noticed actually it was
using up to 50GB swap previously when downloading Polkadot keys on a
slow connection, because it needed to recurse and clone a lot.

After this change it uses only ~1.5GB memory.
2023-10-11 01:41:58 +11:00
Oliver Tale-Yazdi 64877492c5 [FRAME] Warn on unchecked weight witness (#1818)
Adds a warning to FRAME pallets when a function argument that starts
with `_` is used in the weight formula.
This is in most cases an error since the weight witness needs to be
checked.

Example:

```rust
#[pallet::call_index(0)]
#[pallet::weight(T::SystemWeightInfo::remark(_remark.len() as u32))]
pub fn remark(_origin: OriginFor<T>, _remark: Vec<u8>) -> DispatchResultWithPostInfo {
	Ok(().into())
}
```

Produces this warning:

```pre
warning: use of deprecated constant `pallet::warnings::UncheckedWeightWitness_0::_w`: 
                 It is deprecated to not check weight witness data.
                 Please instead ensure that all witness data for weight calculation is checked before usage.
         
                 For more info see:
                     <https://github.com/paritytech/polkadot-sdk/pull/1818>
   --> substrate/frame/system/src/lib.rs:424:40
    |
424 |         pub fn remark(_origin: OriginFor<T>, _remark: Vec<u8>) -> DispatchResultWithPostInfo {
    |                                              ^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default
```

Can be suppressed like this, since in this case it is legit:

```rust
#[pallet::call_index(0)]
#[pallet::weight(T::SystemWeightInfo::remark(remark.len() as u32))]
pub fn remark(_origin: OriginFor<T>, remark: Vec<u8>) -> DispatchResultWithPostInfo {
	let _ = remark; // We dont need to check the weight witness.
	Ok(().into())
}
```

Changes:
- Add warning on uncheded weight witness
- Respect `subkeys` limit in `System::kill_prefix`
- Fix HRMP pallet and other warnings
- Update`proc_macro_warning` dependency
- Delete random folder `substrate/src/src` 🙈 
- Adding Prdoc

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2023-10-10 16:02:35 +02:00
Rahul Subramaniyam 2b4b33d01f Check for parent of first ready block being on chain (#1812)
When retrieving the ready blocks, verify that the parent of the first
ready block is on chain. If the parent is not on chain, we are
downloading from a fork. In this case, keep downloading until we have a
parent on chain (common ancestor).

Resolves https://github.com/paritytech/polkadot-sdk/issues/493.

---------

Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
2023-10-10 12:46:23 +03:00
David Emett 93d9c8c24e Make CheckNonce refuse transactions signed by accounts with no providers (#1578)
See #1453.

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-10-10 09:14:56 +02:00
David Emett a808a3a091 Mixnet integration (#1346)
See #1345, <https://github.com/paritytech/substrate/pull/14207>.

This adds all the necessary mixnet components, and puts them together in
the "kitchen-sink" node/runtime. The components added are:

- A pallet (`frame/mixnet`). This is responsible for determining the
current mixnet session and phase, and the mixnodes to use in each
session. It provides a function that validators can call to register a
mixnode for the next session. The logic of this pallet is very similar
to that of the `im-online` pallet.
- A service (`client/mixnet`). This implements the core mixnet logic,
building on the `mixnet` crate. The service communicates with other
nodes using notifications sent over the "mixnet" protocol.
- An RPC interface. This currently only supports sending transactions
over the mixnet.

---------

Co-authored-by: David Emett <dave@sp4m.net>
Co-authored-by: Javier Viola <javier@parity.io>
2023-10-09 14:56:30 +01:00
Muharem cb944dc548 Treasury spends various asset kinds (#1333)
### Summary 

This PR introduces new dispatchables to the treasury pallet, allowing
spends of various asset types. The enhanced features of the treasury
pallet, in conjunction with the asset-rate pallet, are set up and
enabled for Westend and Rococo.

### Westend and Rococo runtimes.

Polkadot/Kusams/Rococo Treasury can accept proposals for `spends` of
various asset kinds by specifying the asset's location and ID.

#### Treasury Instance New Dispatchables:
- `spend(AssetKind, AssetBalance, Beneficiary, Option<ValidFrom>)` -
propose and approve a spend;
- `payout(SpendIndex)` - payout an approved spend or retry a failed
payout
- `check_payment(SpendIndex)` - check the status of a payout;
- `void_spend(SpendIndex)` - void previously approved spend;
> existing spend dispatchable renamed to spend_local

in this context, the `AssetKind` parameter contains the asset's location
and it's corresponding `asset_id`, for example:
`USDT` on `AssetHub`,
``` rust
location = MultiLocation(0, X1(Parachain(1000)))
asset_id = MultiLocation(0, X2(PalletInstance(50), GeneralIndex(1984)))
```

the `Beneficiary` parameter is a `MultiLocation` in the context of the
asset's location, for example
``` rust
// the Fellowship salary pallet's location / account
FellowshipSalaryPallet = MultiLocation(1, X2(Parachain(1001), PalletInstance(64)))
// or custom `AccountId`
Alice = MultiLocation(0, AccountId32(network: None, id: [1,...]))
```

the `AssetBalance` represents the amount of the `AssetKind` to be
transferred to the `Beneficiary`. For permission checks, the asset
amount is converted to the native amount and compared against the
maximum spendable amount determined by the commanding spend origin.

the `spend` dispatchable allows for batching spends with different
`ValidFrom` arguments, enabling milestone-based spending. If the
expectations tied to an approved spend are not met, it is possible to
void the spend later using the `void_spend` dispatchable.

Asset Rate Pallet provides the conversion rate from the `AssetKind` to
the native balance.

#### Asset Rate Instance Dispatchables:
- `create(AssetKind, Rate)` - initialize a conversion rate to the native
balance for the given asset
- `update(AssetKind, Rate)` - update the conversion rate to the native
balance for the given asset
- `remove(AssetKind)` - remove an existing conversion rate to the native
balance for the given asset

the pallet's dispatchables can be executed by the Root or Treasurer
origins.

### Treasury Pallet

Treasury Pallet can accept proposals for `spends` of various asset kinds
and pay them out through the implementation of the `Pay` trait.

New Dispatchables:
- `spend(Config::AssetKind, AssetBalance, Config::Beneficiary,
Option<ValidFrom>)` - propose and approve a spend;
- `payout(SpendIndex)` - payout an approved spend or retry a failed
payout;
- `check_payment(SpendIndex)` - check the status of a payout;
- `void_spend(SpendIndex)` - void previously approved spend;
> existing spend dispatchable renamed to spend_local

The parameters' types of the `spend` dispatchable exposed via the
pallet's `Config` and allows to propose and accept a spend of a certain
amount.

An approved spend can be claimed via the `payout` within the
`Config::SpendPeriod`. Clients provide an implementation of the `Pay`
trait which can pay an asset of the `AssetKind` to the `Beneficiary` in
`AssetBalance` units.

The implementation of the Pay trait might not have an immediate final
payment status, for example if implemented over `XCM` and the actual
transfer happens on a remote chain.

The `check_status` dispatchable can be executed to update the spend's
payment state and retry the `payout` if the payment has failed.

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: command-bot <>
2023-10-07 19:32:35 +02:00
Dmitry Borodin 35ed272dad migrate babe and authorship to use derive-impl (#1790)
Moving a babe and authorship pallets to the latest and greatest
derive_impl.

Part of https://github.com/paritytech/polkadot-sdk/issues/171

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-10-07 11:04:00 +02:00
dependabot[bot] ddf5e5c04c Bump the known_good_semver group with 1 update (#1802)
Bumps the known_good_semver group with 1 update:
[syn](https://github.com/dtolnay/syn).

<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.38</h2>
<ul>
<li>Fix <em>&quot;method 'peek' has an incompatible type for
trait&quot;</em> error when defining <code>bool</code> as a custom
keyword (<a
href="https://redirect.github.com/dtolnay/syn/issues/1518">#1518</a>,
thanks <a
href="https://github.com/Vanille-N"><code>@​Vanille-N</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/syn/commit/43632bfb6c78ee1f952645a268ab1ac4af162977"><code>43632bf</code></a>
Release 2.0.38</li>
<li><a
href="https://github.com/dtolnay/syn/commit/abd2c214b44da64a5e420d72919308300eebc23d"><code>abd2c21</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/syn/issues/1518">#1518</a>
from Vanille-N/master</li>
<li><a
href="https://github.com/dtolnay/syn/commit/6701e6077e15013ef34b15e3ffdae2657e499d83"><code>6701e60</code></a>
Absolute path to <code>bool</code> in
<code>custom_punctuation.rs</code></li>
<li><a
href="https://github.com/dtolnay/syn/commit/7313d242398111423f046386aa0a75548f63d236"><code>7313d24</code></a>
Resolve single_match_else pedantic clippy lint in code generator</li>
<li><a
href="https://github.com/dtolnay/syn/commit/67ab64f3c09e17b23493c7cda498e7edb8830f21"><code>67ab64f</code></a>
Include unexpected token in the test failure message</li>
<li><a
href="https://github.com/dtolnay/syn/commit/137ae33486de3f2652487f8f64436ad1429df496"><code>137ae33</code></a>
Check no remaining token after the first literal</li>
<li><a
href="https://github.com/dtolnay/syn/commit/258e9e8a11d188c1ee1ffb2b069819239999f9ac"><code>258e9e8</code></a>
Ignore single_match_else pedantic clippy lint in test</li>
<li><a
href="https://github.com/dtolnay/syn/commit/92fd50ee8cb52968d9c66fbe6d67638c1f838e26"><code>92fd50e</code></a>
Test docs.rs documentation build in CI</li>
<li>See full diff in <a
href="https://github.com/dtolnay/syn/compare/2.0.37...2.0.38">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=syn&package-manager=cargo&previous-version=2.0.37&new-version=2.0.38)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-06 18:07:01 +02:00
Oliver Tale-Yazdi 1835c091c4 Revive Substrate Crate (#1477)
Closes https://github.com/paritytech/polkadot-sdk/issues/1450

Bringing back the Substrate crate that was forgotten in the monorepo
import 😅.
It is a doc-only crate. Version number is set to `1.0.0` and publishing
is enabled (so that we can link to docs.rs).

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-10-06 10:08:09 +02:00
dependabot[bot] 0c592329e9 Bump the known_good_semver group with 1 update (#1752)
Bumps the known_good_semver group with 1 update:
[clap](https://github.com/clap-rs/clap).

<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.4.6</h2>
<h2>[4.4.6] - 2023-09-28</h2>
<h3>Internal</h3>
<ul>
<li>Upgrade <code>anstream</code></li>
</ul>
<h2>v4.4.5</h2>
<h2>[4.4.5] - 2023-09-25</h2>
<h3>Fixes</h3>
<ul>
<li><em>(parser)</em> When inferring subcommand <code>name</code> or
<code>long_flag</code>, allow ambiguous-looking matches that
unambiguously map back to the same command</li>
<li><em>(parser)</em> When inferring subcommand <code>long_flag</code>,
don't panic</li>
<li><em>(assert)</em> Clarify what action is causing a positional that
doesn't set values which is especially useful for derive users</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.4.6] - 2023-09-28</h2>
<h3>Internal</h3>
<ul>
<li>Upgrade <code>anstream</code></li>
</ul>
<h2>[4.4.5] - 2023-09-25</h2>
<h3>Fixes</h3>
<ul>
<li><em>(parser)</em> When inferring subcommand <code>name</code> or
<code>long_flag</code>, allow ambiguous-looking matches that
unambiguously map back to the same command</li>
<li><em>(parser)</em> When inferring subcommand <code>long_flag</code>,
don't panic</li>
<li><em>(assert)</em> Clarify what action is causing a positional that
doesn't set values which is especially useful for derive users</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/clap-rs/clap/commit/39f5e807af1c08acedbf7343ce9ec379a4308636"><code>39f5e80</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a5cb6bb988bbacb02e8cf98b6156c860d0801e08"><code>a5cb6bb</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/418c0017a654e9859adfa9b051815f20e4583e31"><code>418c001</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5146">#5146</a>
from epage/update</li>
<li><a
href="https://github.com/clap-rs/clap/commit/485b957c4b90aa010276f813dbd429e1071f8fd9"><code>485b957</code></a>
chore: Upgrade anstream</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a1af8d9ad8f81eb2d71203b50c370a78ce3ec9f3"><code>a1af8d9</code></a>
chore: Update from '_rust/main'</li>
<li><a
href="https://github.com/clap-rs/clap/commit/ac51f0925003597dec21529538597dbd7872d1ac"><code>ac51f09</code></a>
chore(ci): Normalize json5 syntax</li>
<li><a
href="https://github.com/clap-rs/clap/commit/86c29dea384c7392a2b682fa0150f52c0f4c7f00"><code>86c29de</code></a>
chore(ci): Updaet Renovate schema</li>
<li><a
href="https://github.com/clap-rs/clap/commit/204552890d316ec9ae0b21f85298ba1d5d0786f8"><code>2045528</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/55d223001682bc668f5e4db91afd5e76c2a36597"><code>55d2230</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/492ee03b325ff98c7702295e024576b52b71358d"><code>492ee03</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5140">#5140</a>
from epage/dyn</li>
<li>Additional commits viewable in <a
href="https://github.com/clap-rs/clap/compare/v4.4.4...v4.4.6">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=clap&package-manager=cargo&previous-version=4.4.4&new-version=4.4.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-06 00:09:10 +02:00
Sebastian Kunert d21113c13f Delete full db directory with purge-chain subcommand (#1786)
Closes #1767 

Until now the `purge-chain` command would only remove the `full`
subfolder of the db folder. However there is also the `parachains` db
that currently remains and can cause problems on node restart.

Example wiht old code:
```
polkadot purge-chain --database paritydb --base-path /tmp/some-folder
Are you sure to remove "/tmp/some-folder/chains/polkadot/paritydb/full"? [y/N]: y
"/tmp/some-folder/chains/polkadot/paritydb/full" removed.
```
In this case `/tmp/some-folder/chains/polkadot/paritydb/parachains`
would remain and might cause problem on node restart because of version
conflicts as described in #1767. After this PR the whole
`/tmp/some-folder/chains/polkadot/paritydb` folder will be deleted.
2023-10-05 18:44:03 +02:00
Sergejs Kostjucenko 86955eef90 Remove deprecated CI config files (#1799)
This PR removes deprecated CI config files
2023-10-05 17:05:42 +02:00
yjh aad80cce31 feat: compute pallet/storage prefix hash at compile time (#1539)
Since the hash rules of this part of the `pallet_prefix/storage_prefix`
are always fixed, we can put the runtime calculation into compile time.

---
polkadot address: 15ouFh2SHpGbHtDPsJ6cXQfes9Cx1gEFnJJsJVqPGzBSTudr

---------

Co-authored-by: Juan <juangirini@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-03 17:16:46 +02:00
Liam Aharon db3fd68726 Init System Parachain storage versions and add migration check jobs to CI (#1344)
Makes SPs first class citizens along with the relay chains in the
context of our CI runtime upgrade checks.

## Code changes

- Sets missing current storage version in `uniques` pallet
- Adds multisig V1 migration to run where it was missing
- Removes executed migration whos pre/post hooks were failing from
collectives runtime
- Initializes storage versions for SP pallets added after genesis
- Originally I was going to wait for
https://github.com/paritytech/polkadot-sdk/pull/1297 to be merged so
this wouldn't need to be done manually, but it doesn't seem like it'll
be merged any time soon so I've decided to set them manually to unblock
this

## CI changes

- Removed dependency of `westend` runtime upgrades being complete prior
to other ones running. I assume it is supposed to cache the
`try-runtime` build for a performance benefit, but it seems it wasn't
working. Maybe someone from the CI team can look into this or explain
why it needs to be there?

- Adds check-runtime-migration jobs for Parity asset-hubs, bridge-hubs
and contract chains

- Updated VARIABLES to accomodate the `kusama-runtime` package being
renamed to `staging-kusama-runtime` in
https://github.com/paritytech/polkadot-sdk/pull/1241

- Added `EXTRA_ARGS` variable to `check-runtime-migration`, and set
`--no-weight-warnings` to the relay chain runtime upgrade checks (relay
chains don't have weight restrictions).
2023-10-03 02:11:56 +11:00
Bastian Köcher c54ea64af4 no-bound derives: Use absolute path for core (#1763)
Closes: https://github.com/paritytech/polkadot-sdk/issues/1718
2023-10-02 06:45:32 +11:00
Kian Paimani 2ed66a0960 migrate alliance, fast-unstake and bags list to use derive-impl (#1636)
Moving a few pallets to the latest and greatest `derive_impl` to give it
a try.

Part of #171

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2023-10-01 16:16:14 +02:00
Piet e8baac7848 Tvl pool staking (#1322)
What does this PR do?
- Introduced the TotalValueLocked storage for nomination-pools. 
- introduced a slashing api in mock.rs 
- additional test for tracking a slashing event towards a pool without
sub-pools
- migration for the nomination-pools (V6 to V7) with
`VersionedMigration`

Why are these changes needed?
this is the continuation of the work by @kianenigma in this
[PR](https://github.com/paritytech/substrate/pull/13319)

How were these changes implemented and what do they affect?
- It's an extra StorageValue that's modified whenever funds flow in or
out of staking for any of the `bonded_account` of `BondedPools`
- The `PoolSlashed`event is now emitted even when no `SubPools` are
found

Closes https://github.com/paritytech/polkadot-sdk/issues/155
KSM: HHEEgVzcqL3kCXgsxSfJMbsTy8dxoTctuXtpY94n4s8F4pS

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: command-bot <>
2023-10-01 03:36:48 +02:00
yjh 8fe947af60 improve service error (#1734)
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-30 07:52:47 +03:00
Muharem 7d4f82968c frame-support: RuntimeDebug\Eq\PartialEq impls for Imbalance (#1717)
Derive `RuntimeDebug\Eq\PartialEq` but do not bound any generics.

This achieved by using their equivalent no bound versions:
`EqNoBound\PartialEqNoBound\RuntimeDebugNoBound`.

Deriving with `Debug`, `Eq`, and `PartialEq` for the `Debt` and `Credit`
type aliases of `Imbalance` is not feasible due to the `OnDrop` and
`OppositeOnDrop` generic types lacking implementations of the same
traits.

This absence posed challenges in testing and any scenarios that demanded
the traits implementations for the type.
2023-09-29 23:47:17 +02:00
Sebastian Kunert 9485b0b492 Point documentation links to monorepo (#1741) 2023-09-29 22:33:26 +02:00
Ankan f820dc0a1f [NPoS] Fix for Reward Deficit in the pool (#1255)
closes https://github.com/paritytech/polkadot-sdk/issues/158.
partially addresses
https://github.com/paritytech/polkadot-sdk/issues/226.

Instead of fragile calculation of current balance by looking at `free
balance - ED`, Nomination Pool now freezes ED in the pool reward account
to restrict an account from going below minimum balance. This also has a
nice side effect that if ED changes, we know how much is the imbalance
in ED frozen in the pool and the current required ED. A pool operator
can diligently top up the pool with the deficit in ED or vice versa,
withdraw the excess they transferred to the pool.

## Notable changes
- New call `adjust_pool_deposit`: Allows to top up the deficit or
withdraw the excess deposited funds to the pool.
- Uses Fungible trait (instead of Currency trait). Since NP was not
doing any locking/reserving previously, no migration is needed for this.
- One time migration of freezing ED from each of the existing pools (not
very PoV friendly but fine for relay chain).
2023-09-29 17:48:40 +02:00
Dmitry Markin 0691c91e15 Move import queue from ChainSync to SyncingEngine (#1736)
This PR is part of [Sync
2.0](https://github.com/paritytech/polkadot-sdk/issues/534) refactoring
aimed at making `ChainSync` a pure state machine.

Resolves https://github.com/paritytech/polkadot-sdk/issues/501.
2023-09-29 17:58:16 +03:00
Piotr Mikołajczyk d8d90a82a7 Enable mocking contracts (#1331)
# Description
This PR introduces two changes:
- the previous `Tracing` trait has been modified to accept contract
address instead of code hash (seems to be way more convenient)
- a new trait `CallInterceptor` that allows intercepting contract calls;
in particular the default implementation for `()` will just proceed in a
standard way (after compilation optimizations, there will be no
footprint of that); however, implementing type might decide to mock
invocation and return `ExecResult` instead

Note: one might try merging `before_call` and `intercept_call`. However,
IMHO this would be bad, since it would mix two completely different
abstractions - tracing without any effects and actual intervention into
execution process.

This will unblock working on mocking contracts utility in drink and
similar tools (https://github.com/Cardinal-Cryptography/drink/issues/33)

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling
requirements](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md#process)
of this project (at minimum one label for `T` required)
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)
2023-09-29 15:39:13 +02:00
Sebastian Kunert 4902db2198 Use Extensions to register offchain worker custom extensions (#1719)
Closes #1671 

Adds a `type_id` function to the `Extension` trait, allowing to properly store an retrieve
boxed `Extensions`.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-29 09:37:51 +02:00
Tadeo Hepperle 379be3d7c3 [RPC-Spec-V2] chainHead: use integer for block index and adjust RuntimeVersion JSON format (#1666)
This PR adjusts the serialized format of the the returned RuntimeVersion
in the rpc-spec-v2 methods. This is done to match the format defined
here:
https://paritytech.github.io/json-rpc-interface-spec/api/chainHead_unstable_follow.html#about-the-runtime

- ##### `apis` field as object
`apis` field of `RuntimeVersion` is now returned as an object, e.g. 
```
"apis": {
      "0xdf6acb689907609b": 3,
      "0x37e397fc7c91f5e4": 1,
}
```
instead of 
```
"apis": [
      ["0xdf6acb689907609b", 3],
      ["0x37e397fc7c91f5e4", 1],
]
```
- ##### removed `stateVersion` and `authoringVersion`
`stateVersion` and `authoringVersion` are no longer returned in the
`RuntimeVersion` JSON Object.

- ##### block index in chain head events as integer

### Related Issues

Closes: #1507
Closes: #1146

### Testing Done
Adjusted existing tests to make sure data is returned in the correct
format.
2023-09-29 09:08:19 +02:00
btwiuse 7b9861a2f0 Fix subkey inspect output text padding (#1744)
This pull request is to fix the output text misalignment of `subkey
inspect` command:

(the `Network ID` line has an extra space at the end, and the `Secret
seed` line lacks a leading space)

```
[btwiuse@railway ~]$ subkey inspect '' | cat -A
Secret Key URI `` is account:$
  Network ID:        substrate $
 Secret seed:       0xfac7959dbfe72f052e5a0c3c8d6530f202b02fd8f9f5ca3580ec8deb7797479e$
  Public key (hex):  0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a$
  Account ID:        0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a$
  Public key (SS58): 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV$
  SS58 Address:      5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV$
```

The output should be in the same YAML-like format as `subkey generate`:

```
[btwiuse@railway ~]$ subkey generate  | cat -A
Secret phrase:       awkward eagle survey resemble novel resist modify memory pistol shed flower run$
  Network ID:        substrate$
  Secret seed:       0x20502f79366325b7dc7620664e8844ae69d441baf6e5b571a57d3b3ff28e9586$
  Public key (hex):  0x468874b9e5b6b77333fa702b9201b924d6834bf956e33e2bbe37d131134ca830$
  Account ID:        0x468874b9e5b6b77333fa702b9201b924d6834bf956e33e2bbe37d131134ca830$
  Public key (SS58): 5DfBkAMg5xQmsePFr3BWLZm99smiNyy9axWSgembvFgDKh9v$
  SS58 Address:      5DfBkAMg5xQmsePFr3BWLZm99smiNyy9axWSgembvFgDKh9v$
```

This change will fix `subkey` as well as all binaries that embed it as
the `key` subcommand, for example: `substrate key`, `polkadot key`, etc.

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-09-29 05:36:40 +10:00
Alexandru Vasile 945ebbbcf6 archive: Implement height, hashByHeight and call (#1582)
This PR implements:
- `archive_unstable_finalized_height`: Get the height of the most recent
finalized block
- `archive_unstable_hash_by_height`: Get the hashes (possible empty) of
blocks from the given height
- `archive_unstable_call`: Call into the runtime of a block

Builds on top of: https://github.com/paritytech/polkadot-sdk/pull/1560

### Testing Done
- unit tests for the methods with custom block tree for different
heights / forks

Closes: https://github.com/paritytech/polkadot-sdk/issues/1510
Closes: https://github.com/paritytech/polkadot-sdk/issues/1513
Closes: https://github.com/paritytech/polkadot-sdk/issues/1511

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-09-28 20:20:56 +03:00
Alexandru Vasile b5a0708fb7 rpc/client: Propagate rpc_methods method to reported methods (#1713)
The PR exposes the `rpc_methods` name in the `rpc_methods` response.
This feature is useful for servers that only forward requests to methods
that are reported by the `rpc_methods`.

Jsonrpsee exposes the available methods registered so far. Registering
the `rpc_methods` requires a closure that already stores the result. As
such, the `rpc_methods` method name is manually added to the available
methods.

Closes: https://github.com/paritytech/polkadot-sdk/issues/1627

### Testing Done

```
$> curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "rpc_methods", "params":[]}' http://localhost:9944

{"jsonrpc":"2.0","result":{"methods":["account_nextIndex",... "rpc_methods",..."unsubscribe_newHead"]},"id":1}⏎
```

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-09-28 20:19:27 +03:00
Xavier Lau de71fecc4e Add MaxTipAmount for pallet-tips (#1709)
Last week we experienced a governance attack.
Surprisingly, there was no upper limit on the tip amount.

Due to the mechanism of pallet-fragment-election, the council members
will be refreshed immediately. Attacker is easy to control the council
and give a large tip amount.
2023-09-28 21:08:05 +10:00
Xiliang Chen b50d8e6f7f add some events for pallet-bounties (#1706)
Add missing events for pallet-bounties
2023-09-28 21:04:35 +10:00
Dmitry Markin 14e5d23348 Move requests-responses and polling from ChainSync to SyncingEngine (#1650)
Move request-response handling from `ChainSync` to `SyncingEngine` as
part of [Sync
2.0](https://github.com/paritytech/polkadot-sdk/issues/534) refactoring
aimed at making `ChainSync` a pure state machine.

Resolves https://github.com/paritytech/polkadot-sdk/issues/502.

---------

Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-27 19:44:37 +03:00
Dónal Murray 02284a3e82 Add custom error message for StorageNoopGuard (#1727)
Expand `StorageNoopGuard` to be able to add extra context through a
custom error message. When the guard is triggered it panics with an
error message which can be defaulted, set on construction, or set after
it has been constructed.

Turn `StorageNoopGuard` into struct with `storage_root` and
`error_message` and added `from_error_message` constructor and
`set_error_message` setter.

Also added `new()` aliased to `default()`.

Closes #375

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-09-27 19:39:45 +03:00
Joshy Orndorff ec9274eea6 remove unnecessary hash string (#1722)
This PR removes some unnecessary `r#`...`#` around a string and the
corresponding comment that it was done because rustfmt wasn't working
for "some reason". It seems to work fine now and clippy prefers it this
way.

---------

Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
2023-09-27 18:48:24 +03:00
Matteo Muraca 8b061a5c5d Associated type Hasher for QueryPreimage, StorePreimage and Bounded (#1720)
I hope it's enough to fix #1701 
the only solution I found to make it happen is to put an associated type
to the `Bounded` enum as well.
@liamaharon @kianenigma @bkchr 

Polkadot address: 12poSUQPtcF1HUPQGY3zZu2P8emuW9YnsPduA4XG3oCEfJVp

---------

Signed-off-by: muraca <mmuraca247@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-09-27 22:46:14 +10:00
Michal Kucharczyk 5a2833cceb genesis-builder: implemented for all runtimes (#1492)
This PR implements [`GenesisBuilder`
API](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/substrate/primitives/genesis-builder/src/lib.rs#L38)
for all the runtimes in polkadot repo.

Step towards: paritytech/polkadot-sdk#25

---------

Co-authored-by: ordian <write@reusable.software>
2023-09-27 11:59:19 +02:00
Michal Kucharczyk ab3a3bc278 BlockId removal: tx-pool refactor (#1678)
It changes following APIs:
- trait `ChainApi`
-- `validate_transaction`

- trait `TransactionPool` 
--`submit_at`
--`submit_one`
--`submit_and_watch`

and some implementation details, in particular:
- impl `Pool` 
--`submit_at`
--`resubmit_at`
--`submit_one`
--`submit_and_watch`
--`prune_known`
--`prune`
--`prune_tags`
--`resolve_block_number`
--`verify`
--`verify_one`

- revalidation queue

All tests are also adjusted.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-27 11:58:39 +02:00