Commit Graph

8364 Commits

Author SHA1 Message Date
Kevin Krone f0e6d2ad52 Adding try_state hook for Treasury pallet (#1820)
This PR adds a `try_state` hook for the `Treasury` pallet.
Part of #239.
2023-10-12 14:53:01 +02:00
Mira Ressel 447e753323 ci: bump ci image to rust 1.73.0 (#1830)
Co-authored-by: command-bot <>
2023-10-11 19:49:59 +02:00
0xmovses 1d9ec57276 Refactor Identity to benchmark v2 (#1838)
This PR refactors `identity/benchmarkings.rs` to use benchmarking v2.
These changes are needed to improve the readability and maintainability
of the benchmarking code. Changes were implemented using
[this](https://github.com/paritytech/polkadot-sdk/commit/9ec80090f5065ce0e3234886c43bc623e8e60d77)
commit as a guide. The logic of the benchmarks remains the same.

No known issue to backlink.

## Local Testing
To test the new benchmarks:
1. `cargo build --features runtime-benchmarks`
2. `./target/debug/polkadot benchmark pallet --steps=5 --repeat=2
--pallet=pallet_identity --extrinsic='*'`

---------

Co-authored-by: Richard Melkonian <movses@richards-mbp.home>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-11 17:21:01 +02:00
gupnik 294e99831d Fixes path issue in derive-impl (#1823)
Needs https://github.com/sam0x17/macro_magic/pull/13

The associated PR allows the export of tokens from macro_magic at the
specified path. This fixes the path issue in derive-impl. Now, we can
import the default config using the standard rust syntax:

```rust
use frame_system::config_preludes::TestDefaultConfig;

[derive_impl(TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::DefaultConfig for Test {
   //....
}
```
2023-10-11 07:26:13 +02:00
Sam Johnson 5adcb3e106 upgrade to macro_magic 0.4.3 (#1832)
# Description

Upgrades `macro_magic` to 0.4.3, which introduces the ability to have
`export_tokens` use the same name as the underlying item for its
auto-generated macro name. Ultimately this will allow for better dev ux
in our derive_impl feature.
2023-10-10 19:48:50 +02:00
Liam Aharon 55f354429c remote-ext: fix state download stall on slow connections and reduce memory usage (#1295)
Original PR https://github.com/paritytech/substrate/pull/14746

---

## Fixing stall

### Introduction
I experienced an apparent stall downloading state from
`https://rococo-try-runtime-node.parity-chains.parity.io:443` which was
having networking difficulties only responding to my JSONRPC requests
with 50-200KB/s of bandwidth.

This PR fixes the issue causing the stall, and generally improves
performance remote-ext when it downloads state by greatly reducing the
chances of a timeout occuring.

### Description
Introduces a new `REQUEST_DURATION_TARGET` constant and modifies
`get_storage_data_dynamic_batch_size` to

- Increase or decrease the batch size of the next request depending on
whether the elapsed time of the last request was gt or lt the target
- Reset the batch size to 1 if the request times out

This fixes an issue on slow connections that can otherwise cause
multiple timeouts and a stalled download when:

1. The batch size increases rapidly as remote-ext downloads keys with
small associated storage values
2. remote-ext tries to process a large series of subsequent keys all
with extremely large associated storage values (Rococo has a series of
keys 1-5MB large)
3. The huge storage values download for 5 minutes until the request
times out
4. The partially downloaded keys are thrown out and remote-ext tries
again with a smaller batch size, but the batch size is still far too
large and takes 5 minutes to be reduced again
5. The download will be essentially stalled for many hours while the
above step cycles


After this PR, the request size will

- Not grow as large to begin with, as it is regulated downwards as the
request duration exceeds the target
- Drop immediately to 1 if the request times out. A timeout indicates
the keys next in line to download have extremely large storage values
compared to previously downloaded keys, and we need to reset the batch
size to figure out what our new ideal batch size is. By not resetting
down to 1, we risk the next request timing out again.

## Reducing memory

As suggested by @bkchr, I adjusted `get_storage_data_dynamic_batch_size`
from being recursive to a loop which allows removing a bunch of clones
that were chewing through a lot of memory. I noticed actually it was
using up to 50GB swap previously when downloading Polkadot keys on a
slow connection, because it needed to recurse and clone a lot.

After this change it uses only ~1.5GB memory.
2023-10-11 01:41:58 +11:00
Oliver Tale-Yazdi 64877492c5 [FRAME] Warn on unchecked weight witness (#1818)
Adds a warning to FRAME pallets when a function argument that starts
with `_` is used in the weight formula.
This is in most cases an error since the weight witness needs to be
checked.

Example:

```rust
#[pallet::call_index(0)]
#[pallet::weight(T::SystemWeightInfo::remark(_remark.len() as u32))]
pub fn remark(_origin: OriginFor<T>, _remark: Vec<u8>) -> DispatchResultWithPostInfo {
	Ok(().into())
}
```

Produces this warning:

```pre
warning: use of deprecated constant `pallet::warnings::UncheckedWeightWitness_0::_w`: 
                 It is deprecated to not check weight witness data.
                 Please instead ensure that all witness data for weight calculation is checked before usage.
         
                 For more info see:
                     <https://github.com/paritytech/polkadot-sdk/pull/1818>
   --> substrate/frame/system/src/lib.rs:424:40
    |
424 |         pub fn remark(_origin: OriginFor<T>, _remark: Vec<u8>) -> DispatchResultWithPostInfo {
    |                                              ^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default
```

Can be suppressed like this, since in this case it is legit:

```rust
#[pallet::call_index(0)]
#[pallet::weight(T::SystemWeightInfo::remark(remark.len() as u32))]
pub fn remark(_origin: OriginFor<T>, remark: Vec<u8>) -> DispatchResultWithPostInfo {
	let _ = remark; // We dont need to check the weight witness.
	Ok(().into())
}
```

Changes:
- Add warning on uncheded weight witness
- Respect `subkeys` limit in `System::kill_prefix`
- Fix HRMP pallet and other warnings
- Update`proc_macro_warning` dependency
- Delete random folder `substrate/src/src` 🙈 
- Adding Prdoc

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2023-10-10 16:02:35 +02:00
Rahul Subramaniyam 2b4b33d01f Check for parent of first ready block being on chain (#1812)
When retrieving the ready blocks, verify that the parent of the first
ready block is on chain. If the parent is not on chain, we are
downloading from a fork. In this case, keep downloading until we have a
parent on chain (common ancestor).

Resolves https://github.com/paritytech/polkadot-sdk/issues/493.

---------

Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
2023-10-10 12:46:23 +03:00
David Emett 93d9c8c24e Make CheckNonce refuse transactions signed by accounts with no providers (#1578)
See #1453.

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-10-10 09:14:56 +02:00
David Emett a808a3a091 Mixnet integration (#1346)
See #1345, <https://github.com/paritytech/substrate/pull/14207>.

This adds all the necessary mixnet components, and puts them together in
the "kitchen-sink" node/runtime. The components added are:

- A pallet (`frame/mixnet`). This is responsible for determining the
current mixnet session and phase, and the mixnodes to use in each
session. It provides a function that validators can call to register a
mixnode for the next session. The logic of this pallet is very similar
to that of the `im-online` pallet.
- A service (`client/mixnet`). This implements the core mixnet logic,
building on the `mixnet` crate. The service communicates with other
nodes using notifications sent over the "mixnet" protocol.
- An RPC interface. This currently only supports sending transactions
over the mixnet.

---------

Co-authored-by: David Emett <dave@sp4m.net>
Co-authored-by: Javier Viola <javier@parity.io>
2023-10-09 14:56:30 +01:00
Muharem cb944dc548 Treasury spends various asset kinds (#1333)
### Summary 

This PR introduces new dispatchables to the treasury pallet, allowing
spends of various asset types. The enhanced features of the treasury
pallet, in conjunction with the asset-rate pallet, are set up and
enabled for Westend and Rococo.

### Westend and Rococo runtimes.

Polkadot/Kusams/Rococo Treasury can accept proposals for `spends` of
various asset kinds by specifying the asset's location and ID.

#### Treasury Instance New Dispatchables:
- `spend(AssetKind, AssetBalance, Beneficiary, Option<ValidFrom>)` -
propose and approve a spend;
- `payout(SpendIndex)` - payout an approved spend or retry a failed
payout
- `check_payment(SpendIndex)` - check the status of a payout;
- `void_spend(SpendIndex)` - void previously approved spend;
> existing spend dispatchable renamed to spend_local

in this context, the `AssetKind` parameter contains the asset's location
and it's corresponding `asset_id`, for example:
`USDT` on `AssetHub`,
``` rust
location = MultiLocation(0, X1(Parachain(1000)))
asset_id = MultiLocation(0, X2(PalletInstance(50), GeneralIndex(1984)))
```

the `Beneficiary` parameter is a `MultiLocation` in the context of the
asset's location, for example
``` rust
// the Fellowship salary pallet's location / account
FellowshipSalaryPallet = MultiLocation(1, X2(Parachain(1001), PalletInstance(64)))
// or custom `AccountId`
Alice = MultiLocation(0, AccountId32(network: None, id: [1,...]))
```

the `AssetBalance` represents the amount of the `AssetKind` to be
transferred to the `Beneficiary`. For permission checks, the asset
amount is converted to the native amount and compared against the
maximum spendable amount determined by the commanding spend origin.

the `spend` dispatchable allows for batching spends with different
`ValidFrom` arguments, enabling milestone-based spending. If the
expectations tied to an approved spend are not met, it is possible to
void the spend later using the `void_spend` dispatchable.

Asset Rate Pallet provides the conversion rate from the `AssetKind` to
the native balance.

#### Asset Rate Instance Dispatchables:
- `create(AssetKind, Rate)` - initialize a conversion rate to the native
balance for the given asset
- `update(AssetKind, Rate)` - update the conversion rate to the native
balance for the given asset
- `remove(AssetKind)` - remove an existing conversion rate to the native
balance for the given asset

the pallet's dispatchables can be executed by the Root or Treasurer
origins.

### Treasury Pallet

Treasury Pallet can accept proposals for `spends` of various asset kinds
and pay them out through the implementation of the `Pay` trait.

New Dispatchables:
- `spend(Config::AssetKind, AssetBalance, Config::Beneficiary,
Option<ValidFrom>)` - propose and approve a spend;
- `payout(SpendIndex)` - payout an approved spend or retry a failed
payout;
- `check_payment(SpendIndex)` - check the status of a payout;
- `void_spend(SpendIndex)` - void previously approved spend;
> existing spend dispatchable renamed to spend_local

The parameters' types of the `spend` dispatchable exposed via the
pallet's `Config` and allows to propose and accept a spend of a certain
amount.

An approved spend can be claimed via the `payout` within the
`Config::SpendPeriod`. Clients provide an implementation of the `Pay`
trait which can pay an asset of the `AssetKind` to the `Beneficiary` in
`AssetBalance` units.

The implementation of the Pay trait might not have an immediate final
payment status, for example if implemented over `XCM` and the actual
transfer happens on a remote chain.

The `check_status` dispatchable can be executed to update the spend's
payment state and retry the `payout` if the payment has failed.

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: command-bot <>
2023-10-07 19:32:35 +02:00
Dmitry Borodin 35ed272dad migrate babe and authorship to use derive-impl (#1790)
Moving a babe and authorship pallets to the latest and greatest
derive_impl.

Part of https://github.com/paritytech/polkadot-sdk/issues/171

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-10-07 11:04:00 +02:00
dependabot[bot] ddf5e5c04c Bump the known_good_semver group with 1 update (#1802)
Bumps the known_good_semver group with 1 update:
[syn](https://github.com/dtolnay/syn).

<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.38</h2>
<ul>
<li>Fix <em>&quot;method 'peek' has an incompatible type for
trait&quot;</em> error when defining <code>bool</code> as a custom
keyword (<a
href="https://redirect.github.com/dtolnay/syn/issues/1518">#1518</a>,
thanks <a
href="https://github.com/Vanille-N"><code>@​Vanille-N</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/syn/commit/43632bfb6c78ee1f952645a268ab1ac4af162977"><code>43632bf</code></a>
Release 2.0.38</li>
<li><a
href="https://github.com/dtolnay/syn/commit/abd2c214b44da64a5e420d72919308300eebc23d"><code>abd2c21</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/syn/issues/1518">#1518</a>
from Vanille-N/master</li>
<li><a
href="https://github.com/dtolnay/syn/commit/6701e6077e15013ef34b15e3ffdae2657e499d83"><code>6701e60</code></a>
Absolute path to <code>bool</code> in
<code>custom_punctuation.rs</code></li>
<li><a
href="https://github.com/dtolnay/syn/commit/7313d242398111423f046386aa0a75548f63d236"><code>7313d24</code></a>
Resolve single_match_else pedantic clippy lint in code generator</li>
<li><a
href="https://github.com/dtolnay/syn/commit/67ab64f3c09e17b23493c7cda498e7edb8830f21"><code>67ab64f</code></a>
Include unexpected token in the test failure message</li>
<li><a
href="https://github.com/dtolnay/syn/commit/137ae33486de3f2652487f8f64436ad1429df496"><code>137ae33</code></a>
Check no remaining token after the first literal</li>
<li><a
href="https://github.com/dtolnay/syn/commit/258e9e8a11d188c1ee1ffb2b069819239999f9ac"><code>258e9e8</code></a>
Ignore single_match_else pedantic clippy lint in test</li>
<li><a
href="https://github.com/dtolnay/syn/commit/92fd50ee8cb52968d9c66fbe6d67638c1f838e26"><code>92fd50e</code></a>
Test docs.rs documentation build in CI</li>
<li>See full diff in <a
href="https://github.com/dtolnay/syn/compare/2.0.37...2.0.38">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=syn&package-manager=cargo&previous-version=2.0.37&new-version=2.0.38)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-06 18:07:01 +02:00
Oliver Tale-Yazdi 1835c091c4 Revive Substrate Crate (#1477)
Closes https://github.com/paritytech/polkadot-sdk/issues/1450

Bringing back the Substrate crate that was forgotten in the monorepo
import 😅.
It is a doc-only crate. Version number is set to `1.0.0` and publishing
is enabled (so that we can link to docs.rs).

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-10-06 10:08:09 +02:00
dependabot[bot] 0c592329e9 Bump the known_good_semver group with 1 update (#1752)
Bumps the known_good_semver group with 1 update:
[clap](https://github.com/clap-rs/clap).

<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.4.6</h2>
<h2>[4.4.6] - 2023-09-28</h2>
<h3>Internal</h3>
<ul>
<li>Upgrade <code>anstream</code></li>
</ul>
<h2>v4.4.5</h2>
<h2>[4.4.5] - 2023-09-25</h2>
<h3>Fixes</h3>
<ul>
<li><em>(parser)</em> When inferring subcommand <code>name</code> or
<code>long_flag</code>, allow ambiguous-looking matches that
unambiguously map back to the same command</li>
<li><em>(parser)</em> When inferring subcommand <code>long_flag</code>,
don't panic</li>
<li><em>(assert)</em> Clarify what action is causing a positional that
doesn't set values which is especially useful for derive users</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.4.6] - 2023-09-28</h2>
<h3>Internal</h3>
<ul>
<li>Upgrade <code>anstream</code></li>
</ul>
<h2>[4.4.5] - 2023-09-25</h2>
<h3>Fixes</h3>
<ul>
<li><em>(parser)</em> When inferring subcommand <code>name</code> or
<code>long_flag</code>, allow ambiguous-looking matches that
unambiguously map back to the same command</li>
<li><em>(parser)</em> When inferring subcommand <code>long_flag</code>,
don't panic</li>
<li><em>(assert)</em> Clarify what action is causing a positional that
doesn't set values which is especially useful for derive users</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/clap-rs/clap/commit/39f5e807af1c08acedbf7343ce9ec379a4308636"><code>39f5e80</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a5cb6bb988bbacb02e8cf98b6156c860d0801e08"><code>a5cb6bb</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/418c0017a654e9859adfa9b051815f20e4583e31"><code>418c001</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5146">#5146</a>
from epage/update</li>
<li><a
href="https://github.com/clap-rs/clap/commit/485b957c4b90aa010276f813dbd429e1071f8fd9"><code>485b957</code></a>
chore: Upgrade anstream</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a1af8d9ad8f81eb2d71203b50c370a78ce3ec9f3"><code>a1af8d9</code></a>
chore: Update from '_rust/main'</li>
<li><a
href="https://github.com/clap-rs/clap/commit/ac51f0925003597dec21529538597dbd7872d1ac"><code>ac51f09</code></a>
chore(ci): Normalize json5 syntax</li>
<li><a
href="https://github.com/clap-rs/clap/commit/86c29dea384c7392a2b682fa0150f52c0f4c7f00"><code>86c29de</code></a>
chore(ci): Updaet Renovate schema</li>
<li><a
href="https://github.com/clap-rs/clap/commit/204552890d316ec9ae0b21f85298ba1d5d0786f8"><code>2045528</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/55d223001682bc668f5e4db91afd5e76c2a36597"><code>55d2230</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/492ee03b325ff98c7702295e024576b52b71358d"><code>492ee03</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5140">#5140</a>
from epage/dyn</li>
<li>Additional commits viewable in <a
href="https://github.com/clap-rs/clap/compare/v4.4.4...v4.4.6">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=clap&package-manager=cargo&previous-version=4.4.4&new-version=4.4.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-06 00:09:10 +02:00
Sebastian Kunert d21113c13f Delete full db directory with purge-chain subcommand (#1786)
Closes #1767 

Until now the `purge-chain` command would only remove the `full`
subfolder of the db folder. However there is also the `parachains` db
that currently remains and can cause problems on node restart.

Example wiht old code:
```
polkadot purge-chain --database paritydb --base-path /tmp/some-folder
Are you sure to remove "/tmp/some-folder/chains/polkadot/paritydb/full"? [y/N]: y
"/tmp/some-folder/chains/polkadot/paritydb/full" removed.
```
In this case `/tmp/some-folder/chains/polkadot/paritydb/parachains`
would remain and might cause problem on node restart because of version
conflicts as described in #1767. After this PR the whole
`/tmp/some-folder/chains/polkadot/paritydb` folder will be deleted.
2023-10-05 18:44:03 +02:00
Sergejs Kostjucenko 86955eef90 Remove deprecated CI config files (#1799)
This PR removes deprecated CI config files
2023-10-05 17:05:42 +02:00
yjh aad80cce31 feat: compute pallet/storage prefix hash at compile time (#1539)
Since the hash rules of this part of the `pallet_prefix/storage_prefix`
are always fixed, we can put the runtime calculation into compile time.

---
polkadot address: 15ouFh2SHpGbHtDPsJ6cXQfes9Cx1gEFnJJsJVqPGzBSTudr

---------

Co-authored-by: Juan <juangirini@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-03 17:16:46 +02:00
Liam Aharon db3fd68726 Init System Parachain storage versions and add migration check jobs to CI (#1344)
Makes SPs first class citizens along with the relay chains in the
context of our CI runtime upgrade checks.

## Code changes

- Sets missing current storage version in `uniques` pallet
- Adds multisig V1 migration to run where it was missing
- Removes executed migration whos pre/post hooks were failing from
collectives runtime
- Initializes storage versions for SP pallets added after genesis
- Originally I was going to wait for
https://github.com/paritytech/polkadot-sdk/pull/1297 to be merged so
this wouldn't need to be done manually, but it doesn't seem like it'll
be merged any time soon so I've decided to set them manually to unblock
this

## CI changes

- Removed dependency of `westend` runtime upgrades being complete prior
to other ones running. I assume it is supposed to cache the
`try-runtime` build for a performance benefit, but it seems it wasn't
working. Maybe someone from the CI team can look into this or explain
why it needs to be there?

- Adds check-runtime-migration jobs for Parity asset-hubs, bridge-hubs
and contract chains

- Updated VARIABLES to accomodate the `kusama-runtime` package being
renamed to `staging-kusama-runtime` in
https://github.com/paritytech/polkadot-sdk/pull/1241

- Added `EXTRA_ARGS` variable to `check-runtime-migration`, and set
`--no-weight-warnings` to the relay chain runtime upgrade checks (relay
chains don't have weight restrictions).
2023-10-03 02:11:56 +11:00
Bastian Köcher c54ea64af4 no-bound derives: Use absolute path for core (#1763)
Closes: https://github.com/paritytech/polkadot-sdk/issues/1718
2023-10-02 06:45:32 +11:00
Kian Paimani 2ed66a0960 migrate alliance, fast-unstake and bags list to use derive-impl (#1636)
Moving a few pallets to the latest and greatest `derive_impl` to give it
a try.

Part of #171

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2023-10-01 16:16:14 +02:00
Piet e8baac7848 Tvl pool staking (#1322)
What does this PR do?
- Introduced the TotalValueLocked storage for nomination-pools. 
- introduced a slashing api in mock.rs 
- additional test for tracking a slashing event towards a pool without
sub-pools
- migration for the nomination-pools (V6 to V7) with
`VersionedMigration`

Why are these changes needed?
this is the continuation of the work by @kianenigma in this
[PR](https://github.com/paritytech/substrate/pull/13319)

How were these changes implemented and what do they affect?
- It's an extra StorageValue that's modified whenever funds flow in or
out of staking for any of the `bonded_account` of `BondedPools`
- The `PoolSlashed`event is now emitted even when no `SubPools` are
found

Closes https://github.com/paritytech/polkadot-sdk/issues/155
KSM: HHEEgVzcqL3kCXgsxSfJMbsTy8dxoTctuXtpY94n4s8F4pS

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: command-bot <>
2023-10-01 03:36:48 +02:00
yjh 8fe947af60 improve service error (#1734)
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-30 07:52:47 +03:00
Muharem 7d4f82968c frame-support: RuntimeDebug\Eq\PartialEq impls for Imbalance (#1717)
Derive `RuntimeDebug\Eq\PartialEq` but do not bound any generics.

This achieved by using their equivalent no bound versions:
`EqNoBound\PartialEqNoBound\RuntimeDebugNoBound`.

Deriving with `Debug`, `Eq`, and `PartialEq` for the `Debt` and `Credit`
type aliases of `Imbalance` is not feasible due to the `OnDrop` and
`OppositeOnDrop` generic types lacking implementations of the same
traits.

This absence posed challenges in testing and any scenarios that demanded
the traits implementations for the type.
2023-09-29 23:47:17 +02:00
Sebastian Kunert 9485b0b492 Point documentation links to monorepo (#1741) 2023-09-29 22:33:26 +02:00
Ankan f820dc0a1f [NPoS] Fix for Reward Deficit in the pool (#1255)
closes https://github.com/paritytech/polkadot-sdk/issues/158.
partially addresses
https://github.com/paritytech/polkadot-sdk/issues/226.

Instead of fragile calculation of current balance by looking at `free
balance - ED`, Nomination Pool now freezes ED in the pool reward account
to restrict an account from going below minimum balance. This also has a
nice side effect that if ED changes, we know how much is the imbalance
in ED frozen in the pool and the current required ED. A pool operator
can diligently top up the pool with the deficit in ED or vice versa,
withdraw the excess they transferred to the pool.

## Notable changes
- New call `adjust_pool_deposit`: Allows to top up the deficit or
withdraw the excess deposited funds to the pool.
- Uses Fungible trait (instead of Currency trait). Since NP was not
doing any locking/reserving previously, no migration is needed for this.
- One time migration of freezing ED from each of the existing pools (not
very PoV friendly but fine for relay chain).
2023-09-29 17:48:40 +02:00
Dmitry Markin 0691c91e15 Move import queue from ChainSync to SyncingEngine (#1736)
This PR is part of [Sync
2.0](https://github.com/paritytech/polkadot-sdk/issues/534) refactoring
aimed at making `ChainSync` a pure state machine.

Resolves https://github.com/paritytech/polkadot-sdk/issues/501.
2023-09-29 17:58:16 +03:00
Piotr Mikołajczyk d8d90a82a7 Enable mocking contracts (#1331)
# Description
This PR introduces two changes:
- the previous `Tracing` trait has been modified to accept contract
address instead of code hash (seems to be way more convenient)
- a new trait `CallInterceptor` that allows intercepting contract calls;
in particular the default implementation for `()` will just proceed in a
standard way (after compilation optimizations, there will be no
footprint of that); however, implementing type might decide to mock
invocation and return `ExecResult` instead

Note: one might try merging `before_call` and `intercept_call`. However,
IMHO this would be bad, since it would mix two completely different
abstractions - tracing without any effects and actual intervention into
execution process.

This will unblock working on mocking contracts utility in drink and
similar tools (https://github.com/Cardinal-Cryptography/drink/issues/33)

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling
requirements](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md#process)
of this project (at minimum one label for `T` required)
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)
2023-09-29 15:39:13 +02:00
Sebastian Kunert 4902db2198 Use Extensions to register offchain worker custom extensions (#1719)
Closes #1671 

Adds a `type_id` function to the `Extension` trait, allowing to properly store an retrieve
boxed `Extensions`.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-29 09:37:51 +02:00
Tadeo Hepperle 379be3d7c3 [RPC-Spec-V2] chainHead: use integer for block index and adjust RuntimeVersion JSON format (#1666)
This PR adjusts the serialized format of the the returned RuntimeVersion
in the rpc-spec-v2 methods. This is done to match the format defined
here:
https://paritytech.github.io/json-rpc-interface-spec/api/chainHead_unstable_follow.html#about-the-runtime

- ##### `apis` field as object
`apis` field of `RuntimeVersion` is now returned as an object, e.g. 
```
"apis": {
      "0xdf6acb689907609b": 3,
      "0x37e397fc7c91f5e4": 1,
}
```
instead of 
```
"apis": [
      ["0xdf6acb689907609b", 3],
      ["0x37e397fc7c91f5e4", 1],
]
```
- ##### removed `stateVersion` and `authoringVersion`
`stateVersion` and `authoringVersion` are no longer returned in the
`RuntimeVersion` JSON Object.

- ##### block index in chain head events as integer

### Related Issues

Closes: #1507
Closes: #1146

### Testing Done
Adjusted existing tests to make sure data is returned in the correct
format.
2023-09-29 09:08:19 +02:00
btwiuse 7b9861a2f0 Fix subkey inspect output text padding (#1744)
This pull request is to fix the output text misalignment of `subkey
inspect` command:

(the `Network ID` line has an extra space at the end, and the `Secret
seed` line lacks a leading space)

```
[btwiuse@railway ~]$ subkey inspect '' | cat -A
Secret Key URI `` is account:$
  Network ID:        substrate $
 Secret seed:       0xfac7959dbfe72f052e5a0c3c8d6530f202b02fd8f9f5ca3580ec8deb7797479e$
  Public key (hex):  0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a$
  Account ID:        0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a$
  Public key (SS58): 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV$
  SS58 Address:      5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV$
```

The output should be in the same YAML-like format as `subkey generate`:

```
[btwiuse@railway ~]$ subkey generate  | cat -A
Secret phrase:       awkward eagle survey resemble novel resist modify memory pistol shed flower run$
  Network ID:        substrate$
  Secret seed:       0x20502f79366325b7dc7620664e8844ae69d441baf6e5b571a57d3b3ff28e9586$
  Public key (hex):  0x468874b9e5b6b77333fa702b9201b924d6834bf956e33e2bbe37d131134ca830$
  Account ID:        0x468874b9e5b6b77333fa702b9201b924d6834bf956e33e2bbe37d131134ca830$
  Public key (SS58): 5DfBkAMg5xQmsePFr3BWLZm99smiNyy9axWSgembvFgDKh9v$
  SS58 Address:      5DfBkAMg5xQmsePFr3BWLZm99smiNyy9axWSgembvFgDKh9v$
```

This change will fix `subkey` as well as all binaries that embed it as
the `key` subcommand, for example: `substrate key`, `polkadot key`, etc.

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-09-29 05:36:40 +10:00
Alexandru Vasile 945ebbbcf6 archive: Implement height, hashByHeight and call (#1582)
This PR implements:
- `archive_unstable_finalized_height`: Get the height of the most recent
finalized block
- `archive_unstable_hash_by_height`: Get the hashes (possible empty) of
blocks from the given height
- `archive_unstable_call`: Call into the runtime of a block

Builds on top of: https://github.com/paritytech/polkadot-sdk/pull/1560

### Testing Done
- unit tests for the methods with custom block tree for different
heights / forks

Closes: https://github.com/paritytech/polkadot-sdk/issues/1510
Closes: https://github.com/paritytech/polkadot-sdk/issues/1513
Closes: https://github.com/paritytech/polkadot-sdk/issues/1511

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-09-28 20:20:56 +03:00
Alexandru Vasile b5a0708fb7 rpc/client: Propagate rpc_methods method to reported methods (#1713)
The PR exposes the `rpc_methods` name in the `rpc_methods` response.
This feature is useful for servers that only forward requests to methods
that are reported by the `rpc_methods`.

Jsonrpsee exposes the available methods registered so far. Registering
the `rpc_methods` requires a closure that already stores the result. As
such, the `rpc_methods` method name is manually added to the available
methods.

Closes: https://github.com/paritytech/polkadot-sdk/issues/1627

### Testing Done

```
$> curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "rpc_methods", "params":[]}' http://localhost:9944

{"jsonrpc":"2.0","result":{"methods":["account_nextIndex",... "rpc_methods",..."unsubscribe_newHead"]},"id":1}⏎
```

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-09-28 20:19:27 +03:00
Xavier Lau de71fecc4e Add MaxTipAmount for pallet-tips (#1709)
Last week we experienced a governance attack.
Surprisingly, there was no upper limit on the tip amount.

Due to the mechanism of pallet-fragment-election, the council members
will be refreshed immediately. Attacker is easy to control the council
and give a large tip amount.
2023-09-28 21:08:05 +10:00
Xiliang Chen b50d8e6f7f add some events for pallet-bounties (#1706)
Add missing events for pallet-bounties
2023-09-28 21:04:35 +10:00
Dmitry Markin 14e5d23348 Move requests-responses and polling from ChainSync to SyncingEngine (#1650)
Move request-response handling from `ChainSync` to `SyncingEngine` as
part of [Sync
2.0](https://github.com/paritytech/polkadot-sdk/issues/534) refactoring
aimed at making `ChainSync` a pure state machine.

Resolves https://github.com/paritytech/polkadot-sdk/issues/502.

---------

Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-27 19:44:37 +03:00
Dónal Murray 02284a3e82 Add custom error message for StorageNoopGuard (#1727)
Expand `StorageNoopGuard` to be able to add extra context through a
custom error message. When the guard is triggered it panics with an
error message which can be defaulted, set on construction, or set after
it has been constructed.

Turn `StorageNoopGuard` into struct with `storage_root` and
`error_message` and added `from_error_message` constructor and
`set_error_message` setter.

Also added `new()` aliased to `default()`.

Closes #375

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-09-27 19:39:45 +03:00
Joshy Orndorff ec9274eea6 remove unnecessary hash string (#1722)
This PR removes some unnecessary `r#`...`#` around a string and the
corresponding comment that it was done because rustfmt wasn't working
for "some reason". It seems to work fine now and clippy prefers it this
way.

---------

Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
2023-09-27 18:48:24 +03:00
Matteo Muraca 8b061a5c5d Associated type Hasher for QueryPreimage, StorePreimage and Bounded (#1720)
I hope it's enough to fix #1701 
the only solution I found to make it happen is to put an associated type
to the `Bounded` enum as well.
@liamaharon @kianenigma @bkchr 

Polkadot address: 12poSUQPtcF1HUPQGY3zZu2P8emuW9YnsPduA4XG3oCEfJVp

---------

Signed-off-by: muraca <mmuraca247@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-09-27 22:46:14 +10:00
Michal Kucharczyk 5a2833cceb genesis-builder: implemented for all runtimes (#1492)
This PR implements [`GenesisBuilder`
API](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/substrate/primitives/genesis-builder/src/lib.rs#L38)
for all the runtimes in polkadot repo.

Step towards: paritytech/polkadot-sdk#25

---------

Co-authored-by: ordian <write@reusable.software>
2023-09-27 11:59:19 +02:00
Michal Kucharczyk ab3a3bc278 BlockId removal: tx-pool refactor (#1678)
It changes following APIs:
- trait `ChainApi`
-- `validate_transaction`

- trait `TransactionPool` 
--`submit_at`
--`submit_one`
--`submit_and_watch`

and some implementation details, in particular:
- impl `Pool` 
--`submit_at`
--`resubmit_at`
--`submit_one`
--`submit_and_watch`
--`prune_known`
--`prune`
--`prune_tags`
--`resolve_block_number`
--`verify`
--`verify_one`

- revalidation queue

All tests are also adjusted.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-27 11:58:39 +02:00
dependabot[bot] a846b74604 Bump directories from 4.0.1 to 5.0.1 (#1656)
Bumps [directories](https://github.com/soc/directories-rs) from 4.0.1 to
5.0.1.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/soc/directories-rs/commits">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=directories&package-manager=cargo&previous-version=4.0.1&new-version=5.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-26 23:25:46 +02:00
Alexandru Vasile cc50eda098 chainHead/storage: Fix storage iteration using the query key (#1665)
This PR ensures that all storage keys under a prefix are returned by the
`chainHead_storage` method.

Before this PR, the `storage_keys` was used with just the `start_key`.
Before the pagination event was generated, the last reported key
`last_key` was saved internally.
When the pagination is resumed, the `last_key` will serve as the next
`start_key` to the `storage_keys` API.

However, this behavior does not function properly for non-prefixed
storage keys.

Entry keys `a`, `ab`, `abc` share a common prefix and therefore the `ab`
key leads to `abc`.
However, for `a`, `ab`, `aB` and `abc`, the `aB` key does not
immediately lead to `abc`.

To mitigate this, the PR saves the start key of the query, together with
the next pagination key.
Improve testing to ensure we have a key entry that doesn't share the
prefix with the descendant key.

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-09-26 14:36:24 +03:00
Nazar Mokrynskyi bc5005217a Implement more useful traits in Slot type (#1595)
I had quite a few conversions to/from `u64` because these were lacking.

Let me know if others are desirable as well for symmetry.
2023-09-26 00:43:36 +02:00
Kevin Wang e8da320734 contracts: Fix incorrect storage alias in mirgration (#1687)
# Description

We are recently trying to upgrade our Substrate version from
`polkadot-v0.9.43` to `polkadot-v1.0.0` and we noticed a critical issue:
all deployed contracts seem to be experiencing a `CodeNotFound` error.
After a thorough investigation, it appears that the root cause of this
issue lies in the mismatch between the storage alias of `CodeInfoOf<T>`
in the migration and its original definition.

This PR corrects the storage alias to align it with its original
definition

I am uncertain about the proper approach for adding tests to this
change. Would the team consider taking over this PR to bring it to
completion?

---------

Co-authored-by: pgherveou <pgherveou@gmail.com>
2023-09-25 15:13:11 +02:00
Nazar Mokrynskyi c0a4ce1fc8 Fix documentation about justification and finalized == true requirement (#1607)
I was reading source code and noticed these two places that are no
longer true after https://github.com/paritytech/polkadot-sdk/pull/1211
2023-09-25 12:56:43 +02:00
Kian Paimani b5fcdff9f5 tweak pallet macro (genesis_config etc) to cater for RA users as well. (#1689)
Small tweak to https://github.com/paritytech/polkadot-sdk/pull/1642 to
incorporate the ideas from
https://github.com/paritytech/polkadot-sdk/issues/247.

I think this is the good middle ground, where we have good rust-docs,
and the RA users will also have some hope.

cc @wentelteefje @aaronbassett @sam0x17
2023-09-25 11:38:02 +02:00
Maksym H 3d9f5e8bfc update contributing guide and ui-tests scripts (#1668)
- Updated ./docs/CONTRIBUTION.md to clarify how to use command bot and
update ui-tests locally or in PR
- Moved update-ui-tests.sh to a root of project to run them along for
substrate and polkadot together

---------

Co-authored-by: Mira Ressel <mira@parity.io>
2023-09-22 22:48:38 +01:00
Niklas Adolfsson f79fa6c8bd pallet epm: add TrimmingStatus to the mined solution (#1659)
For tools such that is using the `Miner` it's useful to know whether a
solution was trimmed or not and also how much that was trimmed.

---------

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
2023-09-22 19:22:18 +02:00
Kian Paimani b09ab37119 update genesis_config related docs and tests and error messages (#1642)
Follow-up to https://github.com/paritytech/substrate/pull/14306.

I hope this also showcases the important message of: **It is really not
that hard to make the examples codes in rust-docs compile, and therefore
remain correct. Please embrace this :)**

It moves the documentation of proc macros to their re-export, such that
can link other items in frame-support. This is a patter that we should
embrace for all of macro docs, and apply in PRs like
https://github.com/paritytech/substrate/pull/13987 as well.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: command-bot <>
2023-09-22 12:48:48 +01:00