Changes:
- Add CI script to check that the `crate` names that are mentioned in
prdocs are valid.
We can extend it lateron to also validate the correct SemVer bumps as
introduced in https://github.com/paritytech/polkadot-sdk/pull/3441.
Example output:
```pre
$ python3 .github/scripts/check-prdoc.py Cargo.toml prdoc/*.prdoc
🔎 Reading workspace polkadot-sdk/Cargo.toml.
📦 Checking 36 prdocs against 494 crates.
✅ All prdocs are valid.
```
Note that not all old prdocs pass the check since crates have been
renamed:
```pre
$ python3 .github/scripts/check-prdoc.py Cargo.toml prdoc/**/*.prdoc
🔎 Reading workspace polkadot-sdk/Cargo.toml.
📦 Checking 186 prdocs against 494 crates.
❌ Some prdocs are invalid.
💥 prdoc/1.4.0/pr_1926.prdoc lists invalid crate: node-cli
💥 prdoc/1.4.0/pr_2086.prdoc lists invalid crate: xcm-executor
💥 prdoc/1.4.0/pr_2107.prdoc lists invalid crate: xcm
💥 prdoc/1.6.0/pr_2684.prdoc lists invalid crate: xcm-builder
```
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
While investigating some pruning issues I found some room for
improvement in the notification pin handling.
**Problem:** It was not possible to define an upper limit on
notification pins. The block pinning cache has a limit, but only handles
bodies and justifications.
After this PR, bookkeeping for notifications is managed in the pinning
worker. A limit can be defined in the worker. If that limit is crossed,
blocks that were pinned for that notification are unpinned, which now
affects the state as well as bodies and justifications. The pinned
blocks cache still has a limit, but should never be hit.
closes#19
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Instead of only generating the error, we now generate the actual code
and the error. This generates in total less errors and helps the user to
identify the actual problem and not being confronted with tons of
errors.
with the deprecation of Rococo, Encointer needs a new staging
environment. Paseo will be Polkadot-focused and westend Kusama-focused,
so we propose to use Westend
## Problem
During the bumping of the `polkadot-fellows` repository to
`polkadot-sdk@1.6.0`, I encountered a situation where the benchmarks
`teleport_assets` and `reserve_transfer_assets` in AssetHubKusama
started to fail. This issue arose due to a decreased ED balance for
AssetHubs introduced
[here](https://github.com/polkadot-fellows/runtimes/pull/158/files#diff-80668ff8e793b64f36a9a3ec512df5cbca4ad448c157a5d81abda1b15f35f1daR213),
and also because of a [missing CI
pipeline](https://github.com/polkadot-fellows/runtimes/issues/197) to
check the benchmarks, which went unnoticed.
These benchmarks expect the `caller` to have enough:
1. balance to transfer (BTT)
2. balance for paying delivery (BFPD).
So the initial balance was calculated as `ED * 100`, which seems
reasonable:
```
const ED_MULTIPLIER: u32 = 100;
let balance = existential_deposit.saturating_mul(ED_MULTIPLIER.into());`
```
The problem arises when the price for delivery is 100 times higher than
the existential deposit. In other words, when `ED * 100` does not cover
`BTT` + `BFPD`.
I check AHR/AHW/AHK/AHP and this problem has only AssetHubKusama
```
ED: 3333333
calculated price to parent delivery: 1031666634 (from xcm logs from the benchmark)
---
3333333 * 100 - BTT(3333333) - BFPD(1031666634) = −701666667
```
which results in the error;
```
2024-02-23 09:19:42 Unable to charge fee with error Module(ModuleError { index: 31, error: [17, 0, 0, 0], message: Some("FeesNotMet") })
Error: Input("Benchmark pallet_xcm::reserve_transfer_assets failed: FeesNotMet")
```
## Solution
The benchmarks `teleport_assets` and `reserve_transfer_assets` were
fixed by removing `ED * 100` and replacing it with `DeliveryHelper`
logic, which calculates the (almost real) price for delivery and sets it
along with the existential deposit as the initial balance for the
account used in the benchmark.
## TODO
- [ ] patch for 1.6 -
https://github.com/paritytech/polkadot-sdk/pull/3466
- [ ] patch for 1.7 -
https://github.com/paritytech/polkadot-sdk/pull/3465
- [ ] patch for 1.8 - TODO: PR
---------
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Add more debug logs to understand if statement-distribution is in a bad
state, should be useful for debugging
https://github.com/paritytech/polkadot-sdk/issues/3314 on production
networks.
Additionally, increase the number of parallel requests should make,
since I notice that requests take around 100ms on kusama, and the 5
parallel request was picked mostly random, no reason why we can do more
than that.
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: ordian <write@reusable.software>
Changes the runtime hash algorithm used in
`resolve_state_version_from_wasm` from `DefaultHasher` to a
caller-provided one (usually `HashingFor<Block>`), to match the one used
elsewhere.
This fixes an issue where the runtime wasm is compiled 3 times when
starting the `tanssi-node` with `--dev`. With this fix, the runtime wasm
is only compiled 2 times. The other redundant compilation is caused by
the `GenesisConfigBuilderRuntimeCaller` struct, which ignores the
runtime cache.
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Fixes https://github.com/paritytech/polkadot-sdk/issues/3144
Builds on top of https://github.com/paritytech/polkadot-sdk/pull/3229
### Summary
Some preparations for Runtime to support elastic scaling, guarded by
config node features bit `FeatureIndex::ElasticScalingMVP`. This PR
introduces a per-candidate `CoreIndex` but does it in a hacky way to
avoid changing `CandidateCommitments`, `CandidateReceipts` primitives
and networking protocols.
#### Including `CoreIndex` in `BackedCandidate`
If the `ElasticScalingMVP` feature bit is enabled then
`BackedCandidate::validator_indices` is extended by 8 bits.
The value stored in these bits represents the assumed core index for the
candidate.
It is temporary solution which works by creating a mapping from
`BackedCandidate` to `CoreIndex` by assuming the `CoreIndex` can be
discovered by checking in which validator group the validator that
signed the statement is.
TODO:
- [x] fix tests
- [x] add new tests
- [x] Bump runtime API for Kusama, so we have that node features thing!
-> https://github.com/polkadot-fellows/runtimes/pull/194
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: alindima <alin@parity.io>
Co-authored-by: alindima <alin@parity.io>
Related to https://github.com/paritytech/polkadot-sdk/issues/3400
Moving all bridges testing "framework" files under one folder in order
to be able to download the entire folder when we want to add tests in
other repos
No significant functional changes
# Runtime side for PoV Reclaim
## Implementation Overview
- Hostfunction to fetch the storage proof size has been added to the
PVF. It uses the size tracking recorder that was introduced in my
previous PR.
- Mechanisms to use the reclaim HostFunction have been introduced.
- 1. A SignedExtension that checks the node-reported proof size before
and after application of an extrinsic. Then it reclaims the difference.
- 2. A manual helper to make reclaiming easier when manual interaction
is required, for example in `on_idle` or other hooks.
- In order to utilize the manual reclaiming, I modified `WeightMeter` to
support the reduction of consumed weight, at least for storage proof
size.
## How to use
To enable the general functionality for a parachain:
1. Add the SignedExtension to your parachain runtime.
2. Provide the HostFunction to the node
3. Enable proof recording during block import
## TODO
- [x] PRDoc
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
## Summary
* use benchamarked weights instead of hardcoded ones for
`pallet-membership`
* rename benchmark to match extrinsic name
* remove unnecessary dependency from `clear_prime`
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Add a `ApiVersion` constant to the pallet-contracts Config to
communicate with developers the current state of the host functions
exposed by the pallet
The `fee` should be calculated with the reanchored asset, otherwise it
could lead to a failure where the set aside fee ends up not being
enough.
@acatangiu
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
This introduces a check to ensure that the parachain code matches the
validation code stored in the relay chain state. If not, it will print a
warning. This should be mainly useful for parachain builders to make
sure they have setup everything correctly.
As part of BEEFY worker/voter initialization the task waits for certain
chain and backend conditions to be fulfilled:
- BEEFY consensus enabled on-chain & GRANDPA best finalized higher than
on-chain BEEFY genesis block,
- backend has synced headers for BEEFY mandatory blocks between best
BEEFY and best GRANDPA.
During this waiting time, any messages gossiped on the BEEFY topic for
current chain get enqueued in the gossip engine, leading to RAM bloating
and output warning/error messages when the wait time is non-negligible
(like during a clean sync).
This PR adds logic to pump the gossip engine while waiting for other
things to make sure gossiped messages get consumed (practically
discarded until worker is fully initialized).
Also raises the warning threshold for enqueued messages from 10k to
100k. This is in line with the other gossip protocols on the node.
Fixes https://github.com/paritytech/polkadot-sdk/issues/3390
---------
Signed-off-by: Adrian Catangiu <adrian@parity.io>
This PR fixes a subtle bug in `wasm-builder` first introduced in
https://github.com/paritytech/polkadot-sdk/pull/1851 (sorry, my bad! I
should have caught this during review) where the status code of the
`cargo` subprocess is not properly checked, which results in builds
silently succeeding when they shouldn't (that is: if we successfully
build a runtime blob, and then modify the code so that it won't compile,
and recompile it again, then the build will succeed and silently use the
*old* blob).
cc @athei This is the bug you were seeing.
[edit]Also fixes a similar PolkaVM-specific bug where I accidentally
used the wrong comparison operator.[/edit]
First step in implementing
https://github.com/paritytech/polkadot-sdk/issues/3144
### Summary of changes
- switch statement `Table` candidate mapping from `ParaId` to
`CoreIndex`
- introduce experimental `InjectCoreIndex` node feature.
- determine and assume a `CoreIndex` for a candidate based on statement
validator index. If the signature is valid it means validator controls
the validator that index and we can easily map it to a validator
group/core.
- introduce a temporary provisioner fix until we fully enable elastic
scaling in the subystem. The fix ensures we don't fetch the same
backable candidate when calling `get_backable_candidate` for each core.
TODO:
- [x] fix backing tests
- [x] fix statement table tests
- [x] add new test
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: alindima <alin@parity.io>
Co-authored-by: alindima <alin@parity.io>
In Tanssi, we need a way to stop the collator code and then start it
again. This is to support rotating the same collator between different
runtimes. Currently, this works very well, except for the proposer
metrics, because they only get registered the first time they are
started. Afterwards, we see this warning log:
> Failed to register proposer prometheus metrics: Duplicate metrics
collector registration attempted
~~So this PR adds a method to set metrics, to allow us to register
metrics manually before creating the `ProposerFactory`, and then clone
the same metrics every time we need to start the collator.~~ Implemented
Clone instead
This PR backports the changes from the rpc-v2 spec:
https://github.com/paritytech/json-rpc-interface-spec/pull/134
The `Broadcasted` event has been removed:
- it is hard to enforce a `Dropped { broadcasted: bool }` event in cases
of a load-balancer being placed in front of an RPC server
- when the server exists, it is impossible to guarantee this field if
the server did not previously send a `Broadcasted` event
- the number of peers reported by this event does not guarantee that
peers are unique
- the same peer can disconnect and reconnect, increasing this metric
number
- the number of peers that receive this transaction offers no guarantee
about the transaction being included in the chain at a later time
cc @paritytech/subxt-team
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
closes#2844
- adds `list-pallets` option which prints all unique available pallets
for benchmarking
```bash
./target/release/node benchmark pallet --list=pallets
```
- adds `all` option which runs benchmarks for all available pallets and
extrinsics (equivalent to `--pallet * --extrinsic *`)
```bash
./target/release/node benchmark pallet --all
```
- use the `list=pallets` syntax in `run_all_benchmarks.sh` script
cc ggwpez
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
- Adds a test to check the correct digest for Snowbridge outbound
messages. For the correct digest to be in the block, the the
MessageQueue pallet should be configured after the EthereumOutbound
queue pallet. The added test fails if the EthereumOutbound is configured
after the MessageQueue pallet.
- Adds a helper method `run_to_block_with_finalize` to simulate the
block finalizing. The existing `run_to_block` method does not finalize
and so it cannot successfully test this condition.
Closes: https://github.com/paritytech/polkadot-sdk/issues/3208
---------
Co-authored-by: claravanstaden <Cats 4 life!>
This PR is fixing a bug in the sync mechanism between wasmi and
pallet-contracts. This bug leads to essentially double charging all the
gas that was used during the execution of the host function. When the
`call` host function is used for recursion this will lead to a quadratic
amount of gas consumption with regard to the nesting depth.We also took
the chance to refactor the code in question and improve the rust docs.
The bug was caused by not updating `GasMeter::executor_consumed`
(previously `engine_consumed`) when leaving the host function. This lead
to the value being stale (too low) when entering another host function.
---------
Co-authored-by: PG Herveou <pgherveou@gmail.com>
I dont think this bound is required by anything else and it allows now
to use this struct with runtimes that have 20 bytes accounts
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
The rationale behind this, is that it may be useful for some users
actually disable RPC batch requests or limit them by length instead of
the total size bytes of the batch.
This PR adds two new CLI options:
```
--rpc-disable-batch-requests - disable batch requests on the server
--rpc-max-batch-request-len <LEN> - limit batches to LEN on the server.
```
Lifting some more dependencies to the workspace. Just using the
most-often updated ones for now.
It can be reproduced locally.
```sh
# First you can check if there would be semver incompatible bumps (looks good in this case):
$ zepter transpose dependency lift-to-workspace --ignore-errors syn quote thiserror "regex:^serde.*"
# Then apply the changes:
$ zepter transpose dependency lift-to-workspace --version-resolver=highest syn quote thiserror "regex:^serde.*" --fix
# And format the changes:
$ taplo format --config .config/taplo.toml
```
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
## Xcm changes:
- Fix `pallet_xcm::execute`, move the logic into The `ExecuteController`
so it can be shared with anything that implement that trait.
- Make `ExecuteController::execute` retursn `DispatchErrorWithPostInfo`
instead of `DispatchError`, so that we don't charge the full
`max_weight` provided if the execution is incomplete (useful for
force_batch or contracts calls)
- Fix docstring for `pallet_xcm::execute`, to reflect the changes from
#2405
- Update the signature for `ExecuteController::execute`, we don't need
to return the `Outcome` anymore since we only care about
`Outcome::Complete`
## Contracts changes:
- Update host fn `xcm_exexute`, we don't need to write the `Outcome` to
the sandbox memory anymore. This was also not charged as well before so
it if fixes this too.
- One of the issue was that the dry_run of a contract that call
`xcm_execute` would exhaust the `gas_limit`.
This is because `XcmExecuteController::execute` takes a `max_weight`
argument, and since we don't want the user to specify it manually we
were passing everything left by pre-charghing
`ctx.ext.gas_meter().gas_left()`
- To fix it I added a `fn influence_lowest_limit` on the `Token` trait
and make it return false for `RuntimeCost::XcmExecute`.
- Got rid of the `RuntimeToken` indirection, we can just use
`RuntimeCost` directly.
---------
Co-authored-by: command-bot <>
Adds the coretime and on demand pallets to enable Coretime on Westend.
In order for the migration to run successfully, we need the
Broker/Coretime parachain to be live.
TODO:
- [ ] Broker parachain is live
https://github.com/paritytech/polkadot-sdk/pull/3272
---------
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <info@kchr.de>
~The previous fix was actually incomplete because we update the
authorties only on the situation where we decided to reconnect because
we had a low connectivity issue. Now the problem is that
update_authority_ids use the list of connected peers, so on restart that
does contain anything, so calling immediately after
issue_connection_request won't detect all authorities, so we need to
also check every block as the comment said, but that did not match the
code.~
Actually the fix was correct the flow is follow if more than 1/3 of the
authorities can not be resolved we set last_failure and call
`ConnectToResolvedValidators`.
We will call UpdateAuthorities for all the authorities already connected
and for which we already know the address and for the ones that will
connect later on `PeerConnected` will have the AuthorityId field set,
because it is already known, so approval-distribution will update its
cache topology.
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
# Description
This PR removes redundant type definition from test definition config
implementations like
```
#[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::Config for Test {
type A = A;
...
}
```
This changes avoid redundancies in the code as the macro `derive_impl`
defines the relevant types. To implement the changes, it was a simple
fact of running tests and making sure that the tests would still run
while the definition would be removed.
Closes#3237
As a note, here is a brief account of things done from the Issue's
description statement
```
alliance migrate alliance, fast-unstake and bags list to use derive-impl #1636
asset-conversion DONE
asset-rate DONE
assets DONE
atomic-swap DONE
aura DONE
authority-discovery DONE
authorship migrate babe and authorship to use derive-impl #1790
babe migrate babe and authorship to use derive-impl #1790
bags-list migrate alliance, fast-unstake and bags list to use derive-impl #1636
balances DONE
beefy NOTHING TO DO --- also noted this error without failing tests Feb 13 13:49:08.941 ERROR runtime::timestamp: `pallet_timestamp::UnixTime::now` is called at genesis, invalid value returned: 0
beefy-mmr NOTHING TO DO
bounties DONE
child-bounties DONE
collective DONE
contracts DONE
conviction-voting DONE
core-fellowship NOTHING TO DO
democracy DONE
election-provider-multi-phase NOTHING TO DO
elections-phragmen DONE
executive NOTHING TO DO
fast-unstake migrate alliance, fast-unstake and bags list to use derive-impl #1636
glutton DONE
grandpa DONE
identity DONE
im-online NOTHING TO DO
indices Refactor indices pallet #1789
insecure-randomness-collective-flip DONE
lottery DONE
membership DONE
merkle-mountain-range NOTHING TO DO
message-queue DONE
multisig add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
nft-fractionalization DONE
nfts DONE
nicks Refactor pallet-state-trie-migration to fungible::* traits #1801 NOT IN REPO
nis DONE
node-authorization DONE
nomination-pools NOTHING TO DO -- ONLY impl for Runtime
offences DELETED EVERYTHING -- IS THAT CORRECT??
preimage DONE
proxy add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
ranked-collective NOTHING TO DO
recovery DONE
referenda DONE
remark DONE
root-offences DONE
root-testing NOTHING TO DO
salary NOTHING TO DO
scheduler DONE
scored-pool DONE
session DONE -- substrate/frame/session/benchmarking/src/mock.rs untouched
society NOTHING TO DO
staking DONE
staking-bags-benchmarks NOT IN REPO
state-trie-migration NOTHING TO DO
statement DONE
sudo DONE
system DONE
timestamp DONE
tips DONE
transaction-payment NOTHING TO DO
transaction-storage NOTHING TO DO
treasury DONE
try-runtime NOTHING TO DO -- no specific mention of 'for Test'
uniques DONE
utility DONE
vesting DONE
whitelist DONE
```
---------
Co-authored-by: command-bot <>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
This is a follow-up for `im-online` pallet removal that is cleaning up
its off-chain storage. Must be merged no earlier than #2265 is enacted.
Related: #1964
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Oliver Tale-Yazdi