This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
- Less dependencies for the clients in particular
- Return type requires Clone in method call responses
- Moved to tokio channels
- Async subscription API (not used in this PR)
Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate
The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.
Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.
The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.
Hopefully the last release prior to 1.0, sorry in advance for a big PR
Previous attempt: https://github.com/paritytech/substrate/pull/13992
Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
This PR aims to channel the backpressure of the PVF host's preparation
and execution queues to the candidate validation subsystem consumers.
Related: #708
Part of https://github.com/paritytech/polkadot-sdk/issues/239.
Invariant
1. The number of entries in `Tips` should be equal to `Reasons`.
2. If `OpenTip.finders_fee` is true, then `OpenTip.deposit` should be
greater than zero.
3. Reasons exists for each Tip[`OpenTip.reason`], implying equal length
of storage.
polkadot address: 12zsKEDVcHpKEWb99iFt3xrTCQQXZMu477nJQsTBBrof5k2h
---------
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
This PR introduces the following changes:
- [x] Adds a `UniquesTransactor` to asset-hub-rococo
- [x] Adds a `UniquesTransactor` to asset-hub-westend
We can't add a transactor for `pallet-nfts` like we do for
`pallet-uniques` because `pallet-nfts` uses `nonfungibles_v2::Mutate`
instead of `nonfungibles::Mutate`, and making that work would be out of
scope of this PR.
With these modifications, reserve-based NFT cross-chain transfers can be
performed on asset-hub.
---------
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
This PR backports version bumps from release branch
`release-polkadot-v1.6.0` back to `master` and also moved `prdoc` files
related to the release to the appropriate folder
# Note for reviewer
Most changes are just syntax changes necessary for the new version.
Most important files should be the ones under the `xcm` folder.
# Description
Added XCMv4.
## Removed `Multi` prefix
The following types have been renamed:
- MultiLocation -> Location
- MultiAsset -> Asset
- MultiAssets -> Assets
- InteriorMultiLocation -> InteriorLocation
- MultiAssetFilter -> AssetFilter
- VersionedMultiAsset -> VersionedAsset
- WildMultiAsset -> WildAsset
- VersionedMultiLocation -> VersionedLocation
In order to fix a name conflict, the `Assets` in `xcm-executor` were
renamed to `HoldingAssets`, as they represent assets in holding.
## Removed `Abstract` asset id
It was not being used anywhere and this simplifies the code.
Now assets are just constructed as follows:
```rust
let asset: Asset = (AssetId(Location::new(1, Here)), 100u128).into();
```
No need for specifying `Concrete` anymore.
## Outcome is now a named fields struct
Instead of
```rust
pub enum Outcome {
Complete(Weight),
Incomplete(Weight, Error),
Error(Error),
}
```
we now have
```rust
pub enum Outcome {
Complete { used: Weight },
Incomplete { used: Weight, error: Error },
Error { error: Error },
}
```
## Added Reanchorable trait
Now both locations and assets implement this trait, making it easier to
reanchor both.
## New syntax for building locations and junctions
Now junctions are built using the following methods:
```rust
let location = Location {
parents: 1,
interior: [Parachain(1000), PalletInstance(50), GeneralIndex(1984)].into()
};
```
or
```rust
let location = Location::new(1, [Parachain(1000), PalletInstance(50), GeneralIndex(1984)]);
```
And they are matched like so:
```rust
match location.unpack() {
(1, [Parachain(id)]) => ...
(0, Here) => ...,
(1, [_]) => ...,
}
```
This syntax is mandatory in v4, and has been also implemented for v2 and
v3 for easier migration.
This was needed to make all sizes smaller.
# TODO
- [x] Scaffold v4
- [x] Port github.com/paritytech/polkadot/pull/7236
- [x] Remove `Multi` prefix
- [x] Remove `Abstract` asset id
---------
Co-authored-by: command-bot <>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Original PR https://github.com/paritytech/substrate/pull/14655
---
Partial https://github.com/paritytech/polkadot-sdk/issues/225
- [x] Adds conformance tests for Unbalanced
- [x] Adds conformance tests for Balanced
- Several minor fixes to fungible default implementations and the
Balances pallet
- [x] `Unbalanced::decrease_balance` can reap account when
`Preservation` is `Preserve`
- [x] `Balanced::pair` can return pairs of imbalances which do not
cancel each other out
- [x] Balances pallet `active_issuance` 'underflow'
- [x] Refactors the conformance test file structure to match the
fungible file structure: tests for traits in regular.rs go into a test
file named regular.rs, tests for traits in freezes.rs go into a test
file named freezes.rs, etc.
- [x] Improve doc comments
- [x] Simplify macros
## Fixes
### `Unbalanced::decrease_balance` can reap account when called with
`Preservation::Preserve`
There is a potential issue in the default implementation of
`Unbalanced::decrease_balance`. The implementation can delete an account
even when it is called with `preservation: Preservation::Preserve`. This
seems to contradict the documentation of `Preservation::Preserve`:
```rust
/// The account may not be killed and our provider reference must remain (in the context of
/// tokens, this means that the account may not be dusted).
Preserve,
```
I updated `Unbalanced::decrease_balance` to return
`Err(TokenError::BelowMinimum)` when a withdrawal would cause the
account to be reaped and `preservation: Preservation::Preserve`.
- [ ] TODO Confirm with @gavofyork that this is correct behavior
Test for this behavior:
https://github.com/paritytech/polkadot-sdk/blob/e5c876dd6b59e2b7dbacaa4538cb42c802db3730/substrate/frame/support/src/traits/tokens/fungible/conformance_tests/regular.rs#L912-L937
### `Balanced::pair` returning non-canceling pairs
`Balanced::pair` is supposed to create a pair of imbalances that cancel
each other out. However this is not the case when the method is called
with an amount greater than the total supply.
In the existing default implementation, `Balanced::pair` creates a pair
by first rescinding the balance, creating `Debt`, and then issuing the
balance, creating `Credit`.
When creating `Debt`, if the amount to create exceeds the
`total_supply`, `total_supply` units of `Debt` are created *instead* of
`amount` units of `Debt`. This can lead to non-canceling amount of
`Credit` and `Debt` being created.
To address this, I create the credit and debt directly in the method
instead of calling `issue` and `rescind`.
Test for this behavior:
https://github.com/paritytech/polkadot-sdk/blob/e5c876dd6b59e2b7dbacaa4538cb42c802db3730/substrate/frame/support/src/traits/tokens/fungible/conformance_tests/regular.rs#L1323-L1346
### `Balances` pallet `active_issuance` 'underflow'
This PR resolves an issue in the `Balances` pallet that can lead to odd
behavior of `active_issuance`.
Currently, the Balances pallet doesn't check if `InactiveIssuance`
remains less than or equal to `TotalIssuance` when supply is
deactivated. This allows `InactiveIssuance` to be greater than
`TotalIssuance`, which can result in unexpected behavior from the
perspective of the fungible API.
`active_issuance` is derived from
`TotalIssuance.saturating_sub(InactiveIssuance)`.
If an `amount` is deactivated that causes `InactiveIssuance` to become
greater TotalIssuance, `active_issuance` will return 0. However once in
that state, reactivating an amount will not increase `active_issuance`
by the reactivated `amount` as expected.
Consider this test where the last assertion would fail due to this
issue:
https://github.com/paritytech/polkadot-sdk/blob/e5c876dd6b59e2b7dbacaa4538cb42c802db3730/substrate/frame/support/src/traits/tokens/fungible/conformance_tests/regular.rs#L1036-L1071
To address this, I've modified the `deactivate` function to ensure
`InactiveIssuance` never surpasses `TotalIssuance`.
---------
Co-authored-by: Muharem <ismailov.m.h@gmail.com>
Extract `WarpSync` (and `StateSync` as part of warp sync) from
`ChainSync` as independent syncing strategy called by `SyncingEngine`.
Introduce `SyncingStrategy` enum as a proxy between `SyncingEngine` and
specific syncing strategies.
## Limitations
Gap sync is kept in `ChainSync` for now because it shares the same set
of peers as block syncing implementation in `ChainSync`. Extraction of a
common context responsible for peer management in syncing strategies
able to run in parallel is planned for a follow-up PR.
## Further improvements
A possibility of conversion of `SyncingStartegy` into a trait should be
evaluated. The main stopper for this is that different strategies need
to communicate different actions to `SyncingEngine` and respond to
different events / provide different APIs (e.g., requesting
justifications is only possible via `ChainSync` and not through
`WarpSync`; `SendWarpProofRequest` action is only relevant to
`WarpSync`, etc.)
---------
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Previously, it was only possible to retry the same request on a
different protocol name that had the exact same binary payloads.
Introduce a way of trying a different request on a different protocol if
the first one fails with Unsupported protocol.
This helps with adding new req-response versions in polkadot while
preserving compatibility with unupgraded nodes.
The way req-response protocols were bumped previously was that they were
bundled with some other notifications protocol upgrade, like for async
backing (but that is more complicated, especially if the feature does
not require any changes to a notifications protocol). Will be needed for
implementing https://github.com/polkadot-fellows/RFCs/pull/47
TODO:
- [x] add tests
- [x] add guidance docs in polkadot about req-response protocol
versioning
This removes the need to unnecessarily provide a very specific data
structure `DatabaseSource` and removes huge `sc-client-db` dependency
from storage monitor. It is now possible to use storage monitor with any
path.
P.S. I still strongly dislike that it pulls `clap` dependency for such a
small feature, but many other crates do as well, so nothing special
here.
This PR allows _username authorities_ to issue unique usernames that
correspond with an account. It also provides two-way lookup, that is
from `AccountId` to a single, "primary" `Username` (alongside
`Registration`) and multiple unique `Username`s to an `AccountId`.
Key features:
- Username Authorities added (and removed) via privileged origin.
- Authorities have a `suffix` and an `allocation`. They can grant up to
`allocation` usernames. Their `suffix` will be appended to the usernames
that they issue. A suffix may be up to 7 characters long.
- Users can ask an authority to grant them a username. This will take
the form `myusername.suffix`. The entire name (including suffix) must be
less than or equal to 32 alphanumeric characters.
- Users can approve a username for themselves in one of two ways (that
is, authorities cannot grant them arbitrarily):
- Pre-sign the entire username (including suffix) with a secret key that
corresponds to their `AccountId` (for keyed accounts, obviously); or
- Accept the username after it has been granted by an authority (it will
be queued until accepted) (for non-keyed accounts like pure proxies or
multisigs).
- The system does not require any funds or deposits. Users without an
identity will be given a default one (presumably all fields set to
`None`). If they update this info, they will need to place the normal
storage deposit.
- If a user does not have any username, their first one will be set as
`Primary`, and their `AccountId` will map to that one. If they get
subsequent usernames, they can choose which one to be their primary via
`set_primary_username`.
- There are some state cleanup functions to remove expired usernames
that have not been accepted and dangling usernames whose owners have
called `clear_identity`.
TODO:
- [x] Add migration to runtimes
- [x] Probably do off-chain migration into People Chain genesis
- [x] Address a few TODO questions in code (please review)
---------
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Closes#1591.
The purpose of this PR is filter out backing statements from the network
signed by disabled validators. This is just an optimization, since we
will do filtering in the runtime in #1863 to avoid nodes to filter
garbage out at block production time.
- [x] Ensure it's ok to fiddle with the mask of manifests
- [x] Write more unit tests
- [x] Test locally
- [x] simple zombienet test
- [x] PRDoc
---------
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
To align with the documentation of the approve call, we import an
unimported member on approval.
No changes have been made to the benchmarks as they already cover the
worst-case scenario.
#1259 was merged into a feature branch, but we've decided to merge
node-side changes for disabling straight into master.
This is a dependency of #1841 and #2637.
---------
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
This changes `RelaychainDataProvider` to return the latest known relay
chain block number in `on_initialize` et all, aka when the
`validation_data` wasn't yet set by the inherent.
---------
Co-authored-by: Dónal Murray <donal.murray@parity.io>
This splits `cumulus-primitives-parachain-inherent` into two crates, the
previous `cumulus-primitives-parachain-inherent` and a new
`cumulus-client-parachain-inherent`. The idea behind this is to move the
`create_at` logic into the client crate. This removes quite a lot of
unrelated dependencies from the runtime std build and thus, makes the
compilation faster. On my Laptop the compilation is goes down by one
minute for `asset-hub-rococo-runtime`. I also assume that the full build
of the entire workspace probably can be speed-up a little bit, because
more stuff can be compiled in parallel.
---------
Co-authored-by: command-bot <>
This opens up the proposer to only optionally create blocks. Nodes may
only make blocks when there are transactions or the chain is scheduled.
---------
Co-authored-by: command-bot <>
This PR fixes a bug in the tally accrual of approvals/rejections for
Candidates and Defender. The issue happened because:
- The `match maybe_old` is reducing `weight` from the tally:
```
Some(Vote { approve: true, weight }) => tally.approvals.saturating_reduce(weight),
Some(Vote { approve: false, weight }) => tally.rejections.saturating_reduce(weight),
```
- But `match approval` is accruing only `1` to the tally:
```
true => tally.approvals.saturating_accrue(1),
false => tally.rejections.saturating_accrue(1),
```
This way, if a member is rank 1 his vote is going to have weight 1 when
accruing but weight 4 when reducing from the tally. For example, let's
say:
- There's a Candidate with 0 approvals and 12 rejections;
- A ranked Member votes against the Candidate;
- The tally changes to 0 approvals and 13 rejections (should be 16);
- The Member changes his vote to an approval;
- Now tally changes to 1 approvals and 9 rejections, removing the
accrued approvals from other Members;
- If the Member keeps changing his vote, it wipes the tally clean.
So this PR changes `match approval` to accrue `weight` instead of just
`1` and changes the tests:
- Fixes `challenges_work`. This test started failing after the fix. The
reason is that the test assumes that all Members have equal weights to
their votes, but Member 10 is ranked, so his vote should have weight 4
against the Defender. So instead of using Member 10, I added Member 50
of rank 0 to keep the same logic;
- Improves `votes_are_working`. Added some assertions to check if the
tally is correct even after a ranked Member changes his vote a couple
times;
- Fixes `waive_repay_works`. Unrelated to the bug, but this test was
yielding a false positive. The test is ranking up Member 20, but
asserting the rank of Member 10, which is already ranked up.
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
With the current code, when a user interlaces their region, the end
result will be three regions in the state:
- the non-interlaced region
- first part of the interlaced region
- second part of the interlaced region
The existing implementation retains the non-interlaced region in the
state, leading to a problematic scenario:
1. User 1 acquires a region from the market.
2. User 1 then interlaces this region.
3. Subsequently, User 1 transfers one part of the interlaced regions to
User 2.
Despite this transfer, User 1 retains the ability to assign the entire
original non-interlaced region, which is inconsistent with the fact that
they no longer own one of the interlaced parts.
This PR resolves the issue by removing the original region, ensuring
that only the two new interlaced regions remain in the state.
This changes `pallet-sudo` to also accept `Root` origin for
`ensure_sudo`. This can be useful for parachains who allow the relay
chain to have superuser rights to setup the sudo pallet for example.
Also fixes: https://github.com/paritytech/polkadot-sdk/issues/1417
- [x] CoreIndex -> AssignmentProvider mapping will be able to change any
time.
- [x] Implement
- [x] Provide Migrations
- [x] Add and fix tests
- [x] Implement bulk assigner logic
- [x] bulk assigner tests
- [x] Port over current assigner to use bulk designer (+ share on-demand
with bulk): top-level assigner has core ranges: legacy, bulk
- [x] Adjust migrations to reflect new assigner structure
- [x] Move migration code to Assignment code directly and make it
recursive (make it possible to skip releases) -> follow up ticket.
- [x] Test migrations
- [x] Add migration PR to runtimes repo -> follow up ticket.
- [x] Wire up with actual UMP messages
- [x] Write PR docs
---------
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com>
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
Co-authored-by: antonva <anton.asgeirsson@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Marcin S. <marcin@realemail.net>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: command-bot <>
Adds the `authorize_upgrade` -> `enact_authorized_upgrade` pattern to
`frame-system`. This will be useful for upgrading bridged chains that
are under the governance of Polkadot without passing entire runtime Wasm
blobs over a bridge.
Notes:
- Changed `enact_authorized_upgrade` to `apply_authorized_upgrade`.
Personal opinion, "apply" more accurately expresses what it's doing. Can
change back if outvoted.
- Remove `check_version` in favor of two extrinsics, so as to make
_checked_ the default.
- Left calls in `parachain-system` and marked as deprecated to prevent
breaking the API. They just call into the `frame-system` functions.
- Updated `frame-system` benchmarks to v2 syntax.
---------
Co-authored-by: command-bot <>
closes https://github.com/paritytech/polkadot-sdk/issues/1842
Decoupling Pallet from the Concept of Native Currency
Currently, the pallet is intrinsically linked with the concept of native
currency, requiring users to provide implementations of the
`fungible::*` and `fungibles::*` traits to interact with native and non
native assets. This incapsulates some non-related to the pallet
complexity and makes it less adaptable in contexts where the native
currency concept is absent.
With this PR, the dependence on `fungible::*` for liquidity-supplying
assets has been removed. Instead, the native and non-native currencies'
handling is now overseen by a single type that implements the
`fungibles::*` traits. To simplify this integration, types have been
introduced to facilitate the creation of a union between `fungible::*`
and `fungibles::*` implementations, producing a unified `fungibles::*`
type.
One of the reasons driving these changes is the ambition to create a
more user-friendly API for the `SwapCredit` implementation. Given that
it interacts with two distinct credit types from `fungible` and
`fungibles`, a unified type was introduced. Clients now manage potential
conversion failures for those credit types. In certain contexts, it's
vital to guarantee that operations are fail-safe, like in this impl -
[PR](https://github.com/paritytech/polkadot-sdk/pull/1845), place in
[code](https://github.com/paritytech/polkadot-sdk/blob/20b85a5fada8f55c98ba831964f5866ffeadf4da/cumulus/primitives/utility/src/lib.rs#L429).
Additional Updates:
- abstracted the pool ID and its account derivation logic via trait
bounds, along with common implementation offerings;
- removed `inc_providers` on a pool creation for the pool account;
- benchmarks:
-- swap complexity is N, not const;
-- removed `From<u128> + Into<u128>` bound from `T::Balance`;
-- removed swap/liquidity/.. amount constants, resolve them dynamically
based on pallet configuration;
-- migrated to v2 API;
- `OnUnbalanced` handler for the pool creation fee, replacing direct
transfers to a specified account ID;
- renamed `MultiAssetId` to `AssetKind` aligning with naming across
frame crates;
related PRs:
- (depends) https://github.com/paritytech/polkadot-sdk/pull/1677
- (caused) https://github.com/paritytech/polkadot-sdk/pull/2033
- (caused) https://github.com/paritytech/polkadot-sdk/pull/1876
---------
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Introduces a swap implementation that allows the exchange of a credit
(aka Negative Imbalance) of one asset for a credit of another asset.
This is particularly useful when a credit swap is required but may not
have sufficient value to meet the ED constraint, hence cannot be
deposited to temp account before. An example use case is when XCM fees
are paid using an asset held in the XCM executor registry and has to be
swapped for native currency.
Additional Updates:
- encapsulates the existing `Swap` trait impl within a transactional
context, since partial storage mutation is possible when an error
occurs;
- supplied `Currency` and `Assets` impls must be implemented over the
same `Balance` type, the `AssetBalance` generic type is dropped. This
helps to avoid numerous type conversion and overflow cases. If those
types are different it should be handled outside of the pallet;
- `Box` asset kind on a pallet level, unbox on a runtime level - here
[why](https://substrate.stackexchange.com/questions/10039/boxed-argument-of-a-dispatchable/10103#10103);
- `path` uses `Vec` now, instead of `BoundedVec` since it is never used
in PoV;
- removes the `Transfer` event due to it's redundancy with the events
emitted by `fungible/s` implementations;
- modifies the `SwapExecuted` event type;
related issue:
- https://github.com/paritytech/polkadot-sdk/issues/105
related PRs:
- (required for) https://github.com/paritytech/polkadot-sdk/pull/1845
- (caused) https://github.com/paritytech/polkadot-sdk/pull/1717
// DONE make the pallet work only with `fungibles` trait and make it
free from the concept of a `native` asset -
https://github.com/paritytech/polkadot-sdk/issues/1842
---------
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
closes https://github.com/paritytech-secops/srlabs_findings/issues/78.
Removes `BetterUnsignedThreshold` from pallet EPM. This will essentially
mean any solution submitted by the validator that is strictly better
than the current queued solution would be accepted.
The reason for having these thresholds is to limit number of solutions
submitted on-chain. However for unsigned submissions, the number of
solutions that could be submitted on average is limited even without
thresholding (calculation shown in the corresponding issue).
In the move from the old `Currency` traits to the new `fungible/s`
family of traits, we already had the `FungiblesAdapter` and
`NonFungiblesAdapter` for multiple fungible and non fungible assets
respectively. However, for handling only one fungible asset, we were
missing a `FungibleAdapter`, and so used the old `CurrencyAdapter`
instead. This PR aims to fill in that gap, and provide the new adapter
for more updated examples.
I marked the old `CurrencyAdapter` as deprecated as part of this PR, and
I'll change it to the new `FungibleAdapter` in a following PR.
The two stages are separated so as to not bloat this PR with some name
fixes in tests.
---------
Co-authored-by: command-bot <>
Closes#2326.
This PR both fixes a logic bug and replaces an incorrect name.
## Bug Fix: Respecting custom genesis builder
Prior to this PR the standard logic for creating a genesis block was
repeated inside of cumulus. This PR removes that duplicated logic, and
calls into the proper `BuildGenesisBlock` implementation.
One consequence is that if the genesis block has already been
initialized, it will not be re-created, but rather read from the
database like it is for other node invocations. So you need to watch out
for old unpurged data during the development process. Offchain tools may
need to be updated accordingly. I've already filed
https://github.com/paritytech/zombienet/issues/1519
## Rename: It doesn't export state. It exports head data.
The name export-genesis-state was always wrong, nad it's never too late
to right a wrong. I've changed the name of the struct to
`ExportGenesisHeadCommand`.
There is still the question of what to do with individual nodes' public
CLIs. I have updated the parachain template to a reasonable default that
preserves compatibility with tools that will expect
`export-genesis-state` to still work. And I've chosen not to modify the
public CLIs of any other nodes in the repo. I'll leave it up to their
individual owners/maintains to decide whether that is appropriate.
---------
Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Bastian Köcher <info@kchr.de>
Initial implementation for the plan discussed here: https://github.com/paritytech/polkadot-sdk/issues/701
Built on top of https://github.com/paritytech/polkadot-sdk/pull/1178
v0: https://github.com/paritytech/polkadot/pull/7554,
## Overall idea
When approval-voting checks a candidate and is ready to advertise the
approval, defer it in a per-relay chain block until we either have
MAX_APPROVAL_COALESCE_COUNT candidates to sign or a candidate has stayed
MAX_APPROVALS_COALESCE_TICKS in the queue, in both cases we sign what
candidates we have available.
This should allow us to reduce the number of approvals messages we have
to create/send/verify. The parameters are configurable, so we should
find some values that balance:
- Security of the network: Delaying broadcasting of an approval
shouldn't but the finality at risk and to make sure that never happens
we won't delay sending a vote if we are past 2/3 from the no-show time.
- Scalability of the network: MAX_APPROVAL_COALESCE_COUNT = 1 &
MAX_APPROVALS_COALESCE_TICKS =0, is what we have now and we know from
the measurements we did on versi, it bottlenecks
approval-distribution/approval-voting when increase significantly the
number of validators and parachains
- Block storage: In case of disputes we have to import this votes on
chain and that increase the necessary storage with
MAX_APPROVAL_COALESCE_COUNT * CandidateHash per vote. Given that
disputes are not the normal way of the network functioning and we will
limit MAX_APPROVAL_COALESCE_COUNT in the single digits numbers, this
should be good enough. Alternatively, we could try to create a better
way to store this on-chain through indirection, if that's needed.
## Other fixes:
- Fixed the fact that we were sending random assignments to
non-validators, that was wrong because those won't do anything with it
and they won't gossip it either because they do not have a grid topology
set, so we would waste the random assignments.
- Added metrics to be able to debug potential no-shows and
mis-processing of approvals/assignments.
## TODO:
- [x] Get feedback, that this is moving in the right direction. @ordian
@sandreim @eskimor @burdges, let me know what you think.
- [x] More and more testing.
- [x] Test in versi.
- [x] Make MAX_APPROVAL_COALESCE_COUNT &
MAX_APPROVAL_COALESCE_WAIT_MILLIS a parachain host configuration.
- [x] Make sure the backwards compatibility works correctly
- [x] Make sure this direction is compatible with other streams of work:
https://github.com/paritytech/polkadot-sdk/issues/635 &
https://github.com/paritytech/polkadot-sdk/issues/742
- [x] Final versi burn-in before merging
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Niklas Adolfsson