The authority-discovery mechanism has implemented a few exponential
timers for:
- publishing the authority records
- goes from 2 seconds (when freshly booted) to 1 hour if the node is
long-running
- set to 1 hour after successfully publishing the authority record
- discovering other authority records
- goes from 2 seconds (when freshly booted) to 10 minutes if the node is
long-running
This PR resets the exponential publishing and discovery interval to
defaults ensuring that long-running nodes:
- will retry publishing the authority records as aggressively as freshly
booted nodes
- Currently, if a long-running node fails to publish the DHT record when
the keys change (ie DhtEvent::ValuePutFailed), it will only retry after
1 hour
- will rediscover other authorities faster (since there is a chance that
other authority keys changed)
The subp2p-explorer has difficulties discovering the authorities when
the authority set changes in the first few hours. This might be entirely
due to the recursive nature of the DHT and the needed time to propagate
the records. However, there is a small chance that the authority
publishing failed and is only retried in 1h.
Let me know if this makes sense 🙏
cc @paritytech/networking
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Fixes#3826.
The docs on the `candidates` field of `BlockEntry` were incorrectly
stating that they are sorted by core index. The (incorrect) optimization
was introduced in #3747 based on this assumption. The actual ordering is
based on `CandidateIncluded` events ordering in the runtime. We revert
this optimization here.
- [x] verify the underlying issue
- [x] add a regression test
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
The PR provides API for obtaining:
- the weight required to execute an XCM message,
- a list of acceptable `AssetId`s for message execution payment,
- the cost of the weight in the specified acceptable `AssetId`.
It is meant to address an issue where one has to guess how much fee to
pay for execution. Also, at the moment, a client has to guess which
assets are acceptable for fee execution payment.
See the related issue
https://github.com/paritytech/polkadot-sdk/issues/690.
With this API, a client is supposed to query the list of the supported
asset IDs (in the XCM version format the client understands), weigh the
XCM program the client wants to execute and convert the weight into one
of the acceptable assets. Note that the client is supposed to know what
program will be executed on what chains. However, having a small
companion JS library for the pallet-xcm and xtokens should be enough to
determine what XCM programs will be executed and where (since these
pallets compose a known small set of programs).
```Rust
pub trait XcmPaymentApi<Call>
where
Call: Codec,
{
/// Returns a list of acceptable payment assets.
///
/// # Arguments
///
/// * `xcm_version`: Version.
fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec<VersionedAssetId>, Error>;
/// Returns a weight needed to execute a XCM.
///
/// # Arguments
///
/// * `message`: `VersionedXcm`.
fn query_xcm_weight(message: VersionedXcm<Call>) -> Result<Weight, Error>;
/// Converts a weight into a fee for the specified `AssetId`.
///
/// # Arguments
///
/// * `weight`: convertible `Weight`.
/// * `asset`: `VersionedAssetId`.
fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
/// Get delivery fees for sending a specific `message` to a `destination`.
/// These always come in a specific asset, defined by the chain.
///
/// # Arguments
/// * `message`: The message that'll be sent, necessary because most delivery fees are based on the
/// size of the message.
/// * `destination`: The destination to send the message to. Different destinations may use
/// different senders that charge different fees.
fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
}
```
An
[example](https://gist.github.com/PraetorP/4bc323ff85401abe253897ba990ec29d)
of a client side code.
---------
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Daniel Shiposha <mrshiposha@gmail.com>
In preparation for the merkleized metadata, we need to ensure that
constants are actually constant. If we want to test the unsigned phase
we could for example just disable signed voter. Or we add some extra
mechanism to the pallet to disable the signed phase from time to time.
---------
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
**Update:** Pushed additional changes based on the review comments.
**This pull request fixes various spelling mistakes in this
repository.**
Most of the changes are contained in the first **3** commits:
- `Fix spelling mistakes in comments and docs`
- `Fix spelling mistakes in test names`
- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`
Other source code spelling mistakes are separated into individual
commits for easier reviewing:
- `Fix the spelling of 'authority'`
- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`
- `Fix the spelling of 'prev_enqueud_messages'`
- `Fix the spelling of 'endpoint'`
- `Fix the spelling of 'children'`
- `Fix the spelling of 'PenpalSiblingSovereignAccount'`
- `Fix the spelling of 'PenpalSudoAccount'`
- `Fix the spelling of 'insufficient'`
- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`
- `Fix the spelling of 'subtracted'`
- `Fix the spelling of 'CandidatePendingAvailability'`
- `Fix the spelling of 'exclusive'`
- `Fix the spelling of 'until'`
- `Fix the spelling of 'discriminator'`
- `Fix the spelling of 'nonexistent'`
- `Fix the spelling of 'subsystem'`
- `Fix the spelling of 'indices'`
- `Fix the spelling of 'committed'`
- `Fix the spelling of 'topology'`
- `Fix the spelling of 'response'`
- `Fix the spelling of 'beneficiary'`
- `Fix the spelling of 'formatted'`
- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`
- `Fix the spelling of 'succeeded'`
- `Fix the spelling of 'reopened'`
- `Fix the spelling of 'proposer'`
- `Fix the spelling of 'InstantiationNonce'`
- `Fix the spelling of 'depositor'`
- `Fix the spelling of 'expiration'`
- `Fix the spelling of 'phantom'`
- `Fix the spelling of 'AggregatedKeyValue'`
- `Fix the spelling of 'randomness'`
- `Fix the spelling of 'defendant'`
- `Fix the spelling of 'AquaticMammal'`
- `Fix the spelling of 'transactions'`
- `Fix the spelling of 'PassingTracingSubscriber'`
- `Fix the spelling of 'TxSignaturePayload'`
- `Fix the spelling of 'versioning'`
- `Fix the spelling of 'descendant'`
- `Fix the spelling of 'overridden'`
- `Fix the spelling of 'network'`
Let me know if this structure is adequate.
**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.
~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~
~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~
~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~
**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.
~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~
Adds availability-write regression tests.
The results for the `availability-distribution` subsystem are volatile,
so I had to reduce the precision of the test.
Related to
https://github.com/paritytech/parity-bridges-common/issues/2538
This PR doesn't contain any functional changes.
The PR moves specific bridged chain definitions from
`bridges/primitives` to `bridges/chains` folder in order to facilitate
the migration of the `parity-bridges-repo` into `polkadot-sdk` as
discussed in https://hackmd.io/LprWjZ0bQXKpFeveYHIRXw?view
Apart from this it also includes some cosmetic changes to some
`Cargo.toml` files as a result of running `diener workspacify`.
Currently `transfer_assets` from pallet-xcm covers 4 main different
transfer types:
- `localReserve`
- `DestinationReserve`
- `Teleport`
- `RemoteReserve`
For the first three, the local execution and the remote message sending
are separated, and fees are deducted in pallet-xcm itself:
https://github.com/paritytech/polkadot-sdk/blob/3410dfb3929462da88be2da813f121d8b1cf46b3/polkadot/xcm/pallet-xcm/src/lib.rs#L1758.
For the 4th case `RemoteReserve`, pallet-xcm is still relying on the
xcm-executor itself to send the message (through the
`initiateReserveWithdraw` instruction). In this case, if delivery fees
need to be charged, it is not possible to do so because the
`jit_withdraw` mode has not being set.
This PR proposes to still use the `initiateReserveWithdraw` but
prepending a `setFeesMode { jit_withdraw: true }` to make sure delivery
fees can be paid.
A test-case is also added to present the aforementioned case
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Bridging fees are calculated using a static ETH/DOT exchange rate that
can deviate significantly from the real-world exchange rate. We
therefore need to add a safety margin to the fee so that users almost
aways cover the cost of relaying.
# FAQ
> Why introduce a `multiplier` parameter instead of configuring an
exchange rate which already has a safety factor applied?
When converting from ETH to DOT, we need to _divide_ the multiplier by
the exchange rate, and to convert from DOT to ETH we need to _multiply_
the multiplier by the exchange rate.
> Other input parameters to the fee calculation can also deviate from
real-world values. These include substrate weights, gas prices, and so
on. Why does the multiplier introduced here not adjust those?
A single scalar multiplier won't be able to accommodate the different
volatilities efficiently. For example, gas prices are much more volatile
than exchange rates, and substrate weights hardly ever change.
So the pricing config relating to weights and gas prices should already
have some appropriate safety margin pre-applied.
# Detailed Changes:
* Added `multiplier` field to `PricingParameters`
* Outbound-queue fee is multiplied by `multiplier`
* This `multiplier` is synced to the Ethereum side
* Improved Runtime API for calculating outbound-queue fees. This API
makes it much easier to for configure parts of the system in preparation
for launch.
* Improve and clarify code documentation
Upstreamed from https://github.com/Snowfork/polkadot-sdk/pull/127
---------
Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
## Bug Explanation
Adds a check that prevents finalized headers with a gap larger than the
sync committee period being imported, which could cause execution
headers in the gap being unprovable. The current version of the Ethereum
client checks that there is a header at least every sync committee, but
it doesn't check that the headers are within a sync period of each
other. For example:
Header 100 (sync committee period 1)
Header 9000 (sync committee period 2)
(8900 blocks apart)
These headers are in adjacent sync committees, but more than the sync
committee period (8192 blocks) apart.
The reason we need a header every 8192 slots at least, is the header is
used to prove messages within the last 8192 blocks. If we import header
9000, and we receive a message to be verified at header 200, the
`block_roots` field of header 9000 won't contain the header in order to
do the ancestry check.
## Environment
While running in Rococo, this edge case was discovered after the relayer
was offline for a few days. It is unlikely, but not impossible, to
happen again and so it should be backported to polkadot-sdk 1.7.0 (so
that
[polkadot-fellows/runtimes](https://github.com/polkadot-fellows/runtimes)
can be updated with the fix).
Our Ethereum client has been operational on Rococo for the past few
months, and this been the only major issue discovered so far.
### Unrelated Change
An unrelated nit: Removes a left over file that should have been deleted
when the `parachain` directory was removed.
---------
Co-authored-by: claravanstaden <Cats 4 life!>
Based on issue
[#2512](https://github.com/paritytech/polkadot-sdk/issues/2512), it
seems that some ecosystem teams are using these networks to set up their
staging environments and test certain use cases, some of them involving
sending XCMs from the relay with origins not allowed in the current
configuration.
This change reverts the configuration of `SendXcmOrigin`.
---------
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Small refactoring to reduce the algorithmic complexity of the initial
message distribution in approval voting after a sync from O(n_candidates
^ 2) to O(n_candidates).
We witnessed really poor performance on Rococo, where we ended up with
50 on-demand cores. This was due to the fact that for each core the full
queue was processed. With this change full queue processing will happen
way less often (most of the time complexity is O(1) or O(log(n))) and if
it happens then only for one core (in expectation).
Also spot price is now updated before each order to ensure economic back
pressure.
TODO:
- [x] Implement
- [x] Basic tests
- [x] Add more tests (see todos)
- [x] Run benchmark to confirm better performance, first results suggest
> 100x faster.
- [x] Write migrations
- [x] Bump scale-info version and remove patch in Cargo.toml
- [x] Write PR docs: on-demand performance improved, more on-demand
cores are now non problematic anymore. If need by also the max queue
size can be increased again. (Maybe not to 10k)
Optional: Performance can be improved even more, if we called
`pop_assignment_for_core()`, before calling `report_processed` (Avoid
needless affinity drops). The effect gets smaller the larger the claim
queue and I would only go for it, if it does not add complexity to the
scheduler.
---------
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: antonva <anton.asgeirsson@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com>
Co-authored-by: ordian <write@reusable.software>
_This PR is being continued from
https://github.com/paritytech/polkadot-sdk/pull/2206, which was closed
when the developer_hub was merged._
closes https://github.com/paritytech/polkadot-sdk-docs/issues/44
---
# Description
This PR adds a reference document to the `developer-hub` crate (see
https://github.com/paritytech/polkadot-sdk/pull/2102). This specific
reference document covers defensive programming practices common within
the context of developing a runtime with Substrate.
In particular, this covers the following areas:
- Default behavior of how Rust deals with numbers in general
- How to deal with floating point numbers in runtime / fixed point
arithmetic
- How to deal with Integer overflows
- General "safe math" / defensive programming practices for common
pallet development scenarios
- Defensive traits that exist within Substrate, i.e.,
`defensive_saturating_add `, `defensive_unwrap_or`
- More general defensive programming examples (keep it concise)
- Link to relevant examples where these practices are actually in
production / being used
- Unwrapping (or rather lack thereof) 101
todo
--
- [x] Apply feedback from previous PR
- [x] This may warrant a PR to append some of these docs to
`sp_arithmetic`
---------
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>
The PR adds two things:
1. Runtime API exposing the whole claim queue
2. Consumes the API in `collation-generation` to fetch the next
scheduled `ParaEntry` for an occupied core.
Related to https://github.com/paritytech/polkadot-sdk/issues/1797
Introduces `CryptoBytes` type defined as:
```rust
pub struct CryptoBytes<const N: usize, Tag = ()>(pub [u8; N], PhantomData<fn() -> Tag>);
```
The type implements a bunch of methods and traits which are typically
expected from a byte array newtype
(NOTE: some of the methods and trait implementations IMO are a bit
redundant, but I decided to maintain them all to not change too much
stuff in this PR)
It also introduces two (generic) typical consumers of `CryptoBytes`:
`PublicBytes` and `SignatureBytes`.
```rust
pub struct PublicTag;
pub PublicBytes<const N: usize, CryptoTag> = CryptoBytes<N, (PublicTag, CryptoTag)>;
pub struct SignatureTag;
pub SignatureBytes<const N: usize, CryptoTag> = CryptoBytes<N, (SignatureTag, CryptoTag)>;
```
Both of them use a tag to differentiate the two types at a higher level.
Downstream specializations will further specialize using a dedicated
crypto tag. For example in ECDSA:
```rust
pub struct EcdsaTag;
pub type Public = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
pub type Signature = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
```
Overall we have a cleaner and most importantly **consistent** code for
all the types involved
All these details are opaque to the end user which can use `Public` and
`Signature` for the cryptos as before
On top of #3302.
We want the validators to upgrade first before we add changes to the
collation side to send the new variants, which is why this part is
extracted into a separate PR.
The detection of when to send the parent head is based on the core
assignments at the relay parent of the candidate. We probably want to
make it more flexible in the future, but for now, it will work for a
simple use case when a para always has multiple cores assigned to it.
---------
Signed-off-by: Matteo Muraca <mmuraca247@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Matteo Muraca <56828990+muraca@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ignacio Rios <54085674+JuaniRios@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
This PR enables the `transaction_unstable_broadcast ` and
`transaction_unstable_stop` RPC API.
Since the API is unstable, we don't need to expose this in the release
notes.
After merging this, we could validate the API in subxt and stabilize it.
Spec PR that stabilizes the API:
https://github.com/paritytech/json-rpc-interface-spec/pull/139
cc @paritytech/subxt-team
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Part of #3326
cc @kianenigma @ggwpez @liamaharon
polkadot address: 12poSUQPtcF1HUPQGY3zZu2P8emuW9YnsPduA4XG3oCEfJVp
---------
Signed-off-by: Matteo Muraca <mmuraca247@gmail.com>
Changes:
- Port the `pov_mode` attribute from the V1 syntax to V2
- Update `pallet-whitelist` and `frame-benchmarking-pallet-pov`
Follow up: also allow this attribute on top-level benchmark modules.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Currently the xcm-executor returns an `Unimplemented` error if it
receives any HRMP-related instruction.
What I propose here, which is what we are currently doing in our forked
executor at polimec, is to introduce a trait implemented by the executor
which will handle those instructions.
This way, if parachains want to keep the default behavior, they just use
`()` and it will return unimplemented, but they can also implement their
own logic to establish HRMP channels with other chains in an automated
fashion, without requiring to go through governance.
Our implementation is mentioned in the [polkadot HRMP
docs](https://arc.net/l/quote/hduiivbu), and it was suggested to us to
submit a PR to add these changes to polkadot-sdk.
---------
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: command-bot <>
Alexandru Vasile
