pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-12 17:01:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Markdown linter (#1309)

Browse Source 
* Add markdown linting

- add linter default rules
- adapt rules to current code
- fix the code for linting to pass
- add CI check

fix #1243

* Fix markdown for Substrate
* Fix tooling install
* Fix workflow
* Add documentation
* Remove trailing spaces
* Update .github/.markdownlint.yaml

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix mangled markdown/lists
* Fix captalization issues on known words
This commit is contained in:
Chevdor
2023-09-04 11:02:32 +02:00
committed by GitHub
parent 830fde2a60
commit a30092ab42
271 changed files with 6289 additions and 4450 deletions
Download Patch File Download Diff File Expand all files Collapse all files
polkadot/roadmap/implementers-guide/src/runtime/README.md
+65 -24
View File
@@ -1,42 +1,83 @@
# Runtime Architecture
It's clear that we want to separate different aspects of the runtime logic into different modules. Modules define their own storage, routines, and entry-points. They also define initialization and finalization logic.
It's clear that we want to separate different aspects of the runtime logic into different modules. Modules define their
own storage, routines, and entry-points. They also define initialization and finalization logic.
Due to the (lack of) guarantees provided by a particular blockchain-runtime framework, there is no defined or dependable order in which modules' initialization or finalization logic will run. Supporting this blockchain-runtime framework is important enough to include that same uncertainty in our model of runtime modules in this guide. Furthermore, initialization logic of modules can trigger the entry-points or routines of other modules. This is one architectural pressure against dividing the runtime logic into multiple modules. However, in this case the benefits of splitting things up outweigh the costs, provided that we take certain precautions against initialization and entry-point races.
Due to the (lack of) guarantees provided by a particular blockchain-runtime framework, there is no defined or dependable
order in which modules' initialization or finalization logic will run. Supporting this blockchain-runtime framework is
important enough to include that same uncertainty in our model of runtime modules in this guide. Furthermore,
initialization logic of modules can trigger the entry-points or routines of other modules. This is one architectural
pressure against dividing the runtime logic into multiple modules. However, in this case the benefits of splitting
things up outweigh the costs, provided that we take certain precautions against initialization and entry-point races.
We also expect, although it's beyond the scope of this guide, that these runtime modules will exist alongside various other modules. This has two facets to consider. First, even if the modules that we describe here don't invoke each others' entry points or routines during initialization, we still have to protect against those other modules doing that. Second, some of those modules are expected to provide governance capabilities for the chain. Configuration exposed by parachain-host modules is mostly for the benefit of these governance modules, to allow the operators or community of the chain to tweak parameters.
We also expect, although it's beyond the scope of this guide, that these runtime modules will exist alongside various
other modules. This has two facets to consider. First, even if the modules that we describe here don't invoke each
others' entry points or routines during initialization, we still have to protect against those other modules doing that.
Second, some of those modules are expected to provide governance capabilities for the chain. Configuration exposed by
parachain-host modules is mostly for the benefit of these governance modules, to allow the operators or community of the
chain to tweak parameters.
The runtime's primary role is to manage scheduling and updating of parachains, as well as handling misbehavior reports and slashing. This guide doesn't focus on how parachains are registered, only that they are. Also, this runtime description assumes that validator sets are selected somehow, but doesn't assume any other details than a periodic _session change_ event. Session changes give information about the incoming validator set and the validator set of the following session.
The runtime's primary role is to manage scheduling and updating of parachains, as well as handling misbehavior reports
and slashing. This guide doesn't focus on how parachains are registered, only that they are. Also, this runtime
description assumes that validator sets are selected somehow, but doesn't assume any other details than a periodic
_session change_ event. Session changes give information about the incoming validator set and the validator set of the
following session.
The runtime also serves another role, which is to make data available to the Node-side logic via Runtime APIs. These Runtime APIs should be sufficient for the Node-side code to author blocks correctly.
The runtime also serves another role, which is to make data available to the Node-side logic via Runtime APIs. These
Runtime APIs should be sufficient for the Node-side code to author blocks correctly.
There is some functionality of the relay chain relating to parachains that we also consider beyond the scope of this document. In particular, all modules related to how parachains are registered aren't part of this guide, although we do provide routines that should be called by the registration process.
There is some functionality of the relay chain relating to parachains that we also consider beyond the scope of this
document. In particular, all modules related to how parachains are registered aren't part of this guide, although we do
provide routines that should be called by the registration process.
We will split the logic of the runtime up into these modules:
* Initializer: manages initialization order of the other modules.
* Shared: manages shared storage and configurations for other modules.
* Configuration: manages configuration and configuration updates in a non-racy manner.
* Paras: manages chain-head and validation code for parachains.
* Scheduler: manages parachain scheduling as well as validator assignments.
* Inclusion: handles the inclusion and availability of scheduled parachains.
* SessionInfo: manages various session keys of validators and other params stored per session.
* Disputes: handles dispute resolution for included, available parablocks.
* Slashing: handles slashing logic for concluded disputes.
* HRMP: handles horizontal messages between paras.
* UMP: handles upward messages from a para to the relay chain.
* DMP: handles downward messages from the relay chain to the para.
- Initializer: manages initialization order of the other modules.
- Shared: manages shared storage and configurations for other modules.
- Configuration: manages configuration and configuration updates in a non-racy manner.
- Paras: manages chain-head and validation code for parachains.
- Scheduler: manages parachain scheduling as well as validator assignments.
- Inclusion: handles the inclusion and availability of scheduled parachains.
- SessionInfo: manages various session keys of validators and other params stored per session.
- Disputes: handles dispute resolution for included, available parablocks.
- Slashing: handles slashing logic for concluded disputes.
- HRMP: handles horizontal messages between paras.
- UMP: handles upward messages from a para to the relay chain.
- DMP: handles downward messages from the relay chain to the para.
The [Initializer module](initializer.md) is special - it's responsible for handling the initialization logic of the other modules to ensure that the correct initialization order and related invariants are maintained. The other modules won't specify a on-initialize logic, but will instead expose a special semi-private routine that the initialization module will call. The other modules are relatively straightforward and perform the roles described above.
The [Initializer module](initializer.md) is special - it's responsible for handling the initialization logic of the
other modules to ensure that the correct initialization order and related invariants are maintained. The other modules
won't specify a on-initialize logic, but will instead expose a special semi-private routine that the initialization
module will call. The other modules are relatively straightforward and perform the roles described above.
The Parachain Host operates under a changing set of validators. Time is split up into periodic sessions, where each session brings a potentially new set of validators. Sessions are buffered by one, meaning that the validators of the upcoming session `n+1` are determined at the end of session `n-1`, right before session `n` starts. Parachain Host runtime modules need to react to changes in the validator set, as it will affect the runtime logic for processing candidate backing, availability bitfields, and misbehavior reports. The Parachain Host modules can't determine ahead-of-time exactly when session change notifications are going to happen within the block (note: this depends on module initialization order again - better to put session before parachains modules).
The Parachain Host operates under a changing set of validators. Time is split up into periodic sessions, where each
session brings a potentially new set of validators. Sessions are buffered by one, meaning that the validators of the
upcoming session `n+1` are determined at the end of session `n-1`, right before session `n` starts. Parachain Host
runtime modules need to react to changes in the validator set, as it will affect the runtime logic for processing
candidate backing, availability bitfields, and misbehavior reports. The Parachain Host modules can't determine
ahead-of-time exactly when session change notifications are going to happen within the block (note: this depends on
module initialization order again - better to put session before parachains modules).
The relay chain is intended to use BABE or SASSAFRAS, which both have the property that a session changing at a block is determined not by the number of the block but instead by the time the block is authored. In some sense, sessions change in-between blocks, not at blocks. This has the side effect that the session of a child block cannot be determined solely by the parent block's identifier. Being able to unilaterally determine the validator-set at a specific block based on its parent hash would make a lot of Node-side logic much simpler.
The relay chain is intended to use BABE or SASSAFRAS, which both have the property that a session changing at a block is
determined not by the number of the block but instead by the time the block is authored. In some sense, sessions change
in-between blocks, not at blocks. This has the side effect that the session of a child block cannot be determined solely
by the parent block's identifier. Being able to unilaterally determine the validator-set at a specific block based on
its parent hash would make a lot of Node-side logic much simpler.
In order to regain the property that the validator set of a block is predictable by its parent block, we delay session changes' application to Parachains by 1 block. This means that if there is a session change at block X, that session change will be stored and applied during initialization of direct descendants of X. This principal side effect of this change is that the Parachains runtime can disagree with session or consensus modules about which session it currently is. Misbehavior reporting routines in particular will be affected by this, although not severely. The parachains runtime might believe it is the last block of the session while the system is really in the first block of the next session. In such cases, a historical validator-set membership proof will need to accompany any misbehavior report, although they typically do not need to during current-session misbehavior reports.
In order to regain the property that the validator set of a block is predictable by its parent block, we delay session
changes' application to Parachains by 1 block. This means that if there is a session change at block X, that session
change will be stored and applied during initialization of direct descendants of X. This principal side effect of this
change is that the Parachains runtime can disagree with session or consensus modules about which session it currently
is. Misbehavior reporting routines in particular will be affected by this, although not severely. The parachains runtime
might believe it is the last block of the session while the system is really in the first block of the next session. In
such cases, a historical validator-set membership proof will need to accompany any misbehavior report, although they
typically do not need to during current-session misbehavior reports.
So the other role of the initializer module is to forward session change notifications to modules in the initialization order. Session change is also the point at which the [Configuration Module](configuration.md) updates the configuration. Most of the other modules will handle changes in the configuration during their session change operation, so the initializer should provide both the old and new configuration to all the other
modules alongside the session change notification. This means that a session change notification should consist of the following data:
So the other role of the initializer module is to forward session change notifications to modules in the initialization
order. Session change is also the point at which the [Configuration Module](configuration.md) updates the configuration.
Most of the other modules will handle changes in the configuration during their session change operation, so the
initializer should provide both the old and new configuration to all the other modules alongside the session change
notification. This means that a session change notification should consist of the following data:
```rust
struct SessionChangeNotification {
polkadot/roadmap/implementers-guide/src/runtime/configuration.md
+17 -9
View File
@@ -1,6 +1,11 @@
# Configuration Pallet
This module is responsible for managing all configuration of the parachain host in-flight. It provides a central point for configuration updates to prevent races between configuration changes and parachain-processing logic. Configuration can only change during the session change routine, and as this module handles the session change notification first it provides an invariant that the configuration does not change throughout the entire session. Both the [scheduler](scheduler.md) and [inclusion](inclusion.md) modules rely on this invariant to ensure proper behavior of the scheduler.
This module is responsible for managing all configuration of the parachain host in-flight. It provides a central point
for configuration updates to prevent races between configuration changes and parachain-processing logic. Configuration
can only change during the session change routine, and as this module handles the session change notification first it
provides an invariant that the configuration does not change throughout the entire session. Both the
[scheduler](scheduler.md) and [inclusion](inclusion.md) modules rely on this invariant to ensure proper behavior of the
scheduler.
The configuration that we will be tracking is the [`HostConfiguration`](../types/runtime.md#host-configuration) struct.
@@ -23,7 +28,8 @@ The session change routine works as follows:
- If there is no pending configurations, then return early.
- Take all pending configurations that are less than or equal to the current session index.
- Get the pending configuration with the highest session index and apply it to the current configuration. Discard the earlier ones if any.
- Get the pending configuration with the highest session index and apply it to the current configuration. Discard the
earlier ones if any.
## Routines
@@ -41,17 +47,17 @@ pub fn configuration() -> HostConfiguration {
Configuration::get()
}
/// Schedules updating the host configuration. The update is given by the `updater` closure. The
/// closure takes the current version of the configuration and returns the new version.
/// Returns an `Err` if the closure returns a broken configuration. However, there are a couple of
/// exceptions:
/// Schedules updating the host configuration. The update is given by the `updater` closure. The
/// closure takes the current version of the configuration and returns the new version.
/// Returns an `Err` if the closure returns a broken configuration. However, there are a couple of
/// exceptions:
///
/// - if the configuration that was passed in the closure is already broken, then it will pass the
/// - if the configuration that was passed in the closure is already broken, then it will pass the
/// update: you cannot break something that is already broken.
/// - If the `BypassConsistencyCheck` flag is set, then the checks will be skipped.
///
/// The changes made by this function will always be scheduled at session X, where X is the current session index + 2.
/// If there is already a pending update for X, then the closure will receive the already pending configuration for
/// If there is already a pending update for X, then the closure will receive the already pending configuration for
/// session X.
///
/// If there is already a pending update for the current session index + 1, then it won't be touched. Otherwise,
@@ -61,4 +67,6 @@ fn schedule_config_update(updater: impl FnOnce(&mut HostConfiguration<BlockNumbe
## Entry-points
The Configuration module exposes an entry point for each configuration member. These entry-points accept calls only from governance origins. These entry-points will use the `update_configuration` routine to update the specific configuration field.
The Configuration module exposes an entry point for each configuration member. These entry-points accept calls only from
governance origins. These entry-points will use the `update_configuration` routine to update the specific configuration
field.
polkadot/roadmap/implementers-guide/src/runtime/disputes.md
+65 -36
View File
@@ -1,29 +1,47 @@
# Disputes Pallet
After a backed candidate is made available, it is included and proceeds into an acceptance period during which validators are randomly selected to do (secondary) approval checks of the parablock. Any reports disputing the validity of the candidate will cause escalation, where even more validators are requested to check the block, and so on, until either the parablock is determined to be invalid or valid. Those on the wrong side of the dispute are slashed and, if the parablock is deemed invalid, the relay chain is rolled back to a point before that block was included.
After a backed candidate is made available, it is included and proceeds into an acceptance period during which
validators are randomly selected to do (secondary) approval checks of the parablock. Any reports disputing the validity
of the candidate will cause escalation, where even more validators are requested to check the block, and so on, until
either the parablock is determined to be invalid or valid. Those on the wrong side of the dispute are slashed and, if
the parablock is deemed invalid, the relay chain is rolled back to a point before that block was included.
However, this isn't the end of the story. We are working in a forkful blockchain environment, which carries three important considerations:
However, this isn't the end of the story. We are working in a forkful blockchain environment, which carries three
important considerations:
1. For security, validators that misbehave shouldn't only be slashed on one fork, but on all possible forks. Validators that misbehave shouldn't be able to create a new fork of the chain when caught and get away with their misbehavior.
1. For security, validators that misbehave shouldn't only be slashed on one fork, but on all possible forks. Validators
that misbehave shouldn't be able to create a new fork of the chain when caught and get away with their misbehavior.
1. It is possible (and likely) that the parablock being contested has not appeared on all forks.
1. If a block author believes that there is a disputed parablock on a specific fork that will resolve to a reversion of the fork, that block author has more incentive to build on a different fork which does not include that parablock.
1. If a block author believes that there is a disputed parablock on a specific fork that will resolve to a reversion of
the fork, that block author has more incentive to build on a different fork which does not include that parablock.
This means that in all likelihood, there is the possibility of disputes that are started on one fork of the relay chain, and as soon as the dispute resolution process starts to indicate that the parablock is indeed invalid, that fork of the relay chain will be abandoned and the dispute will never be fully resolved on that chain.
This means that in all likelihood, there is the possibility of disputes that are started on one fork of the relay chain,
and as soon as the dispute resolution process starts to indicate that the parablock is indeed invalid, that fork of the
relay chain will be abandoned and the dispute will never be fully resolved on that chain.
Even if this doesn't happen, there is the possibility that there are two disputes underway, and one resolves leading to a reversion of the chain before the other has concluded. In this case we want to both transplant the concluded dispute onto other forks of the chain as well as the unconcluded dispute.
Even if this doesn't happen, there is the possibility that there are two disputes underway, and one resolves leading to
a reversion of the chain before the other has concluded. In this case we want to both transplant the concluded dispute
onto other forks of the chain as well as the unconcluded dispute.
We account for these requirements by having the disputes module handle two kinds of disputes.
1. Local disputes: those contesting the validity of the current fork by disputing a parablock included within it.
1. Remote disputes: a dispute that has partially or fully resolved on another fork which is transplanted to the local fork for completion and eventual slashing.
1. Remote disputes: a dispute that has partially or fully resolved on another fork which is transplanted to the local
fork for completion and eventual slashing.
When a local dispute concludes negatively, the chain needs to be abandoned and reverted back to a block where the state does not contain the bad parablock. We expect that due to the [Approval Checking Protocol](../protocol-approval.md), the current executing block should not be finalized. So we do two things when a local dispute concludes negatively:
When a local dispute concludes negatively, the chain needs to be abandoned and reverted back to a block where the state
does not contain the bad parablock. We expect that due to the [Approval Checking Protocol](../protocol-approval.md), the
current executing block should not be finalized. So we do two things when a local dispute concludes negatively:
1. Freeze the state of parachains so nothing further is backed or included.
1. Issue a digest in the header of the block that signals to nodes that this branch of the chain is to be abandoned.
If, as is expected, the chain is unfinalized, the freeze will have no effect as no honest validator will attempt to build on the frozen chain. However, if the approval checking protocol has failed and the bad parablock is finalized, the freeze serves to put the chain into a governance-only mode.
If, as is expected, the chain is unfinalized, the freeze will have no effect as no honest validator will attempt to
build on the frozen chain. However, if the approval checking protocol has failed and the bad parablock is finalized, the
freeze serves to put the chain into a governance-only mode.
The storage of this module is designed around tracking [`DisputeState`s](../types/disputes.md#disputestate), updating them with votes, and tracking blocks included by this branch of the relay chain. It also contains a `Frozen` parameter designed to freeze the state of all parachains.
The storage of this module is designed around tracking [`DisputeState`s](../types/disputes.md#disputestate), updating
them with votes, and tracking blocks included by this branch of the relay chain. It also contains a `Frozen` parameter
designed to freeze the state of all parachains.
## Storage
@@ -44,15 +62,18 @@ Included: double_map (SessionIndex, CandidateHash) -> Option<BlockNumber>,
Frozen: Option<BlockNumber>,
```
> `byzantine_threshold` refers to the maximum number `f` of validators which may be byzantine. The total number of validators is `n = 3f + e` where `e in { 1, 2, 3 }`.
> `byzantine_threshold` refers to the maximum number `f` of validators which may be byzantine. The total number of
> validators is `n = 3f + e` where `e in { 1, 2, 3 }`.
## Session Change
1. If the current session is not greater than `config.dispute_period + 1`, nothing to do here.
1. Set `pruning_target = current_session - config.dispute_period - 1`. We add the extra `1` because we want to keep things for `config.dispute_period` _full_ sessions.
The stuff at the end of the most recent session has been around for a little over 0 sessions, not a little over 1.
1. Set `pruning_target = current_session - config.dispute_period - 1`. We add the extra `1` because we want to keep
things for `config.dispute_period` _full_ sessions. The stuff at the end of the most recent session has been around
for a little over 0 sessions, not a little over 1.
1. If `LastPrunedSession` is `None`, then set `LastPrunedSession` to `Some(pruning_target)` and return.
2. Otherwise, clear out all disputes and included candidates entries in the range `last_pruned..=pruning_target` and set `LastPrunedSession` to `Some(pruning_target)`.
1. Otherwise, clear out all disputes and included candidates entries in the range `last_pruned..=pruning_target` and set
`LastPrunedSession` to `Some(pruning_target)`.
## Block Initialization
@@ -61,11 +82,10 @@ This is currently a `no op`.
## Routines
* `filter_multi_dispute_data(MultiDisputeStatementSet) -> MultiDisputeStatementSet`:
1. Takes a `MultiDisputeStatementSet` and filters it down to a `MultiDisputeStatementSet`
that satisfies all the criteria of `provide_multi_dispute_data`. That is, eliminating
ancient votes, duplicates and unconfirmed disputes.
This can be used by block authors to create the final submission in a block which is
guaranteed to pass the `provide_multi_dispute_data` checks.
1. Takes a `MultiDisputeStatementSet` and filters it down to a `MultiDisputeStatementSet` that satisfies all the
criteria of `provide_multi_dispute_data`. That is, eliminating ancient votes, duplicates and unconfirmed disputes.
This can be used by block authors to create the final submission in a block which is guaranteed to pass the
`provide_multi_dispute_data` checks.
* `provide_multi_dispute_data(MultiDisputeStatementSet) -> Vec<(SessionIndex, Hash)>`:
1. Pass on each dispute statement set to `provide_dispute_data`, propagating failure.
@@ -76,46 +96,55 @@ This is currently a `no op`.
1. `SessionInfo` is used to check statement signatures and this function should fail if any signatures are invalid.
1. If there is no dispute under `Disputes`, create a new `DisputeState` with blank bitfields.
1. If `concluded_at` is `Some`, and is `concluded_at + config.post_conclusion_acceptance_period < now`, return false.
2. Import all statements into the dispute. This should fail if any statements are duplicate or if the corresponding bit for the corresponding validator is set in the dispute already.
3. If `concluded_at` is `None`, reward all statements.
4. If `concluded_at` is `Some`, reward all statements slightly less.
5. If either side now has supermajority and did not previously, slash the other side. This may be both sides, and we support this possibility in code, but note that this requires validators to participate on both sides which has negative expected value. Set `concluded_at` to `Some(now)` if it was `None`.
6. If just concluded against the candidate and the `Included` map contains `(session, candidate)`: invoke `revert_and_freeze` with the stored block number.
7. Return true if just initiated, false otherwise.
1. Import all statements into the dispute. This should fail if any statements are duplicate or if the corresponding
bit for the corresponding validator is set in the dispute already.
1. If `concluded_at` is `None`, reward all statements.
1. If `concluded_at` is `Some`, reward all statements slightly less.
1. If either side now has supermajority and did not previously, slash the other side. This may be both sides, and we
support this possibility in code, but note that this requires validators to participate on both sides which has
negative expected value. Set `concluded_at` to `Some(now)` if it was `None`.
1. If just concluded against the candidate and the `Included` map contains `(session, candidate)`: invoke
`revert_and_freeze` with the stored block number.
1. Return true if just initiated, false otherwise.
* `disputes() -> Vec<(SessionIndex, CandidateHash, DisputeState)>`: Get a list of all disputes and info about dispute state.
* `disputes() -> Vec<(SessionIndex, CandidateHash, DisputeState)>`: Get a list of all disputes and info about dispute
state.
1. Iterate over all disputes in `Disputes` and collect into a vector.
* `note_included(SessionIndex, CandidateHash, included_in: BlockNumber)`:
1. Add `(SessionIndex, CandidateHash)` to the `Included` map with `included_in - 1` as the value.
1. If there is a dispute under `(SessionIndex, CandidateHash)` that has concluded against the candidate, invoke `revert_and_freeze` with the stored block number.
1. If there is a dispute under `(SessionIndex, CandidateHash)` that has concluded against the candidate, invoke
`revert_and_freeze` with the stored block number.
* `concluded_invalid(SessionIndex, CandidateHash) -> bool`: Returns whether a candidate has already concluded a dispute in the negative.
* `concluded_invalid(SessionIndex, CandidateHash) -> bool`: Returns whether a candidate has already concluded a dispute
in the negative.
* `is_frozen()`: Load the value of `Frozen` from storage. Return true if `Some` and false if `None`.
* `last_valid_block()`: Load the value of `Frozen` from storage and return. None indicates that all blocks in the chain are potentially valid.
* `last_valid_block()`: Load the value of `Frozen` from storage and return. None indicates that all blocks in the chain
are potentially valid.
* `revert_and_freeze(BlockNumber)`:
1. If `is_frozen()` return.
1. Set `Frozen` to `Some(BlockNumber)` to indicate a rollback to the block number.
1. Issue a `Revert(BlockNumber + 1)` log to indicate a rollback of the block's child in the header chain, which is the same as a rollback to the block number.
1. Issue a `Revert(BlockNumber + 1)` log to indicate a rollback of the block's child in the header chain, which is the
same as a rollback to the block number.
# Disputes filtering
All disputes delivered to the runtime by the client are filtered before the actual import. In this context actual import
means persisted in the runtime storage. The filtering has got two purposes:
- Limit the amount of data saved onchain.
- Prevent persisting malicious dispute data onchain.
* Limit the amount of data saved onchain.
* Prevent persisting malicious dispute data onchain.
*Implementation note*: Filtering is performed in function `filter_dispute_data` from `Disputes` pallet.
The filtering is performed on the whole statement set which is about to be imported onchain. The following filters are
applied:
1. Remove ancient disputes - if a dispute is concluded before the block number indicated in `OLDEST_ACCEPTED` parameter
it is removed from the set. `OLDEST_ACCEPTED` is a runtime configuration option.
*Implementation note*: `dispute_post_conclusion_acceptance_period` from
`HostConfiguration` is used in the current Polkadot/Kusama implementation.
it is removed from the set. `OLDEST_ACCEPTED` is a runtime configuration option. *Implementation note*:
`dispute_post_conclusion_acceptance_period` from `HostConfiguration` is used in the current Polkadot/Kusama
implementation.
2. Remove votes from unknown validators. If there is a vote from a validator which wasn't an authority in the session
where the dispute was raised - they are removed. Please note that this step removes only single votes instead of
removing the whole dispute.
@@ -138,4 +167,4 @@ inconclusive disputes are not slashed. Thanks to the applied filtering (describe
confident that there are no spam disputes in the runtime. So if a validator is not voting it is due to another reason
(e.g. being under DoS attack). There is no reason to punish such validators with a slash.
*Implementation note*: Slashing is performed in `process_checked_dispute_data` from `Disputes` pallet.
*Implementation note*: Slashing is performed in `process_checked_dispute_data` from `Disputes` pallet.
polkadot/roadmap/implementers-guide/src/runtime/dmp.md
+7 -6
View File
@@ -28,7 +28,8 @@ No initialization routine runs for this module.
Candidate Acceptance Function:
* `check_processed_downward_messages(P: ParaId, relay_parent_number: BlockNumber, processed_downward_messages: u32)`:
1. Checks that `processed_downward_messages` is at least 1 if `DownwardMessageQueues` for `P` is not empty at the given `relay_parent_number`.
1. Checks that `processed_downward_messages` is at least 1 if `DownwardMessageQueues` for `P` is not empty at the
given `relay_parent_number`.
1. Checks that `DownwardMessageQueues` for `P` is at least `processed_downward_messages` long.
Candidate Enactment:
@@ -38,11 +39,11 @@ Candidate Enactment:
Utility routines.
`queue_downward_message(P: ParaId, M: DownwardMessage)`:
1. Check if the size of `M` exceeds the `config.max_downward_message_size`. If so, return an error.
1. Wrap `M` into `InboundDownwardMessage` using the current block number for `sent_at`.
1. Obtain a new MQC link for the resulting `InboundDownwardMessage` and replace `DownwardMessageQueueHeads` for `P` with the resulting hash.
1. Add the resulting `InboundDownwardMessage` into `DownwardMessageQueues` for `P`.
`queue_downward_message(P: ParaId, M: DownwardMessage)`: 1. Check if the size of `M` exceeds the
`config.max_downward_message_size`. If so, return an error. 1. Wrap `M` into `InboundDownwardMessage` using the
current block number for `sent_at`. 1. Obtain a new MQC link for the resulting `InboundDownwardMessage` and replace
`DownwardMessageQueueHeads` for `P` with the resulting hash. 1. Add the resulting `InboundDownwardMessage` into
`DownwardMessageQueues` for `P`.
## Session Change
polkadot/roadmap/implementers-guide/src/runtime/hrmp.md
+52 -43
View File
@@ -1,6 +1,7 @@
# HRMP Pallet
A module responsible for Horizontally Relay-routed Message Passing (HRMP). See [Messaging Overview](../messaging.md) for more details.
A module responsible for Horizontally Relay-routed Message Passing (HRMP). See [Messaging Overview](../messaging.md) for
more details.
## Storage
@@ -132,7 +133,8 @@ Candidate Acceptance Function:
1. or in `HrmpChannelDigests` for `P` an entry with the block number should exist
* `check_outbound_hrmp(sender: ParaId, Vec<OutboundHrmpMessage>)`:
1. Checks that there are at most `config.hrmp_max_message_num_per_candidate` messages.
1. Checks that horizontal messages are sorted by ascending recipient ParaId and there is no two horizontal messages have the same recipient.
1. Checks that horizontal messages are sorted by ascending recipient ParaId and there is no two horizontal messages
have the same recipient.
1. For each horizontal message `M` with the channel `C` identified by `(sender, M.recipient)` check:
1. exists
1. `M`'s payload size doesn't exceed a preconfigured limit `C.max_message_size`
@@ -143,42 +145,48 @@ Candidate Enactment:
* `queue_outbound_hrmp(sender: ParaId, Vec<OutboundHrmpMessage>)`:
1. For each horizontal message `HM` with the channel `C` identified by `(sender, HM.recipient)`:
1. Append `HM` into `HrmpChannelContents` that corresponds to `C` with `sent_at` equals to the current block number.
1. Locate or create an entry in `HrmpChannelDigests` for `HM.recipient` and append `sender` into the entry's list.
1. Append `HM` into `HrmpChannelContents` that corresponds to `C` with `sent_at` equals to the current block
number.
1. Locate or create an entry in `HrmpChannelDigests` for `HM.recipient` and append `sender` into the entry's
list.
1. Increment `C.msg_count`
1. Increment `C.total_size` by `HM`'s payload size
1. Append a new link to the MQC and save the new head in `C.mqc_head`. Note that the current block number as of enactment is used for the link.
1. Append a new link to the MQC and save the new head in `C.mqc_head`. Note that the current block number as of
enactment is used for the link.
* `prune_hrmp(recipient, new_hrmp_watermark)`:
1. From `HrmpChannelDigests` for `recipient` remove all entries up to an entry with block number equal to `new_hrmp_watermark`.
1. From the removed digests construct a set of paras that sent new messages within the interval between the old and new watermarks.
1. For each channel `C` identified by `(sender, recipient)` for each `sender` coming from the set, prune messages up to the `new_hrmp_watermark`.
1. From `HrmpChannelDigests` for `recipient` remove all entries up to an entry with block number equal to
`new_hrmp_watermark`.
1. From the removed digests construct a set of paras that sent new messages within the interval between the old and
new watermarks.
1. For each channel `C` identified by `(sender, recipient)` for each `sender` coming from the set, prune messages up
to the `new_hrmp_watermark`.
1. For each pruned message `M` from channel `C`:
1. Decrement `C.msg_count`
1. Decrement `C.total_size` by `M`'s payload size.
1. Set `HrmpWatermarks` for `P` to be equal to `new_hrmp_watermark`
> NOTE: That collecting digests can be inefficient and the time it takes grows very fast. Thanks to the aggressive
> parameterization this shouldn't be a big of a deal.
> If that becomes a problem consider introducing an extra dictionary which says at what block the given sender
> sent a message to the recipient.
> parameterization this shouldn't be a big of a deal. If that becomes a problem consider introducing an extra
> dictionary which says at what block the given sender sent a message to the recipient.
## Entry-points
The following entry-points are meant to be used for HRMP channel management.
Those entry-points are meant to be called from a parachain. `origin` is defined as the `ParaId` of
the parachain executed the message.
Those entry-points are meant to be called from a parachain. `origin` is defined as the `ParaId` of the parachain
executed the message.
* `hrmp_init_open_channel(recipient, proposed_max_capacity, proposed_max_message_size)`:
1. Check that the `origin` is not `recipient`.
1. Check that `proposed_max_capacity` is less or equal to `config.hrmp_channel_max_capacity` and greater than zero.
1. Check that `proposed_max_message_size` is less or equal to `config.hrmp_channel_max_message_size` and greater than zero.
1. Check that `proposed_max_message_size` is less or equal to `config.hrmp_channel_max_message_size` and greater
than zero.
1. Check that `recipient` is a valid para.
1. Check that there is no existing channel for `(origin, recipient)` in `HrmpChannels`.
1. Check that there is no existing open channel request (`origin`, `recipient`) in `HrmpOpenChannelRequests`.
1. Check that the sum of the number of already opened HRMP channels by the `origin` (the size
of the set found `HrmpEgressChannelsIndex` for `origin`) and the number of open requests by the
`origin` (the value from `HrmpOpenChannelRequestCount` for `origin`) doesn't exceed the limit of
channels (`config.hrmp_max_parachain_outbound_channels` or `config.hrmp_max_parathread_outbound_channels`) minus 1.
1. Check that the sum of the number of already opened HRMP channels by the `origin` (the size of the set found
`HrmpEgressChannelsIndex` for `origin`) and the number of open requests by the `origin` (the value from
`HrmpOpenChannelRequestCount` for `origin`) doesn't exceed the limit of channels
(`config.hrmp_max_parachain_outbound_channels` or `config.hrmp_max_parathread_outbound_channels`) minus 1.
1. Check that `origin`'s balance is more or equal to `config.hrmp_sender_deposit`
1. Reserve the deposit for the `origin` according to `config.hrmp_sender_deposit`
1. Increase `HrmpOpenChannelRequestCount` by 1 for `origin`.
@@ -189,27 +197,26 @@ the parachain executed the message.
1. Set `max_message_size` to `proposed_max_message_size`
1. Set `max_total_size` to `config.hrmp_channel_max_total_size`
1. Send a downward message to `recipient` notifying about an inbound HRMP channel request.
- The DM is sent using `queue_downward_message`.
- The DM is represented by the `HrmpNewChannelOpenRequest` XCM message.
- `sender` is set to `origin`,
- `max_message_size` is set to `proposed_max_message_size`,
- `max_capacity` is set to `proposed_max_capacity`.
* The DM is sent using `queue_downward_message`.
* The DM is represented by the `HrmpNewChannelOpenRequest` XCM message.
* `sender` is set to `origin`,
* `max_message_size` is set to `proposed_max_message_size`,
* `max_capacity` is set to `proposed_max_capacity`.
* `hrmp_accept_open_channel(sender)`:
1. Check that there is an existing request between (`sender`, `origin`) in `HrmpOpenChannelRequests`
1. Check that it is not confirmed.
1. Check that the sum of the number of inbound HRMP channels opened to `origin` (the size of the set
found in `HrmpIngressChannelsIndex` for `origin`) and the number of accepted open requests by the `origin`
(the value from `HrmpAcceptedChannelRequestCount` for `origin`) doesn't exceed the limit of channels
(`config.hrmp_max_parachain_inbound_channels` or `config.hrmp_max_parathread_inbound_channels`)
minus 1.
1. Check that the sum of the number of inbound HRMP channels opened to `origin` (the size of the set found in
`HrmpIngressChannelsIndex` for `origin`) and the number of accepted open requests by the `origin` (the value from
`HrmpAcceptedChannelRequestCount` for `origin`) doesn't exceed the limit of channels
(`config.hrmp_max_parachain_inbound_channels` or `config.hrmp_max_parathread_inbound_channels`) minus 1.
1. Check that `origin`'s balance is more or equal to `config.hrmp_recipient_deposit`.
1. Reserve the deposit for the `origin` according to `config.hrmp_recipient_deposit`
1. For the request in `HrmpOpenChannelRequests` identified by `(sender, P)`, set `confirmed` flag to `true`.
1. Increase `HrmpAcceptedChannelRequestCount` by 1 for `origin`.
1. Send a downward message to `sender` notifying that the channel request was accepted.
- The DM is sent using `queue_downward_message`.
- The DM is represented by the `HrmpChannelAccepted` XCM message.
- `recipient` is set to `origin`.
* The DM is sent using `queue_downward_message`.
* The DM is represented by the `HrmpChannelAccepted` XCM message.
* `recipient` is set to `origin`.
* `hrmp_cancel_open_request(ch)`:
1. Check that `origin` is either `ch.sender` or `ch.recipient`
1. Check that the open channel request `ch` exists.
@@ -221,15 +228,15 @@ the parachain executed the message.
1. Check that `origin` is either `ch.sender` or `ch.recipient`
1. Check that `HrmpChannels` for `ch` exists.
1. Check that `ch` is not in the `HrmpCloseChannelRequests` set.
1. If not already there, insert a new entry `Some(())` to `HrmpCloseChannelRequests` for `ch`
and append `ch` to `HrmpCloseChannelRequestsList`.
1. If not already there, insert a new entry `Some(())` to `HrmpCloseChannelRequests` for `ch` and append `ch` to
`HrmpCloseChannelRequestsList`.
1. Send a downward message to the opposite party notifying about the channel closing.
- The DM is sent using `queue_downward_message`.
- The DM is represented by the `HrmpChannelClosing` XCM message with:
- `initator` is set to `origin`,
- `sender` is set to `ch.sender`,
- `recipient` is set to `ch.recipient`.
- The opposite party is `ch.sender` if `origin` is `ch.recipient` and `ch.recipient` if `origin` is `ch.sender`.
* The DM is sent using `queue_downward_message`.
* The DM is represented by the `HrmpChannelClosing` XCM message with:
* `initator` is set to `origin`,
* `sender` is set to `ch.sender`,
* `recipient` is set to `ch.recipient`.
* The opposite party is `ch.sender` if `origin` is `ch.recipient` and `ch.recipient` if `origin` is `ch.sender`.
## Session Change
@@ -241,13 +248,15 @@ the parachain executed the message.
1. Remove `HrmpOpenChannelRequests` and `HrmpOpenChannelRequestsList` for `(P, _)` and `(_, P)`.
1. For each removed channel request `C`:
1. Unreserve the sender's deposit if the sender is not present in `outgoing_paras`
1. Unreserve the recipient's deposit if `C` is confirmed and the recipient is not present in `outgoing_paras`
1. For each channel designator `D` in `HrmpOpenChannelRequestsList` we query the request `R` from `HrmpOpenChannelRequests`:
1. Unreserve the recipient's deposit if `C` is confirmed and the recipient is not present in
`outgoing_paras`
1. For each channel designator `D` in `HrmpOpenChannelRequestsList` we query the request `R` from
`HrmpOpenChannelRequests`:
1. if `R.confirmed = true`,
1. if both `D.sender` and `D.recipient` are not offboarded.
1. create a new channel `C` between `(D.sender, D.recipient)`.
1. Initialize the `C.sender_deposit` with `R.sender_deposit` and `C.recipient_deposit`
with the value found in the configuration `config.hrmp_recipient_deposit`.
1. Initialize the `C.sender_deposit` with `R.sender_deposit` and `C.recipient_deposit` with the value
found in the configuration `config.hrmp_recipient_deposit`.
1. Insert `sender` into the set `HrmpIngressChannelsIndex` for the `recipient`.
1. Insert `recipient` into the set `HrmpEgressChannelsIndex` for the `sender`.
1. decrement `HrmpOpenChannelRequestCount` for `D.sender` by 1.
polkadot/roadmap/implementers-guide/src/runtime/inclusion.md
+73 -49
View File
@@ -1,6 +1,7 @@
# Inclusion Pallet
The inclusion module is responsible for inclusion and availability of scheduled parachains. It also manages the UMP dispatch queue of each parachain.
The inclusion module is responsible for inclusion and availability of scheduled parachains. It also manages the UMP
dispatch queue of each parachain.
## Storage
@@ -37,11 +38,9 @@ PendingAvailabilityCommitments: map ParaId => CandidateCommitments;
## Config Dependencies
* `MessageQueue`:
The message queue provides general queueing and processing functionality. Currently it
replaces the old `UMP` dispatch queue. Other use-cases can be implemented as well by
adding new variants to `AggregateMessageOrigin`. Normally it should be set to an instance
of the `MessageQueue` pallet.
* `MessageQueue`: The message queue provides general queueing and processing functionality. Currently it replaces the
old `UMP` dispatch queue. Other use-cases can be implemented as well by adding new variants to
`AggregateMessageOrigin`. Normally it should be set to an instance of the `MessageQueue` pallet.
## Session Change
@@ -49,11 +48,13 @@ PendingAvailabilityCommitments: map ParaId => CandidateCommitments;
1. Clear out all validator bitfields.
Optional:
1. The UMP queue of all outgoing paras can be "swept". This would prevent the dispatch queue from automatically being serviced. It is a consideration for the chain and specific behaviour is not defined.
1. The UMP queue of all outgoing paras can be "swept". This would prevent the dispatch queue from automatically being
serviced. It is a consideration for the chain and specific behaviour is not defined.
## Initialization
No initialization routine runs for this module. However, the initialization of the `MessageQueue` pallet will attempt to process any pending UMP messages.
No initialization routine runs for this module. However, the initialization of the `MessageQueue` pallet will attempt to
process any pending UMP messages.
## Routines
@@ -63,20 +64,19 @@ All failed checks should lead to an unrecoverable error making the block invalid
* `process_bitfields(expected_bits, Bitfields, core_lookup: Fn(CoreIndex) -> Option<ParaId>)`:
1. Call `sanitize_bitfields<true>` and use the sanitized `signed_bitfields` from now on.
1. Call `sanitize_backed_candidates<true>` and use the sanitized `backed_candidates` from now on.
1. Apply each bit of bitfield to the corresponding pending candidate, looking up on-demand parachain cores using the `core_lookup`. Disregard bitfields that have a `1` bit for any free cores.
1. For each applied bit of each availability-bitfield, set the bit for the validator in the `CandidatePendingAvailability`'s `availability_votes` bitfield. Track all candidates that now have >2/3 of bits set in their `availability_votes`. These candidates are now available and can be enacted.
1. Apply each bit of bitfield to the corresponding pending candidate, looking up on-demand parachain cores using the
`core_lookup`. Disregard bitfields that have a `1` bit for any free cores.
1. For each applied bit of each availability-bitfield, set the bit for the validator in the
`CandidatePendingAvailability`'s `availability_votes` bitfield. Track all candidates that now have >2/3 of bits set
in their `availability_votes`. These candidates are now available and can be enacted.
1. For all now-available candidates, invoke the `enact_candidate` routine with the candidate and relay-parent number.
1. Return a list of `(CoreIndex, CandidateHash)` from freed cores consisting of the cores where candidates have become available.
* `sanitize_bitfields<T: crate::inclusion::Config>(
unchecked_bitfields: UncheckedSignedAvailabilityBitfields,
disputed_bitfield: DisputedBitfield,
expected_bits: usize,
parent_hash: T::Hash,
session_index: SessionIndex,
validators: &[ValidatorId],
full_check: FullCheck,
)`:
1. check that `disputed_bitfield` has the same number of bits as the `expected_bits`, iff not return early with an empty vec.
1. Return a list of `(CoreIndex, CandidateHash)` from freed cores consisting of the cores where candidates have become
available.
* `sanitize_bitfields<T: crate::inclusion::Config>( unchecked_bitfields: UncheckedSignedAvailabilityBitfields,
disputed_bitfield: DisputedBitfield, expected_bits: usize, parent_hash: T::Hash, session_index: SessionIndex,
validators: &[ValidatorId], full_check: FullCheck, )`:
1. check that `disputed_bitfield` has the same number of bits as the `expected_bits`, iff not return early with an
empty vec.
1. each of the below checks is for each bitfield. If a check does not pass the bitfield will be skipped.
1. check that there are no bits set that reference a disputed candidate.
1. check that the number of bits is equal to `expected_bits`.
@@ -84,41 +84,58 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. check that the validator bit index is not out of bounds.
1. check the validators signature, iff `full_check=FullCheck::Yes`.
* `sanitize_backed_candidates<T: crate::inclusion::Config, F: FnMut(usize, &BackedCandidate<T::Hash>) -> bool>(
mut backed_candidates: Vec<BackedCandidate<T::Hash>>,
candidate_has_concluded_invalid_dispute: F,
scheduled: &[CoreAssignment],
) `
* `sanitize_backed_candidates<T: crate::inclusion::Config, F: FnMut(usize, &BackedCandidate<T::Hash>) -> bool>( mut
backed_candidates: Vec<BackedCandidate<T::Hash>>, candidate_has_concluded_invalid_dispute: F, scheduled:
&[CoreAssignment], )`
1. filter out any backed candidates that have concluded invalid.
1. filters backed candidates whom's paraid was scheduled by means of the provided `scheduled` parameter.
1. sorts remaining candidates with respect to the core index assigned to them.
* `process_candidates(allowed_relay_parents, BackedCandidates, scheduled: Vec<CoreAssignment>, group_validators: Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>)`:
* `process_candidates(allowed_relay_parents, BackedCandidates, scheduled: Vec<CoreAssignment>, group_validators:
Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>)`:
> For details on `AllowedRelayParentsTracker` see documentation for [Shared](./shared.md) module.
1. check that each candidate corresponds to a scheduled core and that they are ordered in the same order the cores appear in assignments in `scheduled`.
1. check that each candidate corresponds to a scheduled core and that they are ordered in the same order the cores
appear in assignments in `scheduled`.
1. check that `scheduled` is sorted ascending by `CoreIndex`, without duplicates.
1. check that the relay-parent from each candidate receipt is one of the allowed relay-parents.
1. check that there is no candidate pending availability for any scheduled `ParaId`.
1. check that each candidate's `validation_data_hash` corresponds to a `PersistedValidationData` computed from the state of the context block.
1. check that each candidate's `validation_data_hash` corresponds to a `PersistedValidationData` computed from the
state of the context block.
1. If the core assignment includes a specific collator, ensure the backed candidate is issued by that collator.
1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_cooldown` of `Paras::last_code_upgrade(para_id, true)`, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID.
1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_cooldown`
of `Paras::last_code_upgrade(para_id, true)`, if any, comparing against the value of `Paras::FutureCodeUpgrades`
for the given para ID.
1. Check the collator's signature on the candidate data.
1. check the backing of the candidate using the signatures and the bitfields, comparing against the validators assigned to the groups, fetched with the `group_validators` lookup, while group indices are computed by `Scheduler` according to group rotation info.
1. call `check_upward_messages(config, para, commitments.upward_messages)` to check that the upward messages are valid.
1. call `Dmp::check_processed_downward_messages(para, commitments.processed_downward_messages)` to check that the DMQ is properly drained.
1. call `Hrmp::check_hrmp_watermark(para, commitments.hrmp_watermark)` for each candidate to check rules of processing the HRMP watermark.
1. using `Hrmp::check_outbound_hrmp(sender, commitments.horizontal_messages)` ensure that the each candidate sent a valid set of horizontal messages
1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield.
1. check the backing of the candidate using the signatures and the bitfields, comparing against the validators
assigned to the groups, fetched with the `group_validators` lookup, while group indices are computed by `Scheduler`
according to group rotation info.
1. call `check_upward_messages(config, para, commitments.upward_messages)` to check that the upward messages are
valid.
1. call `Dmp::check_processed_downward_messages(para, commitments.processed_downward_messages)` to check that the DMQ
is properly drained.
1. call `Hrmp::check_hrmp_watermark(para, commitments.hrmp_watermark)` for each candidate to check rules of processing
the HRMP watermark.
1. using `Hrmp::check_outbound_hrmp(sender, commitments.horizontal_messages)` ensure that the each candidate sent a
valid set of horizontal messages
1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes`
bitfield.
1. create a corresponding entry in the `PendingAvailabilityCommitments` with the commitments.
1. Return a `Vec<CoreIndex>` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex.
1. Return a `Vec<CoreIndex>` of all scheduled cores of the list of passed assignments that a candidate was
successfully backed for, sorted ascending by CoreIndex.
* `enact_candidate(relay_parent_number: BlockNumber, CommittedCandidateReceipt)`:
1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number, config)`.
> TODO: Note that this is safe as long as we never enact candidates where the relay parent is across a session boundary. In that case, which we should be careful to avoid with contextual execution, the configuration might have changed and the para may de-sync from the host's understanding of it.
1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number,
config)`.
> TODO: Note that this is safe as long as we never enact candidates where the relay parent is across a session
> boundary. In that case, which we should be careful to avoid with contextual execution, the configuration might
> have changed and the para may de-sync from the host's understanding of it.
1. Reward all backing validators of each candidate, contained within the `backers` field.
1. call `receive_upward_messages` for each backed candidate, using the [`UpwardMessage`s](../types/messages.md#upward-message) from the [`CandidateCommitments`](../types/candidate.md#candidate-commitments).
1. call `receive_upward_messages` for each backed candidate, using the
[`UpwardMessage`s](../types/messages.md#upward-message) from the
[`CandidateCommitments`](../types/candidate.md#candidate-commitments).
1. call `Dmp::prune_dmq` with the para id of the candidate and the candidate's `processed_downward_messages`.
1. call `Hrmp::prune_hrmp` with the para id of the candiate and the candidate's `hrmp_watermark`.
1. call `Hrmp::queue_outbound_hrmp` with the para id of the candidate and the list of horizontal messages taken from the commitment,
1. call `Hrmp::queue_outbound_hrmp` with the para id of the candidate and the list of horizontal messages taken from
the commitment,
1. Call `Paras::note_new_head` using the `HeadData` from the receipt and `relay_parent_number`.
* `collect_pending`:
@@ -130,21 +147,28 @@ All failed checks should lead to an unrecoverable error making the block invalid
// return a vector of cleaned-up core IDs.
}
```
* `force_enact(ParaId)`: Forcibly enact the candidate with the given ID as though it had been deemed available by bitfields. Is a no-op if there is no candidate pending availability for this para-id. This should generally not be used but it is useful during execution of Runtime APIs, where the changes to the state are expected to be discarded directly after.
* `candidate_pending_availability(ParaId) -> Option<CommittedCandidateReceipt>`: returns the `CommittedCandidateReceipt` pending availability for the para provided, if any.
* `pending_availability(ParaId) -> Option<CandidatePendingAvailability>`: returns the metadata around the candidate pending availability for the para, if any.
* `collect_disputed(disputed: Vec<CandidateHash>) -> Vec<CoreIndex>`: Sweeps through all paras pending availability. If the candidate hash is one of the disputed candidates, then clean up the corresponding storage for that candidate and the commitments. Return a vector of cleaned-up core IDs.
* `force_enact(ParaId)`: Forcibly enact the candidate with the given ID as though it had been deemed available by
bitfields. Is a no-op if there is no candidate pending availability for this para-id. This should generally not be
used but it is useful during execution of Runtime APIs, where the changes to the state are expected to be discarded
directly after.
* `candidate_pending_availability(ParaId) -> Option<CommittedCandidateReceipt>`: returns the `CommittedCandidateReceipt`
pending availability for the para provided, if any.
* `pending_availability(ParaId) -> Option<CandidatePendingAvailability>`: returns the metadata around the candidate
pending availability for the para, if any.
* `collect_disputed(disputed: Vec<CandidateHash>) -> Vec<CoreIndex>`: Sweeps through all paras pending availability. If
the candidate hash is one of the disputed candidates, then clean up the corresponding storage for that candidate and
the commitments. Return a vector of cleaned-up core IDs.
These functions were formerly part of the UMP pallet:
* `check_upward_messages(P: ParaId, Vec<UpwardMessage>)`:
1. Checks that the parachain is not currently offboarding and error otherwise.
1. Checks that the parachain is not currently offboarding and error otherwise.
1. Checks that there are at most `config.max_upward_message_num_per_candidate` messages to be enqueued.
1. Checks that no message exceeds `config.max_upward_message_size`.
1. Checks that the total resulting queue size would not exceed `co`.
1. Verify that queuing up the messages could not result in exceeding the queue's footprint
according to the config items `config.max_upward_queue_count` and `config.max_upward_queue_size`. The queue's current footprint is provided in `well_known_keys`
in order to facilitate oraclisation on to the para.
1. Verify that queuing up the messages could not result in exceeding the queue's footprint according to the config
items `config.max_upward_queue_count` and `config.max_upward_queue_size`. The queue's current footprint is provided
in `well_known_keys` in order to facilitate oraclisation on to the para.
Candidate Enactment:
polkadot/roadmap/implementers-guide/src/runtime/initializer.md
+16 -5
View File
@@ -1,6 +1,7 @@
# Initializer Pallet
This module is responsible for initializing the other modules in a deterministic order. It also has one other purpose as described in the overview of the runtime: accepting and forwarding session change notifications.
This module is responsible for initializing the other modules in a deterministic order. It also has one other purpose as
described in the overview of the runtime: accepting and forwarding session change notifications.
## Storage
@@ -15,7 +16,9 @@ BufferedSessionChanges: Vec<(BlockNumber, ValidatorSet, ValidatorSet)>;
## Initialization
Before initializing modules, remove all changes from the `BufferedSessionChanges` with number less than or equal to the current block number, and apply the last one. The session change is applied to all modules in the same order as initialization.
Before initializing modules, remove all changes from the `BufferedSessionChanges` with number less than or equal to the
current block number, and apply the last one. The session change is applied to all modules in the same order as
initialization.
The other parachains modules are initialized in this order:
@@ -30,16 +33,24 @@ The other parachains modules are initialized in this order:
1. UMP
1. HRMP
The [Configuration Module](configuration.md) is first, since all other modules need to operate under the same configuration as each other. Then the [Shared](shared.md) module is invoked, which determines the set of active validators. It would lead to inconsistency if, for example, the scheduler ran first and then the configuration was updated before the Inclusion module.
The [Configuration Module](configuration.md) is first, since all other modules need to operate under the same
configuration as each other. Then the [Shared](shared.md) module is invoked, which determines the set of active
validators. It would lead to inconsistency if, for example, the scheduler ran first and then the configuration was
updated before the Inclusion module.
Set `HasInitialized` to true.
## Session Change
Store the session change information in `BufferedSessionChange` along with the block number at which it was submitted, plus one. Although the expected operational parameters of the block authorship system should prevent more than one change from being buffered at any time, it may occur. Regardless, we always need to track the block number at which the session change can be applied so as to remain flexible over session change notifications being issued before or after initialization of the current block.
Store the session change information in `BufferedSessionChange` along with the block number at which it was submitted,
plus one. Although the expected operational parameters of the block authorship system should prevent more than one
change from being buffered at any time, it may occur. Regardless, we always need to track the block number at which the
session change can be applied so as to remain flexible over session change notifications being issued before or after
initialization of the current block.
## Finalization
Finalization order is less important in this case than initialization order, so we finalize the modules in the reverse order from initialization.
Finalization order is less important in this case than initialization order, so we finalize the modules in the reverse
order from initialization.
Set `HasInitialized` to false.
polkadot/roadmap/implementers-guide/src/runtime/parainherent.md
+31 -26
View File
@@ -1,15 +1,27 @@
# `ParaInherent`
This module is responsible for providing all data given to the runtime by the block author to the various parachains modules. The entry-point is mandatory, in that it must be invoked exactly once within every block, and it is also "inherent", in that it is provided with no origin by the block author. The data within it carries its own authentication; i.e. the data takes the form of signed statements by validators. Invalid data will be filtered and not applied.
This module is responsible for providing all data given to the runtime by the block author to the various parachains
modules. The entry-point is mandatory, in that it must be invoked exactly once within every block, and it is also
"inherent", in that it is provided with no origin by the block author. The data within it carries its own
authentication; i.e. the data takes the form of signed statements by validators. Invalid data will be filtered and not
applied.
This module does not have the same initialization/finalization concerns as the others, as it only requires that entry points be triggered after all modules have initialized and that finalization happens after entry points are triggered. Both of these are assumptions we have already made about the runtime's order of operations, so this module doesn't need to be initialized or finalized by the `Initializer`.
This module does not have the same initialization/finalization concerns as the others, as it only requires that entry
points be triggered after all modules have initialized and that finalization happens after entry points are triggered.
Both of these are assumptions we have already made about the runtime's order of operations, so this module doesn't need
to be initialized or finalized by the `Initializer`.
There are a couple of important notes to the operations in this inherent as they relate to disputes.
1. We don't accept bitfields or backed candidates if in "governance-only" mode from having a local dispute conclude on this fork.
1. When disputes are initiated, we remove the block from pending availability. This allows us to roll back chains to the block before blocks are included as opposed to backing. It's important to do this before processing bitfields.
1. `Inclusion::collect_disputed` is kind of expensive so it's important to gate this on whether there are actually any new disputes. Which should be never.
1. And we don't accept parablocks that have open disputes or disputes that have concluded against the candidate. It's important to import dispute statements before backing, but this is already the case as disputes are imported before processing bitfields.
1. We don't accept bitfields or backed candidates if in "governance-only" mode from having a local dispute conclude on
this fork.
1. When disputes are initiated, we remove the block from pending availability. This allows us to roll back chains to the
block before blocks are included as opposed to backing. It's important to do this before processing bitfields.
1. `Inclusion::collect_disputed` is kind of expensive so it's important to gate this on whether there are actually any
new disputes. Which should be never.
1. And we don't accept parablocks that have open disputes or disputes that have concluded against the candidate. It's
important to import dispute statements before backing, but this is already the case as disputes are imported before
processing bitfields.
## Storage
@@ -32,26 +44,19 @@ OnChainVotes: Option<ScrapedOnChainVotes>,
* `enter`: This entry-point accepts one parameter: [`ParaInherentData`](../types/runtime.md#ParaInherentData).
* `create_inherent`: This entry-point accepts one parameter: `InherentData`.
Both entry points share mostly the same code. `create_inherent` will
meaningfully limit inherent data to adhere to the weight limit, in addition to
sanitizing any inputs and filtering out invalid data. Conceptually it is part of
the block production. The `enter` call on the other hand is part of block import
and consumes/imports the data previously produced by `create_inherent`.
Both entry points share mostly the same code. `create_inherent` will meaningfully limit inherent data to adhere to the
weight limit, in addition to sanitizing any inputs and filtering out invalid data. Conceptually it is part of the block
production. The `enter` call on the other hand is part of block import and consumes/imports the data previously produced
by `create_inherent`.
In practice both calls process inherent data and apply it to the state. Block
production and block import should arrive at the same new state. Hence we re-use
the same logic to ensure this is the case.
In practice both calls process inherent data and apply it to the state. Block production and block import should arrive
at the same new state. Hence we re-use the same logic to ensure this is the case.
The only real difference between the two is, that on `create_inherent` we
actually need the processed and filtered inherent data to build the block, while
on `enter` the processed data should for one be identical to the incoming
inherent data (assuming honest block producers) and second it is irrelevant, as
we are not building a block but just processing it, so the processed inherent
data is simply dropped.
This also means that the `enter` function keeps data around for no good reason.
This seems acceptable though as the size of a block is rather limited.
Nevertheless if we ever wanted to optimize this we can easily implement an
inherent collector that has two implementations, where one clones and stores the
data and the other just passes it on.
The only real difference between the two is, that on `create_inherent` we actually need the processed and filtered
inherent data to build the block, while on `enter` the processed data should for one be identical to the incoming
inherent data (assuming honest block producers) and second it is irrelevant, as we are not building a block but just
processing it, so the processed inherent data is simply dropped.
This also means that the `enter` function keeps data around for no good reason. This seems acceptable though as the size
of a block is rather limited. Nevertheless if we ever wanted to optimize this we can easily implement an inherent
collector that has two implementations, where one clones and stores the data and the other just passes it on.
polkadot/roadmap/implementers-guide/src/runtime/paras.md
+53 -44
View File
@@ -1,14 +1,12 @@
# Paras Pallet
The Paras module is responsible for storing information on parachains. Registered
parachains cannot change except at session boundaries and after at least a full
session has passed. This is primarily to ensure that the number and meaning of bits required for the
availability bitfields does not change except at session boundaries.
The Paras module is responsible for storing information on parachains. Registered parachains cannot change except at
session boundaries and after at least a full session has passed. This is primarily to ensure that the number and meaning
of bits required for the availability bitfields does not change except at session boundaries.
It's also responsible for:
- managing parachain validation code upgrades as well as maintaining availability of old parachain
code and its pruning.
- managing parachain validation code upgrades as well as maintaining availability of old parachain code and its pruning.
- vetting PVFs by means of the PVF pre-checking mechanism.
## Storage
@@ -102,8 +100,8 @@ struct PvfCheckActiveVoteState {
#### Para Lifecycle
Because the state changes of parachains are delayed, we track the specific state of
the para using the `ParaLifecycle` enum.
Because the state changes of parachains are delayed, we track the specific state of the para using the `ParaLifecycle`
enum.
```
None Parathread (on-demand parachain) Parachain
@@ -132,8 +130,8 @@ None Parathread (on-demand parachain) Parachain
+ + +
```
Note that if PVF pre-checking is enabled, onboarding of a para may potentially be delayed. This can
happen due to PVF pre-checking voting concluding late.
Note that if PVF pre-checking is enabled, onboarding of a para may potentially be delayed. This can happen due to PVF
pre-checking voting concluding late.
During the transition period, the para object is still considered in its existing state.
@@ -210,7 +208,7 @@ UpcomingUpgrades: Vec<(ParaId, BlockNumberFor<T>)>;
ActionsQueue: map SessionIndex => Vec<ParaId>;
/// Upcoming paras instantiation arguments.
///
/// NOTE that after PVF pre-checking is enabled the para genesis arg will have it's code set
/// NOTE that after PVF pre-checking is enabled the para genesis arg will have it's code set
/// to empty. Instead, the code will be saved into the storage right away via `CodeByHash`.
UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
/// The number of references on the validation code in `CodeByHash` storage.
@@ -223,12 +221,13 @@ CodeByHash: map ValidationCodeHash => Option<ValidationCode>
1. Execute all queued actions for paralifecycle changes:
1. Clean up outgoing paras.
1. This means removing the entries under `Heads`, `CurrentCode`, `FutureCodeUpgrades`,
`FutureCode` and `MostRecentContext`. An according entry should be added to `PastCode`, `PastCodeMeta`, and `PastCodePruning` using the outgoing `ParaId` and removed `CurrentCode` value. This is because any outdated validation code must remain available on-chain for a determined amount
of blocks, and validation code outdated by de-registering the para is still subject to that
invariant.
1. Apply all incoming paras by initializing the `Heads` and `CurrentCode` using the genesis
parameters as well as `MostRecentContext` to `0`.
1. This means removing the entries under `Heads`, `CurrentCode`, `FutureCodeUpgrades`, `FutureCode` and
`MostRecentContext`. An according entry should be added to `PastCode`, `PastCodeMeta`, and `PastCodePruning`
using the outgoing `ParaId` and removed `CurrentCode` value. This is because any outdated validation code must
remain available on-chain for a determined amount of blocks, and validation code outdated by de-registering the
para is still subject to that invariant.
1. Apply all incoming paras by initializing the `Heads` and `CurrentCode` using the genesis parameters as well as
`MostRecentContext` to `0`.
1. Amend the `Parachains` list and `ParaLifecycle` to reflect changes in registered parachains.
1. Amend the `ParaLifecycle` set to reflect changes in registered on-demand parachains.
1. Upgrade all on-demand parachains that should become lease holding parachains, updating the `Parachains` list and
@@ -239,40 +238,50 @@ CodeByHash: map ValidationCodeHash => Option<ValidationCode>
1. Go over all active PVF pre-checking votes:
1. Increment `age` of the vote.
1. If `age` reached `cfg.pvf_voting_ttl`, then enact PVF rejection and remove the vote from the active list.
1. Otherwise, reinitialize the ballots.
1. Resize the `votes_accept`/`votes_reject` to have the same length as the incoming validator set.
1. Zero all the votes.
1. Otherwise, reinitialize the ballots. 1. Resize the `votes_accept`/`votes_reject` to have the same length as the
incoming validator set. 1. Zero all the votes.
## Initialization
1. Do pruning based on all entries in `PastCodePruning` with `BlockNumber <= now`. Update the
corresponding `PastCodeMeta` and `PastCode` accordingly.
1. Do pruning based on all entries in `PastCodePruning` with `BlockNumber <= now`. Update the corresponding
`PastCodeMeta` and `PastCode` accordingly.
1. Toggle the upgrade related signals
1. Collect all `(para_id, expected_at)` from `UpcomingUpgrades` where `expected_at <= now` and prune them. For each para pruned set `UpgradeGoAheadSignal` to `GoAhead`. Reserve weight for the state modification to upgrade each para pruned.
1. Collect all `(para_id, next_possible_upgrade_at)` from `UpgradeCooldowns` where `next_possible_upgrade_at <= now`. For each para obtained this way reserve weight to remove its `UpgradeRestrictionSignal` on finalization.
1. Collect all `(para_id, expected_at)` from `UpcomingUpgrades` where `expected_at <= now` and prune them. For each
para pruned set `UpgradeGoAheadSignal` to `GoAhead`. Reserve weight for the state modification to upgrade each para
pruned.
1. Collect all `(para_id, next_possible_upgrade_at)` from `UpgradeCooldowns` where `next_possible_upgrade_at <= now`.
For each para obtained this way reserve weight to remove its `UpgradeRestrictionSignal` on finalization.
## Routines
* `schedule_para_initialize(ParaId, ParaGenesisArgs)`: Schedule a para to be initialized at the next
session. Noop if para is already registered in the system with some `ParaLifecycle`.
* `schedule_para_cleanup(ParaId)`: Schedule a para to be cleaned up after the next full session.
* `schedule_parathread_upgrade(ParaId)`: Schedule a parathread (on-demand parachain) to be upgraded to a parachain.
* `schedule_parachain_downgrade(ParaId)`: Schedule a parachain to be downgraded from lease holding to on-demand.
* `schedule_code_upgrade(ParaId, new_code, relay_parent: BlockNumber, HostConfiguration)`: Schedule a future code
upgrade of the given parachain. In case the PVF pre-checking is disabled, or the new code is already present in the storage, the upgrade will be applied after inclusion of a block of the same parachain
executed in the context of a relay-chain block with number >= `relay_parent + config.validation_upgrade_delay`. If the upgrade is scheduled `UpgradeRestrictionSignal` is set and it will remain set until `relay_parent + config.validation_upgrade_cooldown`.
In case the PVF pre-checking is enabled, or the new code is not already present in the storage, then the PVF pre-checking run will be scheduled for that validation code. If the pre-checking concludes with rejection, then the upgrade is canceled. Otherwise, after pre-checking is concluded the upgrade will be scheduled and be enacted as described above.
* `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head,
where the new head was executed in the context of a relay-chain block with given number, the latter value is inserted into the `MostRecentContext` mapping. This will apply pending code upgrades based on the block number provided. If an upgrade took place it will clear the `UpgradeGoAheadSignal`.
* `lifecycle(ParaId) -> Option<ParaLifecycle>`: Return the `ParaLifecycle` of a para.
* `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live lease holding parachain,
including those which may be transitioning to an on-demand parachain in the future.
* `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread (on-demand parachain),
- `schedule_para_initialize(ParaId, ParaGenesisArgs)`: Schedule a para to be initialized at the next session. Noop if
para is already registered in the system with some `ParaLifecycle`.
- `schedule_para_cleanup(ParaId)`: Schedule a para to be cleaned up after the next full session.
- `schedule_parathread_upgrade(ParaId)`: Schedule a parathread (on-demand parachain) to be upgraded to a parachain.
- `schedule_parachain_downgrade(ParaId)`: Schedule a parachain to be downgraded from lease holding to on-demand.
- `schedule_code_upgrade(ParaId, new_code, relay_parent: BlockNumber, HostConfiguration)`: Schedule a future code
upgrade of the given parachain. In case the PVF pre-checking is disabled, or the new code is already present in the
storage, the upgrade will be applied after inclusion of a block of the same parachain executed in the context of a
relay-chain block with number >= `relay_parent + config.validation_upgrade_delay`. If the upgrade is scheduled
`UpgradeRestrictionSignal` is set and it will remain set until `relay_parent + config.validation_upgrade_cooldown`. In
case the PVF pre-checking is enabled, or the new code is not already present in the storage, then the PVF pre-checking
run will be scheduled for that validation code. If the pre-checking concludes with rejection, then the upgrade is
canceled. Otherwise, after pre-checking is concluded the upgrade will be scheduled and be enacted as described above.
- `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head, where the new head was
executed in the context of a relay-chain block with given number, the latter value is inserted into the
`MostRecentContext` mapping. This will apply pending code upgrades based on the block number provided. If an upgrade
took place it will clear the `UpgradeGoAheadSignal`.
- `lifecycle(ParaId) -> Option<ParaLifecycle>`: Return the `ParaLifecycle` of a para.
- `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live lease holding parachain, including
those which may be transitioning to an on-demand parachain in the future.
- `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread (on-demand parachain),
including those which may be transitioning to a lease holding parachain in the future.
* `is_valid_para(ParaId) -> bool`: Returns true if the para ID references either a live on-demand parachain
or live lease holding parachain.
* `can_upgrade_validation_code(ParaId) -> bool`: Returns true if the given para can signal code upgrade right now.
* `pvfs_require_prechecking() -> Vec<ValidationCodeHash>`: Returns the list of PVF validation code hashes that require PVF pre-checking votes.
- `is_valid_para(ParaId) -> bool`: Returns true if the para ID references either a live on-demand parachain or live
lease holding parachain.
- `can_upgrade_validation_code(ParaId) -> bool`: Returns true if the given para can signal code upgrade right now.
- `pvfs_require_prechecking() -> Vec<ValidationCodeHash>`: Returns the list of PVF validation code hashes that require
PVF pre-checking votes.
## Finalization
Collect all `(para_id, next_possible_upgrade_at)` from `UpgradeCooldowns` where `next_possible_upgrade_at <= now` and prune them. For each para pruned remove its `UpgradeRestrictionSignal`.
Collect all `(para_id, next_possible_upgrade_at)` from `UpgradeCooldowns` where `next_possible_upgrade_at <= now` and
prune them. For each para pruned remove its `UpgradeRestrictionSignal`.
polkadot/roadmap/implementers-guide/src/runtime/scheduler.md
+106 -31
View File
@@ -1,6 +1,7 @@
# Scheduler Pallet
> TODO: this section is still heavily under construction. key questions about availability cores and validator assignment are still open and the flow of the the section may be contradictory or inconsistent
> TODO: this section is still heavily under construction. key questions about availability cores and validator
> assignment are still open and the flow of the the section may be contradictory or inconsistent
The Scheduler module is responsible for two main tasks:
@@ -9,16 +10,29 @@ The Scheduler module is responsible for two main tasks:
It aims to achieve these tasks with these goals in mind:
- It should be possible to know at least a block ahead-of-time, ideally more, which validators are going to be assigned to which parachains.
- It should be possible to know at least a block ahead-of-time, ideally more, which validators are going to be assigned
to which parachains.
- Parachains that have a candidate pending availability in this fork of the chain should not be assigned.
- Validator assignments should not be gameable. Malicious cartels should not be able to manipulate the scheduler to assign themselves as desired.
- Validator assignments should not be gameable. Malicious cartels should not be able to manipulate the scheduler to
assign themselves as desired.
- High or close to optimal throughput of parachains. Work among validator groups should be balanced.
## Availability Cores
The Scheduler manages resource allocation using the concept of "Availability Cores". There will be one availability core for each lease holding parachain, and a fixed number of cores used for multiplexing on-demand parachains. Validators will be partitioned into groups, with the same number of groups as availability cores. Validator groups will be assigned to different availability cores over time.
The Scheduler manages resource allocation using the concept of "Availability Cores". There will be one availability core
for each lease holding parachain, and a fixed number of cores used for multiplexing on-demand parachains. Validators
will be partitioned into groups, with the same number of groups as availability cores. Validator groups will be assigned
to different availability cores over time.
An availability core can exist in either one of two states at the beginning or end of a block: free or occupied. A free availability core can have a lease holding or on-demand parachain assigned to it for the potential to have a backed candidate included. After backing, the core enters the occupied state as the backed candidate is pending availability. There is an important distinction: a core is not considered occupied until it is in charge of a block pending availability, although the implementation may treat scheduled cores the same as occupied ones for brevity. A core exits the occupied state when the candidate is no longer pending availability - either on timeout or on availability. A core starting in the occupied state can move to the free state and back to occupied all within a single block, as availability bitfields are processed before backed candidates. At the end of the block, there is a possible timeout on availability which can move the core back to the free state if occupied.
An availability core can exist in either one of two states at the beginning or end of a block: free or occupied. A free
availability core can have a lease holding or on-demand parachain assigned to it for the potential to have a backed
candidate included. After backing, the core enters the occupied state as the backed candidate is pending availability.
There is an important distinction: a core is not considered occupied until it is in charge of a block pending
availability, although the implementation may treat scheduled cores the same as occupied ones for brevity. A core exits
the occupied state when the candidate is no longer pending availability - either on timeout or on availability. A core
starting in the occupied state can move to the free state and back to occupied all within a single block, as
availability bitfields are processed before backed candidates. At the end of the block, there is a possible timeout on
availability which can move the core back to the free state if occupied.
Cores are treated as an ordered list and are typically referred to by their index in that list.
@@ -82,19 +96,47 @@ digraph {
## Validator Groups
Validator group assignments do not need to change very quickly. The security benefits of fast rotation are redundant with the challenge mechanism in the [Approval process](../protocol-approval.md). Because of this, we only divide validators into groups at the beginning of the session and do not shuffle membership during the session. However, we do take steps to ensure that no particular validator group has dominance over a single lease holding parachain or on-demand parachain-multiplexer for an entire session to provide better guarantees of live-ness.
Validator group assignments do not need to change very quickly. The security benefits of fast rotation are redundant
with the challenge mechanism in the [Approval process](../protocol-approval.md). Because of this, we only divide
validators into groups at the beginning of the session and do not shuffle membership during the session. However, we do
take steps to ensure that no particular validator group has dominance over a single lease holding parachain or on-demand
parachain-multiplexer for an entire session to provide better guarantees of live-ness.
Validator groups rotate across availability cores in a round-robin fashion, with rotation occurring at fixed intervals. The i'th group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of rotations that have occurred in the session, and `n` is the number of cores. This makes upcoming rotations within the same session predictable.
Validator groups rotate across availability cores in a round-robin fashion, with rotation occurring at fixed intervals.
The i'th group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of rotations that
have occurred in the session, and `n` is the number of cores. This makes upcoming rotations within the same session
predictable.
When a rotation occurs, validator groups are still responsible for distributing availability chunks for any previous cores that are still occupied and pending availability. In practice, rotation and availability-timeout frequencies should be set so this will only be the core they have just been rotated from. It is possible that a validator group is rotated onto a core which is currently occupied. In this case, the validator group will have nothing to do until the previously-assigned group finishes their availability work and frees the core or the availability process times out. Depending on if the core is for a lease holding parachain or on-demand parachain, a different timeout `t` from the [`HostConfiguration`](../types/runtime.md#host-configuration) will apply. Availability timeouts should only be triggered in the first `t-1` blocks after the beginning of a rotation.
When a rotation occurs, validator groups are still responsible for distributing availability chunks for any previous
cores that are still occupied and pending availability. In practice, rotation and availability-timeout frequencies
should be set so this will only be the core they have just been rotated from. It is possible that a validator group is
rotated onto a core which is currently occupied. In this case, the validator group will have nothing to do until the
previously-assigned group finishes their availability work and frees the core or the availability process times out.
Depending on if the core is for a lease holding parachain or on-demand parachain, a different timeout `t` from the
[`HostConfiguration`](../types/runtime.md#host-configuration) will apply. Availability timeouts should only be triggered
in the first `t-1` blocks after the beginning of a rotation.
## Claims
On-demand parachains operate on a system of claims. Collators purchase claims on authoring the next block of an on-demand parachain, although the purchase mechanism is beyond the scope of the scheduler. The scheduler guarantees that they'll be given at least a certain number of attempts to author a candidate that is backed. Attempts that fail during the availability phase are not counted, since ensuring availability at that stage is the responsibility of the backing validators, not of the collator. When a claim is accepted, it is placed into a queue of claims, and each claim is assigned to a particular on-demand parachain-multiplexing core in advance. Given that the current assignments of validator groups to cores are known, and the upcoming assignments are predictable, it is possible for on-demand parachain collators to know who they should be talking to now and how they should begin establishing connections with as a fallback.
On-demand parachains operate on a system of claims. Collators purchase claims on authoring the next block of an
on-demand parachain, although the purchase mechanism is beyond the scope of the scheduler. The scheduler guarantees that
they'll be given at least a certain number of attempts to author a candidate that is backed. Attempts that fail during
the availability phase are not counted, since ensuring availability at that stage is the responsibility of the backing
validators, not of the collator. When a claim is accepted, it is placed into a queue of claims, and each claim is
assigned to a particular on-demand parachain-multiplexing core in advance. Given that the current assignments of
validator groups to cores are known, and the upcoming assignments are predictable, it is possible for on-demand
parachain collators to know who they should be talking to now and how they should begin establishing connections with as
a fallback.
With this information, the Node-side can be aware of which on-demand parachains have a good chance of being includable within the relay-chain block and can focus any additional resources on backing candidates from those on-demand parachains. Furthermore, Node-side code is aware of which validator group will be responsible for that thread. If the necessary conditions are reached for core reassignment, those candidates can be backed within the same block as the core being freed.
With this information, the Node-side can be aware of which on-demand parachains have a good chance of being includable
within the relay-chain block and can focus any additional resources on backing candidates from those on-demand
parachains. Furthermore, Node-side code is aware of which validator group will be responsible for that thread. If the
necessary conditions are reached for core reassignment, those candidates can be backed within the same block as the core
being freed.
On-demand claims, when scheduled onto a free core, may not result in a block pending availability. This may be due to collator error, networking timeout, or censorship by the validator group. In this case, the claims should be retried a certain number of times to give the collator a fair shot.
On-demand claims, when scheduled onto a free core, may not result in a block pending availability. This may be due to
collator error, networking timeout, or censorship by the validator group. In this case, the claims should be retried a
certain number of times to give the collator a fair shot.
## Storage
@@ -104,7 +146,7 @@ Utility structs:
// A claim on authoring the next block for a given parathread (on-demand parachain).
struct ParathreadClaim(ParaId, CollatorId);
// An entry tracking a parathread (on-demand parachain) claim to ensure it does not
// An entry tracking a parathread (on-demand parachain) claim to ensure it does not
// pass the maximum number of retries.
struct ParathreadEntry {
claim: ParathreadClaim,
@@ -165,7 +207,7 @@ ParathreadClaimIndex: Vec<ParaId>;
/// The block number where the session start occurred. Used to track how many group rotations have occurred.
SessionStartBlock: BlockNumber;
/// Currently scheduled cores - free but up to be occupied.
/// The value contained here will not be valid after the end of a block.
/// The value contained here will not be valid after the end of a block.
/// Runtime APIs should be used to determine scheduled cores
/// for the upcoming block.
Scheduled: Vec<CoreAssignment>, // sorted ascending by CoreIndex.
@@ -173,13 +215,17 @@ Scheduled: Vec<CoreAssignment>, // sorted ascending by CoreIndex.
## Session Change
Session changes are the only time that configuration can change, and the [Configuration module](configuration.md)'s session-change logic is handled before this module's. We also lean on the behavior of the [Inclusion module](inclusion.md) which clears all its occupied cores on session change. Thus we don't have to worry about cores being occupied across session boundaries and it is safe to re-size the `AvailabilityCores` bitfield.
Session changes are the only time that configuration can change, and the [Configuration module](configuration.md)'s
session-change logic is handled before this module's. We also lean on the behavior of the [Inclusion
module](inclusion.md) which clears all its occupied cores on session change. Thus we don't have to worry about cores
being occupied across session boundaries and it is safe to re-size the `AvailabilityCores` bitfield.
Actions:
1. Set `SessionStartBlock` to current block number + 1, as session changes are applied at the end of the block.
1. Clear all `Some` members of `AvailabilityCores`. Return all parathread claims to queue with retries un-incremented.
1. Set `configuration = Configuration::configuration()` (see [`HostConfiguration`](../types/runtime.md#host-configuration))
1. Set `configuration = Configuration::configuration()` (see
[`HostConfiguration`](../types/runtime.md#host-configuration))
1. Fetch `Shared::ActiveValidators` as AV.
1. Determine the number of cores & validator groups as `n_cores`. This is the maximum of
1. `Paras::parachains().len() + configuration.parathread_cores`
@@ -187,13 +233,18 @@ Actions:
1. Resize `AvailabilityCores` to have length `n_cores` with all `None` entries.
1. Compute new validator groups by shuffling using a secure randomness beacon
- Note that the total number of validators `V` in AV may not be evenly divided by `n_cores`.
- The groups are selected by partitioning AV. The first `V % N` groups will have `(V / n_cores) + 1` members, while the remaining groups will have `(V / N)` members each.
- Instead of using the indices within AV, which point to the broader set, indices _into_ AV should be used. This implies that groups should have simply ascending validator indices.
- The groups are selected by partitioning AV. The first `V % N` groups will have `(V / n_cores) + 1` members, while
the remaining groups will have `(V / N)` members each.
- Instead of using the indices within AV, which point to the broader set, indices _into_ AV should be used. This
implies that groups should have simply ascending validator indices.
1. Prune the parathread (on-demand parachain) queue to remove all retries beyond `configuration.parathread_retries`.
- Also prune all on-demand claims corresponding to de-registered parachains.
- all pruned claims should have their entry removed from the parathread (on-demand parachain) index.
- assign all non-pruned claims to new cores if the number of on-demand parachain cores has changed between the `new_config` and `old_config` of the `SessionChangeNotification`.
- Assign claims in equal balance across all cores if rebalancing, and set the `next_core` of the `ParathreadQueue` (on-demand queue) by incrementing the relative index of the last assigned core and taking it modulo the number of on-demand cores.
- assign all non-pruned claims to new cores if the number of on-demand parachain cores has changed between the
`new_config` and `old_config` of the `SessionChangeNotification`.
- Assign claims in equal balance across all cores if rebalancing, and set the `next_core` of the `ParathreadQueue`
(on-demand queue) by incrementing the relative index of the last assigned core and taking it modulo the number of
on-demand cores.
## Initialization
@@ -208,28 +259,52 @@ No finalization routine runs for this module.
- `add_parathread_claim(ParathreadClaim)`: Add a parathread (on-demand parachain) claim to the queue.
- Fails if any on-demand claim on the same parachain is currently indexed.
- Fails if the queue length is >= `config.scheduling_lookahead * config.parathread_cores`.
- The core used for the on-demand claim is the `next_core` field of the `ParathreadQueue` (on-demand queue) and adding `Paras::parachains().len()` to it.
- The core used for the on-demand claim is the `next_core` field of the `ParathreadQueue` (on-demand queue) and adding
`Paras::parachains().len()` to it.
- `next_core` is then updated by adding 1 and taking it modulo `config.parathread_cores`.
- The claim is then added to the claim index.
- `free_cores(Vec<(CoreIndex, FreedReason)>)`: indicate previosuly-occupied cores which are to be considered returned and why they are being returned.
- `free_cores(Vec<(CoreIndex, FreedReason)>)`: indicate previosuly-occupied cores which are to be considered returned
and why they are being returned.
- All freed lease holding parachain cores should be assigned to their respective parachain
- All freed on-demand parachain cores whose reason for freeing was `FreedReason::Concluded` should have the claim removed from the claim index.
- All freed on-demand cores whose reason for freeing was `FreedReason::TimedOut` should have the claim added to the parathread queue (on-demand queue) again without retries incremented
- All freed on-demand parachain cores whose reason for freeing was `FreedReason::Concluded` should have the claim
removed from the claim index.
- All freed on-demand cores whose reason for freeing was `FreedReason::TimedOut` should have the claim added to the
parathread queue (on-demand queue) again without retries incremented
- All freed on-demand cores should take the next on-demand parachain entry from the queue.
- `schedule(Vec<(CoreIndex, FreedReason)>, now: BlockNumber)`: schedule new core assignments, with a parameter indicating previously-occupied cores which are to be considered returned and why they are being returned.
- `schedule(Vec<(CoreIndex, FreedReason)>, now: BlockNumber)`: schedule new core assignments, with a parameter
indicating previously-occupied cores which are to be considered returned and why they are being returned.
- Invoke `free_cores(freed_cores)`
- The i'th validator group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of rotations that have occurred in the session, and `n` is the total number of cores. This makes upcoming rotations within the same session predictable. Rotations are based off of `now`.
- The i'th validator group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of
rotations that have occurred in the session, and `n` is the total number of cores. This makes upcoming rotations
within the same session predictable. Rotations are based off of `now`.
- `scheduled() -> Vec<CoreAssignment>`: Get currently scheduled core assignments.
- `occupied(Vec<CoreIndex>)`. Note that the given cores have become occupied.
- Behavior undefined if any given cores were not scheduled.
- Behavior undefined if the given cores are not sorted ascending by core index
- This clears them from `Scheduled` and marks each corresponding `core` in the `AvailabilityCores` as occupied.
- Since both the availability cores and the newly-occupied cores lists are sorted ascending, this method can be implemented efficiently.
- Since both the availability cores and the newly-occupied cores lists are sorted ascending, this method can be
implemented efficiently.
- `core_para(CoreIndex) -> ParaId`: return the currently-scheduled or occupied ParaId for the given core.
- `group_validators(GroupIndex) -> Option<Vec<ValidatorIndex>>`: return all validators in a given group, if the group index is valid for this session.
- `availability_timeout_predicate() -> Option<impl Fn(CoreIndex, BlockNumber) -> bool>`: returns an optional predicate that should be used for timing out occupied cores. if `None`, no timing-out should be done. The predicate accepts the index of the core, and the block number since which it has been occupied. The predicate should be implemented based on the time since the last validator group rotation, and the respective parachain timeouts, i.e. only within `max(config.chain_availability_period, config.thread_availability_period)` of the last rotation would this return `Some`.
- `group_validators(GroupIndex) -> Option<Vec<ValidatorIndex>>`: return all validators in a given group, if the group
index is valid for this session.
- `availability_timeout_predicate() -> Option<impl Fn(CoreIndex, BlockNumber) -> bool>`: returns an optional predicate
that should be used for timing out occupied cores. if `None`, no timing-out should be done. The predicate accepts the
index of the core, and the block number since which it has been occupied. The predicate should be implemented based on
the time since the last validator group rotation, and the respective parachain timeouts, i.e. only within
`max(config.chain_availability_period, config.thread_availability_period)` of the last rotation would this return
`Some`.
- `group_rotation_info(now: BlockNumber) -> GroupRotationInfo`: Returns a helper for determining group rotation.
- `next_up_on_available(CoreIndex) -> Option<ScheduledCore>`: Return the next thing that will be scheduled on this core assuming it is currently occupied and the candidate occupying it became available. Returns in `ScheduledCore` format (todo: link to Runtime APIs page; linkcheck doesn't allow this right now). For lease holding parachains, this is always the ID of the parachain and no specified collator. For on-demand parachains, this is based on the next item in the `ParathreadQueue` (on-demand queue) assigned to that core, and is `None` if there isn't one.
- `next_up_on_time_out(CoreIndex) -> Option<ScheduledCore>`: Return the next thing that will be scheduled on this core assuming it is currently occupied and the candidate occupying it timed out. Returns in `ScheduledCore` format (todo: link to Runtime APIs page; linkcheck doesn't allow this right now). For parachains, this is always the ID of the parachain and no specified collator. For on-demand parachains, this is based on the next item in the `ParathreadQueue` (on-demand queue) assigned to that core, or if there isn't one, the claim that is currently occupying the core. Otherwise `None`.
- `next_up_on_available(CoreIndex) -> Option<ScheduledCore>`: Return the next thing that will be scheduled on this core
assuming it is currently occupied and the candidate occupying it became available. Returns in `ScheduledCore` format
(todo: link to Runtime APIs page; linkcheck doesn't allow this right now). For lease holding parachains, this is
always the ID of the parachain and no specified collator. For on-demand parachains, this is based on the next item in
the `ParathreadQueue` (on-demand queue) assigned to that core, and is `None` if there isn't one.
- `next_up_on_time_out(CoreIndex) -> Option<ScheduledCore>`: Return the next thing that will be scheduled on this core
assuming it is currently occupied and the candidate occupying it timed out. Returns in `ScheduledCore` format (todo:
link to Runtime APIs page; linkcheck doesn't allow this right now). For parachains, this is always the ID of the
parachain and no specified collator. For on-demand parachains, this is based on the next item in the `ParathreadQueue`
(on-demand queue) assigned to that core, or if there isn't one, the claim that is currently occupying the core.
Otherwise `None`.
- `clear()`:
- Free all scheduled cores and return on-demand claims to queue, with retries incremented. Skip on-demand parachains which no longer exist under paras.
- Free all scheduled cores and return on-demand claims to queue, with retries incremented. Skip on-demand parachains
which no longer exist under paras.
polkadot/roadmap/implementers-guide/src/runtime/session_info.md
+10 -4
View File
@@ -1,6 +1,8 @@
# Session Info
For disputes and approvals, we need access to information about validator sets from prior sessions. We also often want easy access to the same information about the current session's validator set. This module aggregates and stores this information in a rolling window while providing easy APIs for access.
For disputes and approvals, we need access to information about validator sets from prior sessions. We also often want
easy access to the same information about the current session's validator set. This module aggregates and stores this
information in a rolling window while providing easy APIs for access.
## Storage
@@ -66,10 +68,14 @@ Sessions: map SessionIndex => Option<SessionInfo>,
## Session Change
1. Update `EarliestStoredSession` based on `config.dispute_period` and remove all entries from `Sessions` from the previous value up to the new value.
1. Create a new entry in `Sessions` with information about the current session. Use `shared::ActiveValidators` to determine the indices into the broader validator sets (validation, assignment, discovery) which are actually used for parachain validation. Only these validators should appear in the `SessionInfo`.
1. Update `EarliestStoredSession` based on `config.dispute_period` and remove all entries from `Sessions` from the
previous value up to the new value.
1. Create a new entry in `Sessions` with information about the current session. Use `shared::ActiveValidators` to
determine the indices into the broader validator sets (validation, assignment, discovery) which are actually used for
parachain validation. Only these validators should appear in the `SessionInfo`.
## Routines
* `earliest_stored_session() -> SessionIndex`: Yields the earliest session for which we have information stored.
* `session_info(session: SessionIndex) -> Option<SessionInfo>`: Yields the session info for the given session, if stored.
* `session_info(session: SessionIndex) -> Option<SessionInfo>`: Yields the session info for the given session, if
stored.
polkadot/roadmap/implementers-guide/src/runtime/shared.md
+17 -15
View File
@@ -2,11 +2,11 @@
This module is responsible for managing shared storage and configuration for other modules.
It is important that other pallets are able to use the Shared Module, so it should not have a
dependency on any other modules in the Parachains Runtime.
It is important that other pallets are able to use the Shared Module, so it should not have a dependency on any other
modules in the Parachains Runtime.
For the moment, it is used exclusively to track the current session index across the Parachains
Runtime system, and when it should be allowed to schedule future changes to Paras or Configurations.
For the moment, it is used exclusively to track the current session index across the Parachains Runtime system, and when
it should be allowed to schedule future changes to Paras or Configurations.
## Constants
@@ -57,24 +57,26 @@ AllowedRelayParents: AllowedRelayParentsTracker<T::Hash, T::BlockNumber>,
The Shared Module currently has no initialization routines.
The Shared Module is initialized directly after the Configuration module, but before all other
modules. It is important to update the Shared Module before any other module since its state may be
used within the logic of other modules, and it is important that the state is consistent across
them.
The Shared Module is initialized directly after the Configuration module, but before all other modules. It is important
to update the Shared Module before any other module since its state may be used within the logic of other modules, and
it is important that the state is consistent across them.
## Session Change
During a session change, the Shared Module receives and stores the current Session Index directly from the initializer module, along with the broader validator set, and it returns the new list of validators.
During a session change, the Shared Module receives and stores the current Session Index directly from the initializer
module, along with the broader validator set, and it returns the new list of validators.
The list of validators should be first shuffled according to the chain's random seed and then truncated. The indices of these validators should be set to `ActiveValidatorIndices` and then returned back to the initializer. `ActiveValidatorKeys` should be set accordingly.
The list of validators should be first shuffled according to the chain's random seed and then truncated. The indices of
these validators should be set to `ActiveValidatorIndices` and then returned back to the initializer.
`ActiveValidatorKeys` should be set accordingly.
This information is used in the:
* Configuration Module: For delaying updates to configurations until at lease one full session has
passed.
* Configuration Module: For delaying updates to configurations until at lease one full session has passed.
* Paras Module: For delaying updates to paras until at least one full session has passed.
Allowed relay parents buffer, which is maintained by [ParaInherent](./parainherent.md) module, is cleared on every session change.
Allowed relay parents buffer, which is maintained by [ParaInherent](./parainherent.md) module, is cleared on every
session change.
## Finalization
@@ -82,6 +84,6 @@ The Shared Module currently has no finalization routines.
## Functions
* `scheduled_sessions() -> SessionIndex`: Return the next session index where updates to the
Parachains Runtime system would be safe to apply.
* `scheduled_sessions() -> SessionIndex`: Return the next session index where updates to the Parachains Runtime system
would be safe to apply.
* `set_session_index(SessionIndex)`: For tests. Set the current session index in the Shared Module.