mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-17 23:01:01 +00:00
I am dumb and can't spell (#1366)
* rename implementor's guide to implementer's guide * fix typos in more places
This commit is contained in:
Robert Habermeier
committed by
GitHub
GitHub
parent
37da08a764
commit
42bd096413
@@ -0,0 +1,57 @@
|
||||
# Runtime Architecture
|
||||
|
||||
It's clear that we want to separate different aspects of the runtime logic into different modules. Modules define their own storage, routines, and entry-points. They also define initialization and finalization logic.
|
||||
|
||||
Due to the (lack of) guarantees provided by a particular blockchain-runtime framework, there is no defined or dependable order in which modules' initialization or finalization logic will run. Supporting this blockchain-runtime framework is important enough to include that same uncertainty in our model of runtime modules in this guide. Furthermore, initialization logic of modules can trigger the entry-points or routines of other modules. This is one architectural pressure against dividing the runtime logic into multiple modules. However, in this case the benefits of splitting things up outweigh the costs, provided that we take certain precautions against initialization and entry-point races.
|
||||
|
||||
We also expect, although it's beyond the scope of this guide, that these runtime modules will exist alongside various other modules. This has two facets to consider. First, even if the modules that we describe here don't invoke each others' entry points or routines during initialization, we still have to protect against those other modules doing that. Second, some of those modules are expected to provide governance capabilities for the chain. Configuration exposed by parachain-host modules is mostly for the benefit of these governance modules, to allow the operators or community of the chain to tweak parameters.
|
||||
|
||||
The runtime's primary roles to manage scheduling and updating of parachains and parathreads, as well as handling misbehavior reports and slashing. This guide doesn't focus on how parachains or parathreads are registered, only that they are. Also, this runtime description assumes that validator sets are selected somehow, but doesn't assume any other details than a periodic _session change_ event. Session changes give information about the incoming validator set and the validator set of the following session.
|
||||
|
||||
The runtime also serves another role, which is to make data available to the Node-side logic via Runtime APIs. These Runtime APIs should be sufficient for the Node-side code to author blocks correctly.
|
||||
|
||||
There is some functionality of the relay chain relating to parachains that we also consider beyond the scope of this document. In particular, all modules related to how parachains are registered aren't part of this guide, although we do provide routines that should be called by the registration process.
|
||||
|
||||
We will split the logic of the runtime up into these modules:
|
||||
|
||||
* Initializer: manage initialization order of the other modules.
|
||||
* Configuration: manage configuration and configuration updates in a non-racy manner.
|
||||
* Paras: manage chain-head and validation code for parachains and parathreads.
|
||||
* Scheduler: manages parachain and parathread scheduling as well as validator assignments.
|
||||
* Inclusion: handles the inclusion and availability of scheduled parachains and parathreads.
|
||||
* Validity: handles secondary checks and dispute resolution for included, available parablocks.
|
||||
|
||||
The [Initializer module](initializer.md) is special - it's responsible for handling the initialization logic of the other modules to ensure that the correct initialization order and related invariants are maintained. The other modules won't specify a on-initialize logic, but will instead expose a special semi-private routine that the initialization module will call. The other modules are relatively straightforward and perform the roles described above.
|
||||
|
||||
The Parachain Host operates under a changing set of validators. Time is split up into periodic sessions, where each session brings a potentially new set of validators. Sessions are buffered by one, meaning that the validators of the upcoming session are fixed and always known. Parachain Host runtime modules need to react to changes in the validator set, as it will affect the runtime logic for processing candidate backing, availability bitfields, and misbehavior reports. The Parachain Host modules can't determine ahead-of-time exactly when session change notifications are going to happen within the block (note: this depends on module initialization order again - better to put session before parachains modules). Ideally, session changes are always handled before initialization. It is clearly a problem if we compute validator assignments to parachains during initialization and then the set of validators changes. In the best case, we can recognize that re-initialization needs to be done. In the worst case, bugs would occur.
|
||||
|
||||
There are 3 main ways that we can handle this issue:
|
||||
|
||||
1. Establish an invariant that session change notifications always happen after initialization. This means that when we receive a session change notification before initialization, we call the initialization routines before handling the session change.
|
||||
1. Require that session change notifications always occur before initialization. Brick the chain if session change notifications ever happen after initialization.
|
||||
1. Handle both the before and after cases.
|
||||
|
||||
Although option 3 is the most comprehensive, it runs counter to our goal of simplicity. Option 1 means requiring the runtime to do redundant work at all sessions and will also mean, like option 3, that designing things in such a way that initialization can be rolled back and reapplied under the new environment. That leaves option 2, although it is a "nuclear" option in a way and requires us to constrain the parachain host to only run in full runtimes with a certain order of operations.
|
||||
|
||||
So the other role of the initializer module is to forward session change notifications to modules in the initialization order, throwing an unrecoverable error if the notification is received after initialization. Session change is the point at which the [Configuration Module](configuration.md) updates the configuration. Most of the other modules will handle changes in the configuration during their session change operation, so the initializer should provide both the old and new configuration to all the other
|
||||
modules alongside the session change notification. This means that a session change notification should consist of the following data:
|
||||
|
||||
```rust
|
||||
struct SessionChangeNotification {
|
||||
// The new validators in the session.
|
||||
validators: Vec<ValidatorId>,
|
||||
// The validators for the next session.
|
||||
queued: Vec<ValidatorId>,
|
||||
// The configuration before handling the session change.
|
||||
prev_config: HostConfiguration,
|
||||
// The configuration after handling the session change.
|
||||
new_config: HostConfiguration,
|
||||
// A secure randomn seed for the session, gathered from BABE.
|
||||
random_seed: [u8; 32],
|
||||
// The session index of the beginning session.
|
||||
session_index: SessionIndex,
|
||||
}
|
||||
```
|
||||
|
||||
> REVIEW: other options? arguments in favor of going for options 1 or 3 instead of 2. we could do a "soft" version of 2 where we note that the chain is potentially broken due to bad initialization order
|
||||
> TODO Diagram: order of runtime operations (initialization, session change)
|
||||
@@ -0,0 +1,42 @@
|
||||
# Configuration Module
|
||||
|
||||
This module is responsible for managing all configuration of the parachain host in-flight. It provides a central point for configuration updates to prevent races between configuration changes and parachain-processing logic. Configuration can only change during the session change routine, and as this module handles the session change notification first it provides an invariant that the configuration does not change throughout the entire session. Both the [scheduler](scheduler.md) and [inclusion](inclusion.md) modules rely on this invariant to ensure proper behavior of the scheduler.
|
||||
|
||||
The configuration that we will be tracking is the [`HostConfiguration`](../types/runtime.md#host-configuration) struct.
|
||||
|
||||
## Storage
|
||||
|
||||
The configuration module is responsible for two main pieces of storage.
|
||||
|
||||
```rust
|
||||
/// The current configuration to be used.
|
||||
Configuration: HostConfiguration;
|
||||
/// A pending configuration to be applied on session change.
|
||||
PendingConfiguration: Option<HostConfiguration>;
|
||||
```
|
||||
|
||||
## Session change
|
||||
|
||||
The session change routine for the Configuration module is simple. If the `PendingConfiguration` is `Some`, take its value and set `Configuration` to be equal to it. Reset `PendingConfiguration` to `None`.
|
||||
|
||||
## Routines
|
||||
|
||||
```rust
|
||||
/// Get the host configuration.
|
||||
pub fn configuration() -> HostConfiguration {
|
||||
Configuration::get()
|
||||
}
|
||||
|
||||
/// Updating the pending configuration to be applied later.
|
||||
fn update_configuration(f: impl FnOnce(&mut HostConfiguration)) {
|
||||
PendingConfiguration::mutate(|pending| {
|
||||
let mut x = pending.unwrap_or_else(Self::configuration);
|
||||
f(&mut x);
|
||||
*pending = Some(x);
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Entry-points
|
||||
|
||||
The Configuration module exposes an entry point for each configuration member. These entry-points accept calls only from governance origins. These entry-points will use the `update_configuration` routine to update the specific configuration field.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Inclusion Module
|
||||
|
||||
The inclusion module is responsible for inclusion and availability of scheduled parachains and parathreads.
|
||||
|
||||
## Storage
|
||||
|
||||
Helper structs:
|
||||
|
||||
```rust
|
||||
struct AvailabilityBitfield {
|
||||
bitfield: BitVec, // one bit per core.
|
||||
submitted_at: BlockNumber, // for accounting, as meaning of bits may change over time.
|
||||
}
|
||||
|
||||
struct CandidatePendingAvailability {
|
||||
core: CoreIndex, // availability core
|
||||
receipt: CandidateReceipt,
|
||||
availability_votes: Bitfield, // one bit per validator.
|
||||
relay_parent_number: BlockNumber, // number of the relay-parent.
|
||||
backed_in_number: BlockNumber,
|
||||
}
|
||||
```
|
||||
|
||||
Storage Layout:
|
||||
|
||||
```rust
|
||||
/// The latest bitfield for each validator, referred to by index.
|
||||
bitfields: map ValidatorIndex => AvailabilityBitfield;
|
||||
/// Candidates pending availability.
|
||||
PendingAvailability: map ParaId => CandidatePendingAvailability;
|
||||
/// The commitments of candidates pending availability, by ParaId.
|
||||
PendingAvailabilityCommitments: map ParaId => CandidateCommitments;
|
||||
|
||||
/// The current validators, by their parachain session keys.
|
||||
Validators: Vec<ValidatorId>;
|
||||
|
||||
/// The current session index.
|
||||
CurrentSessionIndex: SessionIndex;
|
||||
```
|
||||
|
||||
## Session Change
|
||||
|
||||
1. Clear out all candidates pending availability.
|
||||
1. Clear out all validator bitfields.
|
||||
1. Update `Validators` with the validators from the session change notification.
|
||||
1. Update `CurrentSessionIndex` with the session index from the session change notification.
|
||||
|
||||
## Routines
|
||||
|
||||
All failed checks should lead to an unrecoverable error making the block invalid.
|
||||
|
||||
* `process_bitfields(Bitfields, core_lookup: Fn(CoreIndex) -> Option<ParaId>)`:
|
||||
1. check that the number of bitfields and bits in each bitfield is correct.
|
||||
1. check that there are no duplicates
|
||||
1. check all validator signatures.
|
||||
1. apply each bit of bitfield to the corresponding pending candidate. looking up parathread cores using the `core_lookup`. Disregard bitfields that have a `1` bit for any free cores.
|
||||
1. For each applied bit of each availability-bitfield, set the bit for the validator in the `CandidatePendingAvailability`'s `availability_votes` bitfield. Track all candidates that now have >2/3 of bits set in their `availability_votes`. These candidates are now available and can be enacted.
|
||||
1. For all now-available candidates, invoke the `enact_candidate` routine with the candidate and relay-parent number.
|
||||
1. > TODO: pass it onwards to `Validity` module.
|
||||
1. Return a list of freed cores consisting of the cores where candidates have become available.
|
||||
* `process_candidates(BackedCandidates, scheduled: Vec<CoreAssignment>, group_validators: Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>)`:
|
||||
1. check that each candidate corresponds to a scheduled core and that they are ordered in the same order the cores appear in assignments in `scheduled`.
|
||||
1. check that `scheduled` is sorted ascending by `CoreIndex`, without duplicates.
|
||||
1. check that there is no candidate pending availability for any scheduled `ParaId`.
|
||||
1. If the core assignment includes a specific collator, ensure the backed candidate is issued by that collator.
|
||||
1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_frequency` of `Paras::last_code_upgrade(para_id, true)`, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID.
|
||||
1. Check the collator's signature on the candidate data.
|
||||
1. Transform each [`CommittedCandidateReceipt`](../types/candidate.md#committed-candidate-receipt) into the corresponding [`CandidateReceipt`](../types/candidate.md#candidate-receipt), setting the commitments aside.
|
||||
1. check the backing of the candidate using the signatures and the bitfields, comparing against the validators assigned to the groups, fetched with the `group_validators` lookup.
|
||||
1. check that the upward messages, when combined with the existing queue size, are not exceeding `config.max_upward_queue_count` and `config.watermark_upward_queue_size` parameters.
|
||||
1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield.
|
||||
1. create a corresponding entry in the `PendingAvailabilityCommitments` with the commitments.
|
||||
1. Return a `Vec<CoreIndex>` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex.
|
||||
* `enact_candidate(relay_parent_number: BlockNumber, CommittedCandidateReceipt)`:
|
||||
1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number + config.validationl_upgrade_delay)`.
|
||||
> TODO: Note that this is safe as long as we never enact candidates where the relay parent is across a session boundary. In that case, which we should be careful to avoid with contextual execution, the configuration might have changed and the para may de-sync from the host's understanding of it.
|
||||
1. call `Router::queue_upward_messages` for each backed candidate, using the [`UpwardMessage`s](../types/messages.md#upward-message) from the [`CandidateCommitments`](../types/candidate.md#candidate-commitments).
|
||||
1. Call `Paras::note_new_head` using the `HeadData` from the receipt and `relay_parent_number`.
|
||||
* `collect_pending`:
|
||||
|
||||
```rust
|
||||
fn collect_pending(f: impl Fn(CoreIndex, BlockNumber) -> bool) -> Vec<u32> {
|
||||
// sweep through all paras pending availability. if the predicate returns true, when given the core index and
|
||||
// the block number the candidate has been pending availability since, then clean up the corresponding storage for that candidate.
|
||||
// return a vector of cleaned-up core IDs.
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,25 @@
|
||||
# InclusionInherent
|
||||
|
||||
This module is responsible for all the logic carried by the `Inclusion` entry-point. This entry-point is mandatory, in that it must be invoked exactly once within every block, and it is also "inherent", in that it is provided with no origin by the block author. The data within it carries its own authentication. If any of the steps within fails, the entry-point is considered as having failed and the block will be invalid.
|
||||
|
||||
This module does not have the same initialization/finalization concerns as the others, as it only requires that entry points be triggered after all modules have initialized and that finalization happens after entry points are triggered. Both of these are assumptions we have already made about the runtime's order of operations, so this module doesn't need to be initialized or finalized by the `Initializer`.
|
||||
|
||||
## Storage
|
||||
|
||||
```rust
|
||||
Included: Option<()>,
|
||||
```
|
||||
|
||||
## Finalization
|
||||
|
||||
1. Take (get and clear) the value of `Included`. If it is not `Some`, throw an unrecoverable error.
|
||||
|
||||
## Entry Points
|
||||
|
||||
* `inclusion`: This entry-point accepts two parameters: [`Bitfields`](../types/availability.md#signed-availability-bitfield) and [`BackedCandidates`](../types/backing.md#backed-candidate).
|
||||
1. The `Bitfields` are first forwarded to the `Inclusion::process_bitfields` routine, returning a set of freed cores. Provide a `Scheduler::core_para` as a core-lookup to the `process_bitfields` routine. Annotate each of these freed cores with `FreedReason::Concluded`.
|
||||
1. If `Scheduler::availability_timeout_predicate` is `Some`, invoke `Inclusion::collect_pending` using it, and add timed-out cores to the free cores, annotated with `FreedReason::TimedOut`.
|
||||
1. Invoke `Scheduler::schedule(freed)`
|
||||
1. Invoke the `Inclusion::process_candidates` routine with the parameters `(backed_candidates, Scheduler::scheduled(), Scheduler::group_validators)`.
|
||||
1. Call `Scheduler::occupied` using the return value of the `Inclusion::process_candidates` call above, first sorting the list of assigned core indices.
|
||||
1. If all of the above succeeds, set `Included` to `Some(())`.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Initializer Module
|
||||
|
||||
This module is responsible for initializing the other modules in a deterministic order. It also has one other purpose as described above: accepting and forwarding session change notifications.
|
||||
|
||||
## Storage
|
||||
|
||||
```rust
|
||||
HasInitialized: bool
|
||||
```
|
||||
|
||||
## Initialization
|
||||
|
||||
The other modules are initialized in this order:
|
||||
|
||||
1. Configuration
|
||||
1. Paras
|
||||
1. Scheduler
|
||||
1. Inclusion
|
||||
1. Validity.
|
||||
1. Router.
|
||||
|
||||
The [Configuration Module](configuration.md) is first, since all other modules need to operate under the same configuration as each other. It would lead to inconsistency if, for example, the scheduler ran first and then the configuration was updated before the Inclusion module.
|
||||
|
||||
Set `HasInitialized` to true.
|
||||
|
||||
## Session Change
|
||||
|
||||
If `HasInitialized` is true, throw an unrecoverable error (panic).
|
||||
Otherwise, forward the session change notification to other modules in initialization order.
|
||||
|
||||
## Finalization
|
||||
|
||||
Finalization order is less important in this case than initialization order, so we finalize the modules in the reverse order from initialization.
|
||||
|
||||
Set `HasInitialized` to false.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Paras Module
|
||||
|
||||
The Paras module is responsible for storing information on parachains and parathreads. Registered parachains and parathreads cannot change except at session boundaries. This is primarily to ensure that the number of bits required for the availability bitfields does not change except at session boundaries.
|
||||
|
||||
It's also responsible for managing parachain validation code upgrades as well as maintaining availability of old parachain code and its pruning.
|
||||
|
||||
## Storage
|
||||
|
||||
Utility structs:
|
||||
|
||||
```rust
|
||||
// the two key times necessary to track for every code replacement.
|
||||
pub struct ReplacementTimes {
|
||||
/// The relay-chain block number that the code upgrade was expected to be activated.
|
||||
/// This is when the code change occurs from the para's perspective - after the
|
||||
/// first parablock included with a relay-parent with number >= this value.
|
||||
expected_at: BlockNumber,
|
||||
/// The relay-chain block number at which the parablock activating the code upgrade was
|
||||
/// actually included. This means considered included and available, so this is the time at which
|
||||
/// that parablock enters the acceptance period in this fork of the relay-chain.
|
||||
activated_at: BlockNumber,
|
||||
}
|
||||
|
||||
/// Metadata used to track previous parachain validation code that we keep in
|
||||
/// the state.
|
||||
pub struct ParaPastCodeMeta {
|
||||
// Block numbers where the code was expected to be replaced and where the code
|
||||
// was actually replaced, respectively. The first is used to do accurate lookups
|
||||
// of historic code in historic contexts, whereas the second is used to do
|
||||
// pruning on an accurate timeframe. These can be used as indices
|
||||
// into the `PastCode` map along with the `ParaId` to fetch the code itself.
|
||||
upgrade_times: Vec<ReplacementTimes>,
|
||||
// This tracks the highest pruned code-replacement, if any.
|
||||
last_pruned: Option<BlockNumber>,
|
||||
}
|
||||
|
||||
enum UseCodeAt {
|
||||
// Use the current code.
|
||||
Current,
|
||||
// Use the code that was replaced at the given block number.
|
||||
ReplacedAt(BlockNumber),
|
||||
}
|
||||
|
||||
struct ParaGenesisArgs {
|
||||
/// The initial head-data to use.
|
||||
genesis_head: HeadData,
|
||||
/// The validation code to start with.
|
||||
validation_code: ValidationCode,
|
||||
/// True if parachain, false if parathread.
|
||||
parachain: bool,
|
||||
}
|
||||
```
|
||||
|
||||
Storage layout:
|
||||
|
||||
```rust
|
||||
/// All parachains. Ordered ascending by ParaId. Parathreads are not included.
|
||||
Parachains: Vec<ParaId>,
|
||||
/// All parathreads.
|
||||
Parathreads: map ParaId => Option<()>,
|
||||
/// The head-data of every registered para.
|
||||
Heads: map ParaId => Option<HeadData>;
|
||||
/// The validation code of every live para.
|
||||
ValidationCode: map ParaId => Option<ValidationCode>;
|
||||
/// Actual past code, indicated by the para id as well as the block number at which it became outdated.
|
||||
PastCode: map (ParaId, BlockNumber) => Option<ValidationCode>;
|
||||
/// Past code of parachains. The parachains themselves may not be registered anymore,
|
||||
/// but we also keep their code on-chain for the same amount of time as outdated code
|
||||
/// to keep it available for secondary checkers.
|
||||
PastCodeMeta: map ParaId => ParaPastCodeMeta;
|
||||
/// Which paras have past code that needs pruning and the relay-chain block at which the code was replaced.
|
||||
/// Note that this is the actual height of the included block, not the expected height at which the
|
||||
/// code upgrade would be applied, although they may be equal.
|
||||
/// This is to ensure the entire acceptance period is covered, not an offset acceptance period starting
|
||||
/// from the time at which the parachain perceives a code upgrade as having occurred.
|
||||
/// Multiple entries for a single para are permitted. Ordered ascending by block number.
|
||||
PastCodePruning: Vec<(ParaId, BlockNumber)>;
|
||||
/// The block number at which the planned code change is expected for a para.
|
||||
/// The change will be applied after the first parablock for this ID included which executes
|
||||
/// in the context of a relay chain block with a number >= `expected_at`.
|
||||
FutureCodeUpgrades: map ParaId => Option<BlockNumber>;
|
||||
/// The actual future code of a para.
|
||||
FutureCode: map ParaId => Option<ValidationCode>;
|
||||
|
||||
/// Upcoming paras (chains and threads). These are only updated on session change. Corresponds to an
|
||||
/// entry in the upcoming-genesis map.
|
||||
UpcomingParas: Vec<ParaId>;
|
||||
/// Upcoming paras instantiation arguments.
|
||||
UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
|
||||
/// Paras that are to be cleaned up at the end of the session.
|
||||
OutgoingParas: Vec<ParaId>;
|
||||
```
|
||||
|
||||
## Session Change
|
||||
|
||||
1. Clean up outgoing paras. This means removing the entries under `Heads`, `ValidationCode`, `FutureCodeUpgrades`, and `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and `PastCodePruning` using the outgoing `ParaId` and removed `ValidationCode` value. This is because any outdated validation code must remain available on-chain for a determined amount of blocks, and validation code outdated by de-registering the para is still subject to that invariant.
|
||||
1. Apply all incoming paras by initializing the `Heads` and `ValidationCode` using the genesis parameters.
|
||||
1. Amend the `Parachains` list to reflect changes in registered parachains.
|
||||
1. Amend the `Parathreads` set to reflect changes in registered parathreads.
|
||||
|
||||
## Initialization
|
||||
|
||||
1. Do pruning based on all entries in `PastCodePruning` with `BlockNumber <= now`. Update the corresponding `PastCodeMeta` and `PastCode` accordingly.
|
||||
|
||||
## Routines
|
||||
|
||||
* `schedule_para_initialize(ParaId, ParaGenesisArgs)`: schedule a para to be initialized at the next session.
|
||||
* `schedule_para_cleanup(ParaId)`: schedule a para to be cleaned up at the next session.
|
||||
* `schedule_code_upgrade(ParaId, ValidationCode, expected_at: BlockNumber)`: Schedule a future code upgrade of the given parachain, to be applied after inclusion of a block of the same parachain executed in the context of a relay-chain block with number >= `expected_at`.
|
||||
* `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head, where the new head was executed in the context of a relay-chain block with given number. This will apply pending code upgrades based on the block number provided.
|
||||
* `validation_code_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Fetches the validation code to be used when validating a block in the context of the given relay-chain height. A second block number parameter may be used to tell the lookup to proceed as if an intermediate parablock has been included at the given relay-chain height. This may return past, current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if provided, must be before `at`. If the validation code has been pruned, this will return `None`.
|
||||
* `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread.
|
||||
|
||||
* `last_code_upgrade(id: ParaId, include_future: bool) -> Option<BlockNumber>`: The block number of the last scheduled upgrade of the requested para. Includes future upgrades if the flag is set. This is the `expected_at` number, not the `activated_at` number.
|
||||
|
||||
## Finalization
|
||||
|
||||
No finalization routine runs for this module.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Router Module
|
||||
|
||||
The Router module is responsible for storing and dispatching Upward and Downward messages from and to parachains respectively. It is intended to later handle the XCMP logic as well.
|
||||
|
||||
For each enacted block the `queue_upward_messages` entry-point is called.
|
||||
|
||||
## Storage
|
||||
|
||||
Storage layout:
|
||||
|
||||
```rust,ignore
|
||||
/// Messages ready to be dispatched onto the relay chain.
|
||||
/// This is subject to `max_upward_queue_count` and
|
||||
/// `watermark_queue_size` from `HostConfiguration`.
|
||||
RelayDispatchQueues: map ParaId => Vec<UpwardMessage>;
|
||||
/// Size of the dispatch queues. Caches sizes of the queues in `RelayDispatchQueue`.
|
||||
/// First item in the tuple is the count of messages and second
|
||||
/// is the total length (in bytes) of the message payloads.
|
||||
RelayDispatchQueueSize: map ParaId => (u32, u32);
|
||||
/// The ordered list of `ParaId`s that have a `RelayDispatchQueue` entry.
|
||||
NeedsDispatch: Vec<ParaId>;
|
||||
```
|
||||
|
||||
## Initialization
|
||||
|
||||
No initialization routine runs for this module.
|
||||
|
||||
## Routines
|
||||
|
||||
* `queue_upward_messages(ParaId, Vec<UpwardMessage>)`:
|
||||
1. Updates `NeedsDispatch`, and enqueues upward messages into `RelayDispatchQueue` and modifies the respective entry in `RelayDispatchQueueSize`.
|
||||
|
||||
## Finalization
|
||||
|
||||
1. Dispatch queued upward messages from `RelayDispatchQueues` in a FIFO order applying the `config.watermark_upward_queue_size` and `config.max_upward_queue_count` limits.
|
||||
@@ -0,0 +1,201 @@
|
||||
# Scheduler Module
|
||||
|
||||
> TODO: this section is still heavily under construction. key questions about availability cores and validator assignment are still open and the flow of the the section may be contradictory or inconsistent
|
||||
|
||||
The Scheduler module is responsible for two main tasks:
|
||||
|
||||
- Partitioning validators into groups and assigning groups to parachains and parathreads.
|
||||
- Scheduling parachains and parathreads
|
||||
|
||||
It aims to achieve these tasks with these goals in mind:
|
||||
|
||||
- It should be possible to know at least a block ahead-of-time, ideally more, which validators are going to be assigned to which parachains.
|
||||
- Parachains that have a candidate pending availability in this fork of the chain should not be assigned.
|
||||
- Validator assignments should not be gameable. Malicious cartels should not be able to manipulate the scheduler to assign themselves as desired.
|
||||
- High or close to optimal throughput of parachains and parathreads. Work among validator groups should be balanced.
|
||||
|
||||
The Scheduler manages resource allocation using the concept of "Availability Cores". There will be one availability core for each parachain, and a fixed number of cores used for multiplexing parathreads. Validators will be partitioned into groups, with the same number of groups as availability cores. Validator groups will be assigned to different availability cores over time.
|
||||
|
||||
An availability core can exist in either one of two states at the beginning or end of a block: free or occupied. A free availability core can have a parachain or parathread assigned to it for the potential to have a backed candidate included. After inclusion, the core enters the occupied state as the backed candidate is pending availability. There is an important distinction: a core is not considered occupied until it is in charge of a block pending availability, although the implementation may treat scheduled cores the same as occupied ones for brevity. A core exits the occupied state when the candidate is no longer pending availability - either on timeout or on availability. A core starting in the occupied state can move to the free state and back to occupied all within a single block, as availability bitfields are processed before backed candidates. At the end of the block, there is a possible timeout on availability which can move the core back to the free state if occupied.
|
||||
|
||||
```text
|
||||
Availability Core State Machine
|
||||
|
||||
Assignment &
|
||||
Backing
|
||||
+-----------+ +-----------+
|
||||
| +--------------> |
|
||||
| Free | | Occupied |
|
||||
| <--------------+ |
|
||||
+-----------+ Availability +-----------+
|
||||
or Timeout
|
||||
|
||||
```
|
||||
|
||||
```text
|
||||
Availability Core Transitions within Block
|
||||
|
||||
+-----------+ | +-----------+
|
||||
| | | | |
|
||||
| Free | | | Occupied |
|
||||
| | | | |
|
||||
+--/-----\--+ | +--/-----\--+
|
||||
/- -\ | /- -\
|
||||
No Backing /- \ Backing | Availability /- \ No availability
|
||||
/- \ | / \
|
||||
/- -\ | /- -\
|
||||
+-----v-----+ +----v------+ | +-----v-----+ +-----v-----+
|
||||
| | | | | | | | |
|
||||
| Free | | Occupied | | | Free | | Occupied |
|
||||
| | | | | | | | |
|
||||
+-----------+ +-----------+ | +-----|---\-+ +-----|-----+
|
||||
| | \ |
|
||||
| No backing | \ Backing | (no change)
|
||||
| | -\ |
|
||||
| +-----v-----+ \ +-----v-----+
|
||||
| | | \ | |
|
||||
| | Free -----+---> Occupied |
|
||||
| | | | |
|
||||
| +-----------+ +-----------+
|
||||
| Availability Timeout
|
||||
```
|
||||
|
||||
Validator group assignments do not need to change very quickly. The security benefits of fast rotation is redundant with the challenge mechanism in the [Validity module](validity.md). Because of this, we only divide validators into groups at the beginning of the session and do not shuffle membership during the session. However, we do take steps to ensure that no particular validator group has dominance over a single parachain or parathread-multiplexer for an entire session to provide better guarantees of liveness.
|
||||
|
||||
Validator groups rotate across availability cores in a round-robin fashion, with rotation occurring at fixed intervals. The i'th group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of rotations that have occurred in the session, and `n` is the number of cores. This makes upcoming rotations within the same session predictable.
|
||||
|
||||
When a rotation occurs, validator groups are still responsible for distributing availability chunks for any previous cores that are still occupied and pending availability. In practice, rotation and availability-timeout frequencies should be set so this will only be the core they have just been rotated from. It is possible that a validator group is rotated onto a core which is currently occupied. In this case, the validator group will have nothing to do until the previously-assigned group finishes their availability work and frees the core or the availability process times out. Depending on if the core is for a parachain or parathread, a different timeout `t` from the [`HostConfiguration`](../types/runtime.md#host-configuration) will apply. Availability timeouts should only be triggered in the first `t-1` blocks after the beginning of a rotation.
|
||||
|
||||
Parathreads operate on a system of claims. Collators participate in auctions to stake a claim on authoring the next block of a parathread, although the auction mechanism is beyond the scope of the scheduler. The scheduler guarantees that they'll be given at least a certain number of attempts to author a candidate that is backed. Attempts that fail during the availability phase are not counted, since ensuring availability at that stage is the responsibility of the backing validators, not of the collator. When a claim is accepted, it is placed into a queue of claims, and each claim is assigned to a particular parathread-multiplexing core in advance. Given that the current assignments of validator groups to cores are known, and the upcoming assignments are predictable, it is possible for parathread collators to know who they should be talking to now and how they should begin establishing connections with as a fallback.
|
||||
|
||||
With this information, the Node-side can be aware of which parathreads have a good chance of being includable within the relay-chain block and can focus any additional resources on backing candidates from those parathreads. Furthermore, Node-side code is aware of which validator group will be responsible for that thread. If the necessary conditions are reached for core reassignment, those candidates can be backed within the same block as the core being freed.
|
||||
|
||||
Parathread claims, when scheduled onto a free core, may not result in a block pending availability. This may be due to collator error, networking timeout, or censorship by the validator group. In this case, the claims should be retried a certain number of times to give the collator a fair shot.
|
||||
|
||||
Cores are treated as an ordered list of cores and are typically referred to by their index in that list.
|
||||
|
||||
## Storage
|
||||
|
||||
Utility structs:
|
||||
|
||||
```rust
|
||||
// A claim on authoring the next block for a given parathread.
|
||||
struct ParathreadClaim(ParaId, CollatorId);
|
||||
|
||||
// An entry tracking a claim to ensure it does not pass the maximum number of retries.
|
||||
struct ParathreadEntry {
|
||||
claim: ParathreadClaim,
|
||||
retries: u32,
|
||||
}
|
||||
|
||||
// A queued parathread entry, pre-assigned to a core.
|
||||
struct QueuedParathread {
|
||||
claim: ParathreadEntry,
|
||||
/// offset within the set of para-threads ranged `0..config.parathread_cores`.
|
||||
core_offset: u32,
|
||||
}
|
||||
|
||||
struct ParathreadQueue {
|
||||
queue: Vec<QueuedParathread>,
|
||||
/// offset within the set of para-threads ranged `0..config.parathread_cores`.
|
||||
next_core_offset: u32,
|
||||
}
|
||||
|
||||
enum CoreOccupied {
|
||||
Parathread(ParathreadEntry), // claim & retries
|
||||
Parachain,
|
||||
}
|
||||
|
||||
enum AssignmentKind {
|
||||
Parachain,
|
||||
Parathread(CollatorId, u32),
|
||||
}
|
||||
|
||||
struct CoreAssignment {
|
||||
core: CoreIndex,
|
||||
para_id: ParaId,
|
||||
kind: AssignmentKind,
|
||||
group_idx: GroupIndex,
|
||||
}
|
||||
// reasons a core might be freed.
|
||||
enum FreedReason {
|
||||
Concluded,
|
||||
TimedOut,
|
||||
}
|
||||
```
|
||||
|
||||
Storage layout:
|
||||
|
||||
```rust
|
||||
/// All the validator groups. One for each core.
|
||||
ValidatorGroups: Vec<Vec<ValidatorIndex>>;
|
||||
/// A queue of upcoming claims and which core they should be mapped onto.
|
||||
ParathreadQueue: ParathreadQueue;
|
||||
/// One entry for each availability core. Entries are `None` if the core is not currently occupied. Can be
|
||||
/// temporarily `Some` if scheduled but not occupied.
|
||||
/// The i'th parachain belongs to the i'th core, with the remaining cores all being
|
||||
/// parathread-multiplexers.
|
||||
AvailabilityCores: Vec<Option<CoreOccupied>>;
|
||||
/// An index used to ensure that only one claim on a parathread exists in the queue or is
|
||||
/// currently being handled by an occupied core.
|
||||
ParathreadClaimIndex: Vec<ParaId>;
|
||||
/// The block number where the session start occurred. Used to track how many group rotations have occurred.
|
||||
SessionStartBlock: BlockNumber;
|
||||
/// Currently scheduled cores - free but up to be occupied. Ephemeral storage item that's wiped on finalization.
|
||||
Scheduled: Vec<CoreAssignment>, // sorted ascending by CoreIndex.
|
||||
```
|
||||
|
||||
## Session Change
|
||||
|
||||
Session changes are the only time that configuration can change, and the [Configuration module](configuration.md)'s session-change logic is handled before this module's. We also lean on the behavior of the [Inclusion module](inclusion.md) which clears all its occupied cores on session change. Thus we don't have to worry about cores being occupied across session boundaries and it is safe to re-size the `AvailabilityCores` bitfield.
|
||||
|
||||
Actions:
|
||||
|
||||
1. Set `SessionStartBlock` to current block number.
|
||||
1. Clear all `Some` members of `AvailabilityCores`. Return all parathread claims to queue with retries un-incremented.
|
||||
1. Set `configuration = Configuration::configuration()` (see [`HostConfiguration`](../types/runtime.md#host-configuration))
|
||||
1. Resize `AvailabilityCores` to have length `Paras::parachains().len() + configuration.parathread_cores with all`None` entries.
|
||||
1. Compute new validator groups by shuffling using a secure randomness beacon
|
||||
- We need a total of `N = Paras::parathreads().len() + configuration.parathread_cores` validator groups.
|
||||
- The total number of validators `V` in the `SessionChangeNotification`'s `validators` may not be evenly divided by `V`.
|
||||
- First, we obtain "shuffled validators" `SV` by shuffling the validators using the `SessionChangeNotification`'s random seed.
|
||||
- The groups are selected by partitioning `SV`. The first V % N groups will have (V / N) + 1 members, while the remaining groups will have (V / N) members each.
|
||||
1. Prune the parathread queue to remove all retries beyond `configuration.parathread_retries`.
|
||||
- Also prune all parathread claims corresponding to de-registered parathreads.
|
||||
- all pruned claims should have their entry removed from the parathread index.
|
||||
- assign all non-pruned claims to new cores if the number of parathread cores has changed between the `new_config` and `old_config` of the `SessionChangeNotification`.
|
||||
- Assign claims in equal balance across all cores if rebalancing, and set the `next_core` of the `ParathreadQueue` by incrementing the relative index of the last assigned core and taking it modulo the number of parathread cores.
|
||||
|
||||
## Initialization
|
||||
|
||||
1. Schedule free cores using the `schedule(Vec::new())`.
|
||||
|
||||
## Finalization
|
||||
|
||||
Actions:
|
||||
|
||||
1. Free all scheduled cores and return parathread claims to queue, with retries incremented.
|
||||
|
||||
## Routines
|
||||
|
||||
- `add_parathread_claim(ParathreadClaim)`: Add a parathread claim to the queue.
|
||||
- Fails if any parathread claim on the same parathread is currently indexed.
|
||||
- Fails if the queue length is >= `config.scheduling_lookahead * config.parathread_cores`.
|
||||
- The core used for the parathread claim is the `next_core` field of the `ParathreadQueue` and adding `Paras::parachains().len()` to it.
|
||||
- `next_core` is then updated by adding 1 and taking it modulo `config.parathread_cores`.
|
||||
- The claim is then added to the claim index.
|
||||
- `schedule(Vec<(CoreIndex, FreedReason)>)`: schedule new core assignments, with a parameter indicating previously-occupied cores which are to be considered returned and why they are being returned.
|
||||
- All freed parachain cores should be assigned to their respective parachain
|
||||
- All freed parathread cores whose reason for freeing was `FreedReason::Concluded` should have the claim removed from the claim index.
|
||||
- All freed parathread cores whose reason for freeing was `FreedReason::TimedOut` should have the claim added to the parathread queue again without retries incremented
|
||||
- All freed parathread cores should take the next parathread entry from the queue.
|
||||
- The i'th validator group will be assigned to the `(i+k)%n`'th core at any point in time, where `k` is the number of rotations that have occurred in the session, and `n` is the total number of cores. This makes upcoming rotations within the same session predictable.
|
||||
- `scheduled() -> Vec<CoreAssignment>`: Get currently scheduled core assignments.
|
||||
- `occupied(Vec<CoreIndex>)`. Note that the given cores have become occupied.
|
||||
- Behavior undefined if any given cores were not scheduled.
|
||||
- Behavior undefined if the given cores are not sorted ascending by core index
|
||||
- This clears them from `Scheduled` and marks each corresponding `core` in the `AvailabilityCores` as occupied.
|
||||
- Since both the availability cores and the newly-occupied cores lists are sorted ascending, this method can be implemented efficiently.
|
||||
- `core_para(CoreIndex) -> ParaId`: return the currently-scheduled or occupied ParaId for the given core.
|
||||
- `group_validators(GroupIndex) -> Option<Vec<ValidatorIndex>>`: return all validators in a given group, if the group index is valid for this session.
|
||||
- `availability_timeout_predicate() -> Option<impl Fn(CoreIndex, BlockNumber) -> bool>`: returns an optional predicate that should be used for timing out occupied cores. if `None`, no timing-out should be done. The predicate accepts the index of the core, and the block number since which it has been occupied. The predicate should be implemented based on the time since the last validator group rotation, and the respective parachain and parathread timeouts, i.e. only within `max(config.chain_availability_period, config.thread_availability_period)` of the last rotation would this return `Some`.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Validity Module
|
||||
|
||||
After a backed candidate is made available, it is included and proceeds into an acceptance period during which validators are randomly selected to do (secondary) approval checks of the parablock. Any reports disputing the validity of the candidate will cause escalation, where even more validators are requested to check the block, and so on, until either the parablock is determined to be invalid or valid. Those on the wrong side of the dispute are slashed and, if the parablock is deemed invalid, the relay chain is rolled back to a point before that block was included.
|
||||
|
||||
However, this isn't the end of the story. We are working in a forkful blockchain environment, which carries three important considerations:
|
||||
|
||||
1. For security, validators that misbehave shouldn't only be slashed on one fork, but on all possible forks. Validators that misbehave shouldn't be able to create a new fork of the chain when caught and get away with their misbehavior.
|
||||
1. It is possible that the parablock being contested has not appeared on all forks.
|
||||
1. If a block author believes that there is a disputed parablock on a specific fork that will resolve to a reversion of the fork, that block author is better incentivized to build on a different fork which does not include that parablock.
|
||||
|
||||
This means that in all likelihood, there is the possibility of disputes that are started on one fork of the relay chain, and as soon as the dispute resolution process starts to indicate that the parablock is indeed invalid, that fork of the relay chain will be abandoned and the dispute will never be fully resolved on that chain.
|
||||
|
||||
Even if this doesn't happen, there is the possibility that there are two disputes underway, and one resolves leading to a reversion of the chain before the other has concluded. In this case we want to both transplant the concluded dispute onto other forks of the chain as well as the unconcluded dispute.
|
||||
|
||||
We account for these requirements by having the validity module handle two kinds of disputes.
|
||||
|
||||
1. Local disputes: those contesting the validity of the current fork by disputing a parablock included within it.
|
||||
1. Remote disputes: a dispute that has partially or fully resolved on another fork which is transplanted to the local fork for completion and eventual slashing.
|
||||
|
||||
## Local Disputes
|
||||
|
||||
> TODO: store all included candidate and attestations on them here. accept additional backing after the fact. accept reports based on VRF. candidate included in session S should only be reported on by validator keys from session S. trigger slashing. probably only slash for session S even if the report was submitted in session S+k because it is hard to unify identity
|
||||
|
||||
One first question is to ask why different logic for local disputes is necessary. It seems that local disputes are necessary in order to create the first escalation that leads to block producers abandoning the chain and making remote disputes possible.
|
||||
|
||||
Local disputes are only allowed on parablocks that have been included on the local chain and are in the acceptance period.
|
||||
|
||||
For each such parablock, it is guaranteed by the inclusion pipeline that the parablock is available and the relevant validation code is available.
|
||||
|
||||
Disputes may occur against blocks that have happened in the session prior to the current one, from the perspective of the chain. In this case, the prior validator set is responsible for handling the dispute and to do so with their keys from the last session. This means that validator duty actually extends 1 session beyond leaving the validator set.
|
||||
|
||||
Validators self-select based on the BABE VRF output included by the block author in the block that the candidate became available.
|
||||
|
||||
> TODO: some more details from Jeff's paper.
|
||||
|
||||
After enough validators have self-selected, the quorum will be clear and validators on the wrong side will be slashed. After concluding, the dispute will remain open for some time in order to collect further evidence of misbehaving validators, and then issue a signal in the header-chain that this fork should be abandoned along with the hash of the last ancestor before inclusion, which the chain should be reverted to, along with information about the invalid block that should be used to blacklist it from being included.
|
||||
|
||||
## Remote Disputes
|
||||
|
||||
When a dispute has occurred on another fork, we need to transplant that dispute to every other fork. This poses some major challenges.
|
||||
|
||||
There are two types of remote disputes. The first is a remote roll-up of a concluded dispute. These are simply all attestations for the block, those against it, and the result of all (secondary) approval checks. A concluded remote dispute can be resolved in a single transaction as it is an open-and-shut case of a quorum of validators disagreeing with another.
|
||||
|
||||
The second type of remote dispute is the unconcluded dispute. An unconcluded remote dispute is started by any validator, using these things:
|
||||
|
||||
- A candidate
|
||||
- The session that the candidate has appeared in.
|
||||
- Backing for that candidate
|
||||
- The validation code necessary for validation of the candidate.
|
||||
> TODO: optimize by excluding in case where code appears in `Paras::CurrentCode` of this fork of relay-chain
|
||||
- Secondary checks already done on that candidate, containing one or more disputes by validators. None of the disputes are required to have appeared on other chains.
|
||||
> TODO: validator-dispute could be instead replaced by a fisherman w/ bond
|
||||
|
||||
When beginning a remote dispute, at least one escalation by a validator is required, but this validator may be malicious and desires to be slashed. There is no guarantee that the para is registered on this fork of the relay chain or that the para was considered available on any fork of the relay chain.
|
||||
|
||||
So the first step is to have the remote dispute proceed through an availability process similar to the one in the [Inclusion Module](inclusion.md), but without worrying about core assignments or compactness in bitfields.
|
||||
|
||||
We assume that remote disputes are with respect to the same validator set as on the current fork, as BABE and GRANDPA assure that forks are never long enough to diverge in validator set.
|
||||
> TODO: this is at least directionally correct. handling disputes on other validator sets seems useless anyway as they wouldn't be bonded.
|
||||
|
||||
As with local disputes, the validators of the session the candidate was included on another chain are responsible for resolving the dispute and determining availability of the candidate.
|
||||
|
||||
If the candidate was not made available on another fork of the relay chain, the availability process will time out and the disputing validator will be slashed on this fork. The escalation used by the validator(s) can be replayed onto other forks to lead the wrongly-escalating validator(s) to be slashed on all other forks as well. We assume that the adversary cannot censor validators from seeing any particular forks indefinitely
|
||||
|
||||
> TODO: set the availability timeout for this accordingly - unlike in the inclusion pipeline we are slashing for unavailability here!
|
||||
|
||||
If the availability process passes, the remote dispute is ready to be included on this chain. As with the local dispute, validators self-select based on a VRF. Given that a remote dispute is likely to be replayed across multiple forks, it is important to choose a VRF in a way that all forks processing the remote dispute will have the same one. Choosing the VRF is important as it should not allow an adversary to have control over who will be selected as a secondary approval checker.
|
||||
|
||||
After enough validator self-select, under the same escalation rules as for local disputes, the Remote dispute will conclude, slashing all those on the wrong side of the dispute. After concluding, the remote dispute remains open for a set amount of blocks to accept any further proof of additional validators being on the wrong side.
|
||||
|
||||
## Slashing and Incentivization
|
||||
|
||||
The goal of the dispute is to garner a `>2/3` (`2f + 1`) supermajority either in favor of or against the candidate.
|
||||
|
||||
For remote disputes, it is possible that the parablock disputed has never actually passed any availability process on any chain. In this case, validators will not be able to obtain the PoV of the parablock and there will be relatively few votes. We want to disincentivize voters claiming validity of the block from preventing it from becoming available, so we charge them a small distraction fee for wasting the others' time if the dispute does not garner a 2/3+ supermajority on either side. This fee can take the form of a small slash or a reduction in rewards.
|
||||
|
||||
When a supermajority is achieved for the dispute in either the valid or invalid direction, we will penalize non-voters either by issuing a small slash or reducing their rewards. We prevent censorship of the remaining validators by leaving the dispute open for some blocks after resolution in order to accept late votes.
|
||||
Reference in New Issue
Block a user