feat: initialize Kurdistan SDK - independent fork of Polkadot SDK
This commit is contained in:
@@ -0,0 +1,7 @@
|
||||
# Availability Subsystems
|
||||
|
||||
The availability subsystems are responsible for ensuring that Proofs of Validity of backed candidates are widely
|
||||
available within the validator set, without requiring every node to retain a full copy. They accomplish this by broadly
|
||||
distributing erasure-coded chunks of the PoV, keeping track of which validator has which chunk by means of signed
|
||||
bitfields. They are also responsible for reassembling a complete PoV when required, e.g. when an approval checker needs
|
||||
to validate a teyrchain block.
|
||||
@@ -0,0 +1,84 @@
|
||||
# Availability Distribution
|
||||
|
||||
This subsystem is responsible for distribution availability data to peers. Availability data are chunks, `PoV`s and
|
||||
`AvailableData` (which is `PoV` + `PersistedValidationData`). It does so via request response protocols.
|
||||
|
||||
In particular this subsystem is responsible for:
|
||||
|
||||
- Respond to network requests requesting availability data by querying the [Availability
|
||||
Store](../utility/availability-store.md).
|
||||
- Request chunks from backing validators to put them in the local `Availability Store` whenever we find an occupied core
|
||||
on any fresh leaf, this is to ensure availability by at least 2/3+ of all validators, this happens after a candidate
|
||||
is backed.
|
||||
- Fetch `PoV` from validators, when requested via `FetchPoV` message from backing (`pov_requester` module).
|
||||
|
||||
The backing subsystem is responsible of making available data available in the local `Availability Store` upon
|
||||
validation. This subsystem will serve any network requests by querying that store.
|
||||
|
||||
## Protocol
|
||||
|
||||
This subsystem does not handle any peer set messages, but the `pov_requester` does connect to validators of the same
|
||||
backing group on the validation peer set, to ensure fast propagation of statements between those validators and for
|
||||
ensuring already established connections for requesting `PoV`s. Other than that this subsystem drives request/response
|
||||
protocols.
|
||||
|
||||
Input:
|
||||
|
||||
- `OverseerSignal::ActiveLeaves(ActiveLeavesUpdate)`
|
||||
- `AvailabilityDistributionMessage{msg: ChunkFetchingRequest}`
|
||||
- `AvailabilityDistributionMessage{msg: PoVFetchingRequest}`
|
||||
- `AvailabilityDistributionMessage{msg: FetchPoV}`
|
||||
|
||||
Output:
|
||||
|
||||
- `NetworkBridgeMessage::SendRequests(Requests, IfDisconnected::TryConnect)`
|
||||
- `AvailabilityStore::QueryChunk(candidate_hash, index, response_channel)`
|
||||
- `AvailabilityStore::StoreChunk(candidate_hash, chunk)`
|
||||
- `AvailabilityStore::QueryAvailableData(candidate_hash, response_channel)`
|
||||
- `RuntimeApiRequest::SessionIndexForChild`
|
||||
- `RuntimeApiRequest::SessionInfo`
|
||||
- `RuntimeApiRequest::AvailabilityCores`
|
||||
|
||||
## Functionality
|
||||
|
||||
### PoV Requester
|
||||
|
||||
The PoV requester in the `pov_requester` module takes care of staying connected to validators of the current backing
|
||||
group of this very validator on the `Validation` peer set and it will handle `FetchPoV` requests by issuing network
|
||||
requests to those validators. It will check the hash of the received `PoV`, but will not do any further validation. That
|
||||
needs to be done by the original `FetchPoV` sender (backing subsystem).
|
||||
|
||||
### Chunk Requester
|
||||
|
||||
After a candidate is backed, the availability of the PoV block must be confirmed by 2/3+ of all validators. The chunk
|
||||
requester is responsible of making that availability a reality.
|
||||
|
||||
It does that by querying checking occupied cores for all active leaves. For each occupied core it will spawn a task
|
||||
fetching the erasure chunk which has the `ValidatorIndex` of the node. For this an `ChunkFetchingRequest` is issued, via
|
||||
Substrate's generic request/response protocol.
|
||||
|
||||
The spawned task will start trying to fetch the chunk from validators in responsible group of the occupied core, in a
|
||||
random order. For ensuring that we use already open TCP connections wherever possible, the requester maintains a cache
|
||||
and preserves that random order for the entire session.
|
||||
|
||||
Note however that, because not all validators in a group have to be actual backers, not all of them are required to have
|
||||
the needed chunk. This in turn could lead to low throughput, as we have to wait for fetches to fail, before reaching a
|
||||
validator finally having our chunk. We do rank back validators not delivering our chunk, but as backers could vary from
|
||||
block to block on a perfectly legitimate basis, this is still not ideal. See issues
|
||||
[2509](https://github.com/paritytech/polkadot/issues/2509) and
|
||||
[2512](https://github.com/paritytech/polkadot/issues/2512) for more information.
|
||||
|
||||
The current implementation also only fetches chunks for occupied cores in blocks in active leaves. This means though, if
|
||||
active leaves skips a block or we are particularly slow in fetching our chunk, we might not fetch our chunk if
|
||||
availability reached 2/3 fast enough (slot becomes free). This is not desirable as we would like as many validators as
|
||||
possible to have their chunk. See this [issue](https://github.com/paritytech/polkadot/issues/2513) for more details.
|
||||
|
||||
|
||||
### Serving
|
||||
|
||||
On the other side the subsystem will listen for incoming `ChunkFetchingRequest`s and `PoVFetchingRequest`s from the
|
||||
network bridge and will respond to queries, by looking the requested chunks and `PoV`s up in the availability store,
|
||||
this happens in the `responder` module.
|
||||
|
||||
We rely on the backing subsystem to make available data available locally in the `Availability Store` after it has
|
||||
validated it.
|
||||
@@ -0,0 +1,184 @@
|
||||
# Availability Recovery
|
||||
|
||||
This subsystem is responsible for recovering the data made available via the
|
||||
[Availability Distribution](availability-distribution.md) subsystem, necessary for candidate validation during the
|
||||
approval/disputes processes. Additionally, it is also being used by collators to recover PoVs in adversarial scenarios
|
||||
where the other collators of the para are censoring blocks.
|
||||
|
||||
According to the Pezkuwi protocol, in order to recover any given `AvailableData`, we generally must recover at least
|
||||
`f + 1` pieces from validators of the session. Thus, we should connect to and query randomly chosen validators until we
|
||||
have received `f + 1` pieces.
|
||||
|
||||
In practice, there are various optimisations implemented in this subsystem which avoid querying all chunks from
|
||||
different validators and/or avoid doing the chunk reconstruction altogether.
|
||||
|
||||
## Protocol
|
||||
|
||||
This version of the availability recovery subsystem is based only on request-response network protocols.
|
||||
|
||||
Input:
|
||||
|
||||
* `AvailabilityRecoveryMessage::RecoverAvailableData(candidate, session, backing_group, core_index, response)`
|
||||
|
||||
Output:
|
||||
|
||||
* `NetworkBridgeMessage::SendRequests`
|
||||
* `AvailabilityStoreMessage::QueryAllChunks`
|
||||
* `AvailabilityStoreMessage::QueryAvailableData`
|
||||
* `AvailabilityStoreMessage::QueryChunkSize`
|
||||
|
||||
|
||||
## Functionality
|
||||
|
||||
We hold a state which tracks the currently ongoing recovery tasks. A `RecoveryTask` is a structure encapsulating all
|
||||
network tasks needed in order to recover the available data in respect to a candidate.
|
||||
|
||||
Each `RecoveryTask` has a collection of ordered recovery strategies to try.
|
||||
|
||||
```rust
|
||||
/// Subsystem state.
|
||||
struct State {
|
||||
/// Each recovery task is implemented as its own async task,
|
||||
/// and these handles are for communicating with them.
|
||||
ongoing_recoveries: FuturesUnordered<RecoveryHandle>,
|
||||
/// A recent block hash for which state should be available.
|
||||
live_block: (BlockNumber, Hash),
|
||||
/// An LRU cache of recently recovered data.
|
||||
availability_lru: LruMap<CandidateHash, CachedRecovery>,
|
||||
/// Cached runtime info.
|
||||
runtime_info: RuntimeInfo,
|
||||
}
|
||||
|
||||
struct RecoveryParams {
|
||||
/// Discovery ids of `validators`.
|
||||
pub validator_authority_keys: Vec<AuthorityDiscoveryId>,
|
||||
/// Number of validators.
|
||||
pub n_validators: usize,
|
||||
/// The number of regular chunks needed.
|
||||
pub threshold: usize,
|
||||
/// The number of systematic chunks needed.
|
||||
pub systematic_threshold: usize,
|
||||
/// A hash of the relevant candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// The root of the erasure encoding of the candidate.
|
||||
pub erasure_root: Hash,
|
||||
/// Metrics to report.
|
||||
pub metrics: Metrics,
|
||||
/// Do not request data from availability-store. Useful for collators.
|
||||
pub bypass_availability_store: bool,
|
||||
/// The type of check to perform after available data was recovered.
|
||||
pub post_recovery_check: PostRecoveryCheck,
|
||||
/// The blake2-256 hash of the PoV.
|
||||
pub pov_hash: Hash,
|
||||
/// Protocol name for ChunkFetchingV1.
|
||||
pub req_v1_protocol_name: ProtocolName,
|
||||
/// Protocol name for ChunkFetchingV2.
|
||||
pub req_v2_protocol_name: ProtocolName,
|
||||
/// Whether or not chunk mapping is enabled.
|
||||
pub chunk_mapping_enabled: bool,
|
||||
/// Channel to the erasure task handler.
|
||||
pub erasure_task_tx: mpsc::Sender<ErasureTask>,
|
||||
}
|
||||
|
||||
pub struct RecoveryTask<Sender: overseer::AvailabilityRecoverySenderTrait> {
|
||||
sender: Sender,
|
||||
params: RecoveryParams,
|
||||
strategies: VecDeque<Box<dyn RecoveryStrategy<Sender>>>,
|
||||
state: task::State,
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
/// Common trait for runnable recovery strategies.
|
||||
pub trait RecoveryStrategy<Sender: overseer::AvailabilityRecoverySenderTrait>: Send {
|
||||
/// Main entry point of the strategy.
|
||||
async fn run(
|
||||
mut self: Box<Self>,
|
||||
state: &mut task::State,
|
||||
sender: &mut Sender,
|
||||
common_params: &RecoveryParams,
|
||||
) -> Result<AvailableData, RecoveryError>;
|
||||
|
||||
/// Return the name of the strategy for logging purposes.
|
||||
fn display_name(&self) -> &'static str;
|
||||
|
||||
/// Return the strategy type for use as a metric label.
|
||||
fn strategy_type(&self) -> &'static str;
|
||||
}
|
||||
```
|
||||
|
||||
### Signal Handling
|
||||
|
||||
On `ActiveLeavesUpdate`, if `activated` is non-empty, set `state.live_block_hash` to the first block in `Activated`.
|
||||
|
||||
Ignore `BlockFinalized` signals.
|
||||
|
||||
On `Conclude`, shut down the subsystem.
|
||||
|
||||
#### `AvailabilityRecoveryMessage::RecoverAvailableData(...)`
|
||||
|
||||
1. Check the `availability_lru` for the candidate and return the data if present.
|
||||
1. Check if there is already a recovery handle for the request. If so, add the response handle to it.
|
||||
1. Otherwise, load the session info for the given session under the state of `live_block_hash`, and initiate a recovery
|
||||
task with `launch_recovery_task`. Add a recovery handle to the state and add the response channel to it.
|
||||
1. If the session info is not available, return `RecoveryError::Unavailable` on the response channel.
|
||||
|
||||
### Recovery logic
|
||||
|
||||
#### `handle_recover(...) -> Result<()>`
|
||||
|
||||
Instantiate the appropriate `RecoveryStrategy`es, based on the subsystem configuration, params and session info.
|
||||
Call `launch_recovery_task()`.
|
||||
|
||||
#### `launch_recovery_task(state, ctx, response_sender, recovery_strategies, params) -> Result<()>`
|
||||
|
||||
Create the `RecoveryTask` and launch it as a background task running `recovery_task.run()`.
|
||||
|
||||
#### `recovery_task.run(mut self) -> Result<AvailableData, RecoveryError>`
|
||||
|
||||
* Loop:
|
||||
* Pop a strategy from the queue. If none are left, return `RecoveryError::Unavailable`.
|
||||
* Run the strategy.
|
||||
* If the strategy returned successfully or returned `RecoveryError::Invalid`, break the loop.
|
||||
|
||||
### Recovery strategies
|
||||
|
||||
#### `FetchFull`
|
||||
|
||||
This strategy tries requesting the full available data from the validators in the backing group to
|
||||
which the node is already connected. They are tried one by one in a random order.
|
||||
It is very performant if there's enough network bandwidth and the backing group is not overloaded.
|
||||
The costly reed-solomon reconstruction is not needed.
|
||||
|
||||
#### `FetchSystematicChunks`
|
||||
|
||||
Very similar to `FetchChunks` below but requests from the validators that hold the systematic chunks, so that we avoid
|
||||
reed-solomon reconstruction. Only possible if `node_features::FeatureIndex::AvailabilityChunkMapping` is enabled and
|
||||
the `core_index` is supplied (currently only for recoveries triggered by approval voting).
|
||||
|
||||
More info in
|
||||
[RFC-47](https://github.com/polkadot-fellows/RFCs/blob/main/text/0047-assignment-of-availability-chunks.md).
|
||||
|
||||
#### `FetchChunks`
|
||||
|
||||
The least performant strategy but also the most comprehensive one. It's the only one that cannot fail under the
|
||||
byzantine threshold assumption, so it's always added as the last one in the `recovery_strategies` queue.
|
||||
|
||||
Performs parallel chunk requests to validators. When enough chunks were received, do the reconstruction.
|
||||
In the worst case, all validators will be tried.
|
||||
|
||||
### Default recovery strategy configuration
|
||||
|
||||
#### For validators
|
||||
|
||||
If the estimated available data size is smaller than a configured constant (currently 1Mib for Pezkuwi or 4Mib for
|
||||
other networks), try doing `FetchFull` first.
|
||||
Next, if the preconditions described in `FetchSystematicChunks` above are met, try systematic recovery.
|
||||
As a last resort, do `FetchChunks`.
|
||||
|
||||
#### For collators
|
||||
|
||||
Collators currently only use `FetchChunks`, as they only attempt recoveries in rare scenarios.
|
||||
|
||||
Moreover, the recovery task is specially configured to not attempt requesting data from the local availability-store
|
||||
(because it doesn't exist) and to not reencode the data after a successful recovery (because it's an expensive check
|
||||
that is not needed; checking the pov_hash is enough for collators).
|
||||
@@ -0,0 +1,40 @@
|
||||
# Bitfield Distribution
|
||||
|
||||
Validators vote on the availability of a backed candidate by issuing signed bitfields, where each bit corresponds to a
|
||||
single candidate. These bitfields can be used to compactly determine which backed candidates are available or not based
|
||||
on a 2/3+ quorum.
|
||||
|
||||
## Protocol
|
||||
|
||||
`PeerSet`: `Validation`
|
||||
|
||||
Input: [`BitfieldDistributionMessage`](../../types/overseer-protocol.md#bitfield-distribution-message) which are
|
||||
gossiped to all peers, no matter if validator or not.
|
||||
|
||||
Output:
|
||||
|
||||
- `NetworkBridge::SendValidationMessage([PeerId], message)` gossip a verified incoming bitfield on to interested
|
||||
subsystems within this validator node.
|
||||
- `NetworkBridge::ReportPeer(PeerId, cost_or_benefit)` improve or penalize the reputation of peers based on the messages
|
||||
that are received relative to the current view.
|
||||
- `ProvisionerMessage::ProvisionableData(ProvisionableData::Bitfield(relay_parent, SignedAvailabilityBitfield))` pass on
|
||||
the bitfield to the other submodules via the overseer.
|
||||
|
||||
## Functionality
|
||||
|
||||
This is implemented as a gossip system.
|
||||
|
||||
It is necessary to track peer connection, view change, and disconnection events, in order to maintain an index of which
|
||||
peers are interested in which relay parent bitfields.
|
||||
|
||||
|
||||
Before gossiping incoming bitfields, they must be checked to be signed by one of the validators of the validator set
|
||||
relevant to the current relay parent. Only accept bitfields relevant to our current view and only distribute bitfields
|
||||
to other peers when relevant to their most recent view. Accept and distribute only one bitfield per validator.
|
||||
|
||||
|
||||
When receiving a bitfield either from the network or from a `DistributeBitfield` message, forward it along to the block
|
||||
authorship (provisioning) subsystem for potential inclusion in a block.
|
||||
|
||||
Peers connecting after a set of valid bitfield gossip messages was received, those messages must be cached and sent upon
|
||||
connection of new peers or re-connecting peers.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Bitfield Signing
|
||||
|
||||
Validators vote on the availability of a backed candidate by issuing signed bitfields, where each bit corresponds to a
|
||||
single candidate. These bitfields can be used to compactly determine which backed candidates are available or not based
|
||||
on a 2/3+ quorum.
|
||||
|
||||
## Protocol
|
||||
|
||||
Input:
|
||||
|
||||
There is no dedicated input mechanism for bitfield signing. Instead, Bitfield Signing produces a bitfield representing
|
||||
the current state of availability on `StartWork`.
|
||||
|
||||
Output:
|
||||
|
||||
- `BitfieldDistribution::DistributeBitfield`: distribute a locally signed bitfield
|
||||
- `AvailabilityStore::QueryChunk(CandidateHash, validator_index, response_channel)`
|
||||
|
||||
## Functionality
|
||||
|
||||
Upon receipt of an `ActiveLeavesUpdate`, launch bitfield signing job for each `activated` head referring to a fresh
|
||||
leaf. Stop the job for each `deactivated` head.
|
||||
|
||||
## Bitfield Signing Job
|
||||
|
||||
Localized to a specific relay-parent `r` If not running as a validator, do nothing.
|
||||
|
||||
- For each fresh leaf, begin by waiting a fixed period of time so availability distribution has the chance to make
|
||||
candidates available.
|
||||
- Determine our validator index `i`, the set of backed candidates pending availability in `r`, and which bit of the
|
||||
bitfield each corresponds to.
|
||||
- Start with an empty bitfield. For each bit in the bitfield, if there is a candidate pending availability, query the
|
||||
[Availability Store](../utility/availability-store.md) for whether we have the availability chunk for our validator
|
||||
index. The `OccupiedCore` struct contains the candidate hash so the full candidate does not need to be fetched from
|
||||
runtime.
|
||||
- For all chunks we have, set the corresponding bit in the bitfield.
|
||||
- Sign the bitfield and dispatch a `BitfieldDistribution::DistributeBitfield` message.
|
||||
Reference in New Issue
Block a user