pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-05-31 14:31:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

CandidateBackingSubsystem (#1312)

Browse Source 
* Updates guide for CandidateBacking

* Move assignment types to primitives

* Initial implementation.

* More functionality

* use assert_matches

* Changes to report misbehaviors

* Some fixes after a review

* Remove a blank line

* Update guide and some types

* Adds run_job function

* Some comments and refactorings

* Fix review

* Remove warnings

* Use summary in kicking off validation

* Parallelize requests

* Validation provides local and global validation params

* Test issued validity tracking

* Nits from review
This commit is contained in:
Fedor Sakharov
2020-07-09 23:23:58 +03:00
committed by GitHub
parent 54bace2b5d
commit c119627835
14 changed files with 2638 additions and 453 deletions
Download Patch File Download Diff File Expand all files Collapse all files
polkadot/roadmap/implementers-guide/src/node/backing/candidate-backing.md
+71 -26
View File
@@ -2,30 +2,43 @@
The Candidate Backing subsystem ensures every parablock considered for relay block inclusion has been seconded by at least one validator, and approved by a quorum. Parablocks for which no validator will assert correctness are discarded. If the block later proves invalid, the initial backers are slashable; this gives polkadot a rational threat model during subsequent stages.
Its role is to produce backable candidates for inclusion in new relay-chain blocks. It does so by issuing signed [`Statement`s](../../types/backing.md#statement-type) and tracking received statements signed by other validators. Once enough statements are received, they can be combined into backing for specific candidates.
Its role is to produce backable candidates for inclusion in new relay-chain blocks. It does so by issuing signed [`Statement`s][Statement] and tracking received statements signed by other validators. Once enough statements are received, they can be combined into backing for specific candidates.
Note that though the candidate backing subsystem attempts to produce as many backable candidates as possible, it does _not_ attempt to choose a single authoritative one. The choice of which actually gets included is ultimately up to the block author, by whatever metrics it may use; those are opaque to this subsystem.
Once a sufficient quorum has agreed that a candidate is valid, this subsystem notifies the [Provisioner](../utility/provisioner.md), which in turn engages block production mechanisms to include the parablock.
Once a sufficient quorum has agreed that a candidate is valid, this subsystem notifies the [Provisioner][PV], which in turn engages block production mechanisms to include the parablock.
## Protocol
The [Candidate Selection subsystem](candidate-selection.md) is the primary source of non-overseer messages into this subsystem. That subsystem generates appropriate [`CandidateBackingMessage`s](../../types/overseer-protocol.md#candidate-backing-message), and passes them to this subsystem.
Input: [`CandidateBackingMessage`][CBM]
This subsystem validates the candidates and generates an appropriate [`SignedStatement`](../../types/backing.md#signed-statement-type). All `SignedStatement`s are then passed on to the [Statement Distribution subsystem](statement-distribution.md) to be gossiped to peers. All [Proofs of Validity](../../types/availability.md#proof-of-validity) should be distributed via the [PoV Distribution](pov-distribution.md) subsystem. When this subsystem decides that a candidate is invalid, and it was recommended to us to second by our own Candidate Selection subsystem, a message is sent to the Candidate Selection subsystem with the candidate's hash so that the collator which recommended it can be penalized.
Output:
- [`CandidateValidationMessage`][CVM]
- [`RuntimeApiMessage`][RAM]
- [`CandidateSelectionMessage`][CSM]
- [`ProvisionerMessage`][PM]
- [`PoVDistributionMessage`][PDM]
- [`StatementDistributionMessage`][SDM]
## Functionality
The [Candidate Selection][CS] subsystem is the primary source of non-overseer messages into this subsystem. That subsystem generates appropriate [`CandidateBackingMessage`s][CBM] and passes them to this subsystem.
This subsystem requests validation from the [Candidate Validation][CV] and generates an appropriate [`Statement`][Statement]. All `Statement`s are then passed on to the [Statement Distribution][SD] subsystem to be gossiped to peers. When [Candidate Validation][CV] decides that a candidate is invalid, and it was recommended to us to second by our own [Candidate Selection][CS] subsystem, a message is sent to the [Candidate Selection][CS] subsystem with the candidate's hash so that the collator which recommended it can be penalized.
The subsystem should maintain a set of handles to Candidate Backing Jobs that are currently live, as well as the relay-parent to which they correspond.
### On Overseer Signal
* If the signal is an [`OverseerSignal`](../../types/overseer-protocol.md#overseer-signal)`::StartWork(relay_parent)`, spawn a Candidate Backing Job with the given relay parent, storing a bidirectional channel with the Candidate Backing Job in the set of handles.
* If the signal is an [`OverseerSignal`](../../types/overseer-protocol.md#overseer-signal)`::StopWork(relay_parent)`, cease the Candidate Backing Job under that relay parent, if any.
* If the signal is an [`OverseerSignal`][OverseerSignal]`::StartWork(relay_parent)`, spawn a Candidate Backing Job with the given relay parent, storing a bidirectional channel with the Candidate Backing Job in the set of handles.
* If the signal is an [`OverseerSignal`][OverseerSignal]`::StopWork(relay_parent)`, cease the Candidate Backing Job under that relay parent, if any.
### On `CandidateBackingMessage`
### On Receiving `CandidateBackingMessage`
* If the message corresponds to a particular relay-parent, forward the message to the Candidate Backing Job for that relay-parent, if any is live.
* If the message is a [`CandidateBackingMessage`][CBM]`::GetBackedCandidates`, get all backable candidates from the statement table and send them back.
* If the message is a [`CandidateBackingMessage`][CBM]`::Second`, sign and dispatch a `Seconded` statement only if we have not seconded any other candidate and have not signed a `Valid` statement for the requested candidate. Signing both a `Seconded` and `Valid` message is a double-voting misbehavior with a heavy penalty, and this could occur if another validator has seconded the same candidate and we've received their message before the internal seconding request.
* If the message is a [`CandidateBackingMessage`][CBM]`::Statement`, count the statement to the quorum. If the statement in the message is `Seconded` and it contains a candidate that belongs to our assignment, request the corresponding `PoV` from the `PoVDistribution` and launch validation. Issue our own `Valid` or `Invalid` statement as a result.
> big TODO: "contextual execution"
>
@@ -39,36 +52,49 @@ The subsystem should maintain a set of handles to Candidate Backing Jobs that ar
The Candidate Backing Job represents the work a node does for backing candidates with respect to a particular relay-parent.
The goal of a Candidate Backing Job is to produce as many backable candidates as possible. This is done via signed [`Statement`s](../../types/backing.md#statement-type) by validators. If a candidate receives a majority of supporting Statements from the Parachain Validators currently assigned, then that candidate is considered backable.
The goal of a Candidate Backing Job is to produce as many backable candidates as possible. This is done via signed [`Statement`s][STMT] by validators. If a candidate receives a majority of supporting Statements from the Parachain Validators currently assigned, then that candidate is considered backable.
### On Startup
* Fetch current validator set, validator -> parachain assignments from runtime API.
* Fetch current validator set, validator -> parachain assignments from [`Runtime API`][RA] subsystem using [`RuntimeApiRequest::Validators`][RAM] and [`RuntimeApiRequest::ValidatorGroups`][RAM]
* Determine if the node controls a key in the current validator set. Call this the local key if so.
* If the local key exists, extract the parachain head and validation function for the parachain the local key is assigned to.
* If the local key exists, extract the parachain head and validation function from the [`Runtime API`][RA] for the parachain the local key is assigned to by issuing a [`RuntimeApiRequest::Validators`][RAM]
* Issue a [`RuntimeApiRequest::SigningContext`][RAM] message to get a context that will later be used upon signing.
### On Receiving New Signed Statement
### On Receiving New Candidate Backing Message
```rust
if let Statement::Seconded(candidate) = signed.statement {
if candidate is unknown and in local assignment {
spawn_validation_work(candidate, parachain head, validation function)
match msg {
CetBackedCandidates(hash, tx) => {
// Send back a set of backable candidates.
}
CandidateBackingMessage::Second(hash, candidate) => {
if candidate is unknown and in local assignment {
spawn_validation_work(candidate, parachain head, validation function)
}
}
CandidateBackingMessage::Statement(hash, statement) => {
// count to the votes on this candidate
if let Statement::Seconded(candidate) = statement {
if candidate.parachain_id == our_assignment {
spawn_validation_work(candidate, parachain head, validation function)
}
}
}
}
// add `Seconded` statements and `Valid` statements to a quorum. If quorum reaches validator-group
// majority, send a `BlockAuthorshipProvisioning::BackableCandidate(relay_parent, Candidate, Backing)` message.
```
### Spawning Validation Work
Add `Seconded` statements and `Valid` statements to a quorum. If quorum reaches validator-group majority, send a [`ProvisionerMessage`][PM]`::ProvisionableData(ProvisionableData::BackedCandidate(BackedCandidate))` message.
`Invalid` statements that conflict with already witnessed `Seconded` and `Valid` statements for the given candidate, statements that are double-votes, self-contradictions and so on, should result in issuing a [`ProvisionerMessage`][PM]`::MisbehaviorReport` message for each newly detected case of this kind.
### Validating Candidates.
```rust
fn spawn_validation_work(candidate, parachain head, validation function) {
asynchronously {
let pov = (fetch pov block).await
// dispatched to sub-process (OS process) pool.
let valid = validate_candidate(candidate, validation function, parachain head, pov).await;
let valid = (validate pov block).await;
if valid {
// make PoV available for later distribution. Send data to the availability store to keep.
// sign and dispatch `valid` statement to network if we have not seconded the given candidate.
@@ -82,11 +108,30 @@ fn spawn_validation_work(candidate, parachain head, validation function) {
### Fetch Pov Block
Create a `(sender, receiver)` pair.
Dispatch a `PovFetchSubsystemMessage(relay_parent, candidate_hash, sender)` and listen on the receiver for a response.
Dispatch a [`PoVDistributionMessage`][PDM]`::FecthPoV(relay_parent, candidate_hash, sender)` and listen on the receiver for a response.
### On Receiving `CandidateBackingMessage`
### Validate PoV Block
* If the message is a `CandidateBackingMessage::RegisterBackingWatcher`, register the watcher and trigger it each time a new candidate is backable. Also trigger it once initially if there are any backable candidates at the time of receipt.
* If the message is a `CandidateBackingMessage::Second`, sign and dispatch a `Seconded` statement only if we have not seconded any other candidate and have not signed a `Valid` statement for the requested candidate. Signing both a `Seconded` and `Valid` message is a double-voting misbehavior with a heavy penalty, and this could occur if another validator has seconded the same candidate and we've received their message before the internal seconding request.
Create a `(sender, receiver)` pair.
Dispatch a `CandidateValidationMessage::Validate(validation function, candidate, pov, sender)` and listen on the receiver for a response.
> TODO: send statements to Statement Distribution subsystem, handle shutdown signal from candidate backing subsystem
### Distribute Signed Statemnet
Dispatch a [`StatementDistributionMessage`][PDM]`::Share(relay_parent, SignedFullStatement)`.
[OverseerSignal]: ../../types/overseer-protocol.md#overseer-signal
[Statement]: ../../types/backing.md#statement-type
[STMT]: ../../types/backing.md#statement-type
[CSM]: ../../types/overseer-protocol.md#candidate-selection-message
[RAM]: ../../types/overseer-protocol.md#runtime-api-message
[CVM]: ../../types/overseer-protocol.md#validation-request-type
[PM]: ../../types/overseer-protocol.md#provisioner-message
[CBM]: ../../types/overseer-protocol.md#candidate-backing-message
[PDM]: ../../types/overseer-protocol.md#pov-distribution-message
[SDM]: ../../types/overseer-protocol.md#statement-distribution-message
[CS]: candidate-selection.md
[CV]: ../utility/candidate-validation.md
[SD]: statement-distribution.md
[RA]: ../utility/runtime-api.md
[PV]: ../utility/provisioner.md
polkadot/roadmap/implementers-guide/src/types/overseer-protocol.md
+37 -6
View File
@@ -86,9 +86,9 @@ enum BitfieldSigningMessage { }
```rust
enum CandidateBackingMessage {
/// Registers a stream listener for updates to the set of backable candidates that could be backed
/// in a child of the given relay-parent, referenced by its hash.
RegisterBackingWatcher(Hash, TODO),
/// Requests a set of backable candidates that could be backed in a child of the given
/// relay-parent, referenced by its hash.
GetBackedCandidates(Hash, ResponseChannel<Vec<NewBackedCandidate>>),
/// Note that the Candidate Backing subsystem should second the given candidate in the context of the
/// given relay-parent (ref. by hash). This candidate must be validated using the provided PoV.
Second(Hash, CandidateReceipt, PoV),
@@ -230,9 +230,24 @@ The Runtime API subsystem is responsible for providing an interface to the state
Other subsystems query this data by sending these messages.
```rust
/// The information on validator groups, core assignments,
/// upcoming paras and availability cores.
struct SchedulerRoster {
/// Validator-to-groups assignments.
validator_groups: Vec<Vec<ValidatorIndex>>,
/// All scheduled paras.
scheduled: Vec<CoreAssignment>,
/// Upcoming paras (chains and threads).
upcoming: Vec<ParaId>,
/// Occupied cores.
availability_cores: Vec<Option<CoreOccupied>>,
}
enum RuntimeApiRequest {
/// Get the current validator set.
Validators(ResponseChannel<Vec<ValidatorId>>),
/// Get the assignments of validators to cores, upcoming parachains.
SchedulerRoster(ResponseChannel<SchedulerRoster>),
/// Get a signing context for bitfields and statements.
SigningContext(ResponseChannel<SigningContext>),
/// Get the validation code for a specific para, assuming execution under given block number, and
@@ -270,10 +285,26 @@ enum StatementDistributionMessage {
Various modules request that the [Candidate Validation subsystem](../node/utility/candidate-validation.md) validate a block with this message
```rust
/// Result of the validation of the candidate.
enum ValidationResult {
/// Candidate is valid.
Valid,
/// Candidate is invalid.
Invalid,
}
enum CandidateValidationMessage {
/// Validate a candidate with provided parameters. Returns `Err` if an only if an internal
/// error is encountered. A bad candidate will return `Ok(false)`, while a good one will
/// return `Ok(true)`.
Validate(ValidationCode, CandidateReceipt, PoV, ResponseChannel<Result<bool>>),
/// error is encountered.
/// In case no internal error was encontered it returns a tuple containing the result of
/// validation and `GlobalValidationSchedule` and `LocalValidationData` structures that
/// may be used by the caller to make the candidate available.
/// A bad candidate will return `Ok((ValidationResult::Invalid, _, _)`, while a good one will
/// return `Ok((ValidationResult::Valid, _, _))`.
Validate(
Hash, CandidateReceipt, HeadData, PoV, ResponseChannel<
Result<(ValidationResult, GlobalValidationSchedule, LocalValidationData)>
>),
}
```