Refactor primitives (#1383)

* create a v1 primitives module * Improve guide on availability types * punctuate * new parachains runtime uses new primitives * tests of new runtime now use new primitives * add ErasureChunk to guide * export erasure chunk from v1 primitives * subsystem crate uses v1 primitives * node-primitives uses new v1 primitives * port overseer to new primitives * new-proposer uses v1 primitives (no ParachainHost anymore) * fix no-std compilation for primitives * service-new uses v1 primitives * network-bridge uses new primitives * statement distribution uses v1 primitives * PoV distribution uses v1 primitives; add PoV::hash fn * move parachain to v0 * remove inclusion_inherent module and place into v1 * remove everything from primitives crate root * remove some unused old types from v0 primitives * point everything else at primitives::v0 * squanch some warns up * add RuntimeDebug import to no-std as well * port over statement-table and validation * fix final errors in validation and node-primitives * add dummy Ord impl to committed candidate receipt * guide: update CandidateValidationMessage * add primitive for validationoutputs * expand CandidateValidationMessage further * bikeshed * add some impls to omitted-validation-data and available-data * expand CandidateValidationMessage * make erasure-coding generic over v1/v0 * update usages of erasure-coding * implement commitments.hash() * use Arc<Pov> for CandidateValidation * improve new erasure-coding method names * fix up candidate backing * update docs a bit * fix most tests and add short-circuiting to make_pov_available * fix remainder of candidate backing tests * squanching warns * squanch it up * some fallout * overseer fallout * free from polkadot-test-service hell
2026-05-31 08:41:02 +00:00 · 2020-07-09 21:23:03 -04:00
parent 6957847b6b
commit 3b13cd9a85
76 changed files with 1542 additions and 999 deletions
@@ -9,7 +9,7 @@ A bitfield [signed](backing.md#signed-wrapper) by a particular validator about t


 ```rust
-pub type SignedAvailabilityBitfield = Signed<Bitvec>;
+type SignedAvailabilityBitfield = Signed<Bitvec>;

 struct Bitfields(Vec<(SignedAvailabilityBitfield)>), // bitfields sorted by validator index, ascending
 ```
@@ -21,3 +21,50 @@ Often referred to as PoV, this is a type-safe wrapper around bytes (`Vec<u8>`) w
 ```rust
 struct PoV(Vec<u8>);
 ```
+
+## Omitted Validation Data
+
+Validation data that is often omitted from types describing candidates as it can be derived from the relay-parent of the candidate. However, with the expectation of state pruning, these are best kept available elsewhere as well.
+
+This contains the [`GlobalValidationSchedule`](candidate.md#globalvalidationschedule) and [`LocalValidationData`](candidate.md#localvalidationdata)
+
+```rust
+struct OmittedValidationData {
+    /// The global validation schedule.
+    global_validation: GlobalValidationSchedule,
+    /// The local validation data.
+    local_validation: LocalValidationData,
+}
+```
+
+
+## Available Data
+
+This is the data we want to keep available for each [candidate](candidate.md) included in the relay chain.
+
+```rust
+struct AvailableData {
+    /// The Proof-of-Validation of the candidate.
+    pov: PoV,
+    /// The omitted validation data.
+    omitted_validation: OmittedValidationData,
+}
+```
+
+> TODO: With XCMP, we also need to keep available the outgoing messages as a result of para-validation.
+
+## Erasure Chunk
+
+The [`AvailableData`](#availabledata) is split up into an erasure-coding as part of the availability process. Each validator gets a chunk. This describes one of those chunks, along with its proof against a merkle root hash, which should be apparent from context, and is the `erasure_root` field of a [`CandidateDescriptor`](candidate.md#candidatedescriptor).
+
+
+```rust
+struct ErasureChunk {
+    /// The erasure-encoded chunk of data belonging to the candidate block.
+    chunk: Vec<u8>,
+    /// The index of this erasure-encoded chunk of data.
+    index: u32,
+    /// Proof for this chunk's branch in the Merkle tree.
+    proof: Vec<Vec<u8>>,
+}
+```
@@ -1,6 +1,7 @@
 # Candidate Types

 Para candidates are some of the most common types, both within the runtime and on the Node-side.
+Candidates are the fundamental datatype for advancing parachains and parathreads, encapsulating the collator's signature, the context of the parablock, the commitments to the output, and a commitment to the data which proves it valid.

 In a way, this entire guide is about these candidates: how they are scheduled, constructed, backed, included, and challenged.

@@ -179,3 +180,24 @@ struct SigningContext {
 	session_index: SessionIndex,
 }
 ```
+
+## Validation Outputs
+
+This struct encapsulates the outputs of candidate validation.
+
+```rust
+struct ValidationOutputs {
+	/// The head-data produced by validation.
+	head_data: HeadData,
+	/// The global validation schedule.
+	global_validation_schedule: GlobalValidationSchedule,
+	/// The local validation data.
+	local_validation_data: LocalValidationData,
+	/// Upwards messages to the relay chain.
+	upwards_messages: Vec<UpwardsMessage>,
+	/// Fees paid to the validators of the relay-chain.
+	fees: Balance,
+	/// The new validation code submitted by the execution, if any.
+	new_validation_code: Option<ValidationCode>,
+}
+```
@@ -91,6 +91,7 @@ enum CandidateBackingMessage {
  GetBackedCandidates(Hash, ResponseChannel<Vec<NewBackedCandidate>>),
  /// Note that the Candidate Backing subsystem should second the given candidate in the context of the
  /// given relay-parent (ref. by hash). This candidate must be validated using the provided PoV.
+  /// The PoV is expected to match the `pov_hash` in the descriptor.
  Second(Hash, CandidateReceipt, PoV),
  /// Note a peer validator's statement about a particular candidate. Disagreements about validity must be escalated
  /// to a broader check by Misbehavior Arbitration. Agreements are simply tallied until a quorum is reached.
@@ -282,29 +283,41 @@ enum StatementDistributionMessage {

 ## Validation Request Type

-Various modules request that the [Candidate Validation subsystem](../node/utility/candidate-validation.md) validate a block with this message
+Various modules request that the [Candidate Validation subsystem](../node/utility/candidate-validation.md) validate a block with this message. It returns [`ValidationOutputs`](candidate.md#validationoutputs) for successful validation.

 ```rust

 /// Result of the validation of the candidate.
 enum ValidationResult {
-	/// Candidate is valid.
-	Valid,
+	/// Candidate is valid, and here are the outputs. In practice, this should be a shared type
+	/// so that validation caching can be done.
+	Valid(ValidationOutputs),
 	/// Candidate is invalid.
 	Invalid,
 }

+/// Messages issued to the candidate validation subsystem.
+///
+/// ## Validation Requests
+///
+/// Validation requests made to the subsystem should return an error only on internal error.
+/// Otherwise, they should return either `Ok(ValidationResult::Valid(_))` or `Ok(ValidationResult::Invalid)`.
 enum CandidateValidationMessage {
-	/// Validate a candidate with provided parameters. Returns `Err` if an only if an internal
-	/// error is encountered.
-	/// In case no internal error was encontered it returns a tuple containing the result of
-	/// validation and `GlobalValidationSchedule` and `LocalValidationData` structures that
-	/// may be used by the caller to make the candidate available.
-	/// A bad candidate will return `Ok((ValidationResult::Invalid, _, _)`, while a good one will
-	/// return `Ok((ValidationResult::Valid, _, _))`.
-	Validate(
-		Hash, CandidateReceipt, HeadData, PoV, ResponseChannel<
-			Result<(ValidationResult, GlobalValidationSchedule, LocalValidationData)>
-		>),
+	/// Validate a candidate with provided parameters. This will implicitly attempt to gather the
+	/// `OmittedValidationData` and `ValidationCode` from the runtime API of the chain,
+	/// based on the `relay_parent` of the `CandidateDescriptor`.
+	/// If there is no state available which can provide this data, an error is returned.
+	ValidateFromChainState(CandidateDescriptor, PoV, ResponseChannel<Result<ValidationResult>>),
+
+	/// Validate a candidate with provided parameters. Explicitly provide the `OmittedValidationData`
+	/// and `ValidationCode` so this can do full validation without needing to access the state of
+	/// the relay-chain.
+	ValidateFromExhaustive(
+		OmittedValidationData,
+		ValidationCode,
+		CandidateDescriptor,
+		PoV,
+		ResponseChannel<Result<ValidationResult>>,
+	),
 }
 ```