Runtime API for checking validation outputs (#1842)

* annoying whitespaces

* update guide

Add `CheckValidationOutputs` runtime api and also change the
candidate-validation stuff

* promote ValidationOutputs to global primitives

i.e. move it from node specific primitives to global v1 primitives. This
will be needed when we share it later in the runtime inclusion module

* refactor acceptance checks in the inclusion module

factor out the common code to share it during the block inclusion and
for the forthcoming CheckValidationOutputs runtime api.

Also note that the acceptance criteria was updated to incorporate checks
that exist now in candidate-validation

* plumb the runtime api outside

* extract validation_data from ValidationOutputs

* use runtime-api to check validation outputs

apart from that refactor, update docs and tidy a bit

* Update the maxium code size

This is to fix a test that performs an upgrade.

polkadot/roadmap/implementers-guide/src/node/utility/candidate-validation.md

@@ -24,7 +24,7 @@ Upon receiving a validation request, the first thing the candidate validation su
 
 ### Determining Parameters
 
-For a [`CandidateValidationMessage`][CVM]`::ValidateFromExhaustive`, these parameters are exhaustively provided. The [`TransientValidationData`](../../types/candidate.md#transientvalidationdata) is optional, and is used to perform further checks on the outputs of validation.
+For a [`CandidateValidationMessage`][CVM]`::ValidateFromExhaustive`, these parameters are exhaustively provided.
 
 For a [`CandidateValidationMessage`][CVM]`::ValidateFromChainState`, some more work needs to be done. Due to the uncertainty of Availability Cores (implemented in the [`Scheduler`](../../runtime/scheduler.md) module of the runtime), a candidate at a particular relay-parent and for a particular para may have two different valid validation-data to be executed under depending on what is assumed to happen if the para is occupying a core at the onset of the new block. This is encoded as an `OccupiedCoreAssumption` in the runtime API.
 
@@ -40,9 +40,8 @@ Once we have all parameters, we can spin up a background task to perform the val
   * The collator signature is valid
   * The PoV provided matches the `pov_hash` field of the descriptor
 
-After that, we can invoke the validation function. Lastly, if available, we do some final checks on the output using the `TransientValidationData`:
-  * The produced head-data is no larger than the maximum allowed.
-  * The produced code upgrade, if any, is no larger than the maximum allowed, and a code upgrade was allowed to be signaled.
-  * The amount and size of produced upward messages is not too large.
+### Checking Validation Outputs
+
+If we can assume the presence of the relay-chain state (that is, during processing [`CandidateValidationMessage`][CVM]`::ValidateFromChainState`) we can run all the checks that the relay-chain would run at the inclusion time thus confirming that the candidate will be accepted.
 
 [CVM]: ../../types/overseer-protocol.md#validationrequesttype

polkadot/roadmap/implementers-guide/src/types/overseer-protocol.md

@@ -348,6 +348,12 @@ enum RuntimeApiRequest {
 		OccupiedCoreAssumption,
 		ResponseChannel<Option<ValidationData>>,
 	),
+	/// Sends back `true` if the commitments pass all acceptance criteria checks.
+	CheckValidationOutputs(
+		ParaId,
+		CandidateCommitments,
+		RuntimeApiSender<bool>,
+	),
 	/// Get information about all availability cores.
 	AvailabilityCores(ResponseChannel<Vec<CoreState>>),
 	/// Get a committed candidate receipt for all candidates pending availability.
@@ -392,39 +398,52 @@ Various modules request that the [Candidate Validation subsystem](../node/utilit
 
 /// Result of the validation of the candidate.
 enum ValidationResult {
-	/// Candidate is valid, and here are the outputs. In practice, this should be a shared type
-	/// so that validation caching can be done.
-	Valid(ValidationOutputs),
+	/// Candidate is valid, and here are the outputs and the validation data used to form inputs.
+	/// In practice, this should be a shared type so that validation caching can be done.
+	Valid(ValidationOutputs, PersistedValidationData),
 	/// Candidate is invalid.
 	Invalid,
 }
 
-/// Messages issued to the candidate validation subsystem.
+/// Messages received by the Validation subsystem.
 ///
 /// ## Validation Requests
 ///
 /// Validation requests made to the subsystem should return an error only on internal error.
-/// Otherwise, they should return either `Ok(ValidationResult::Valid(_))` or `Ok(ValidationResult::Invalid)`.
-enum CandidateValidationMessage {
-	/// Validate a candidate with provided parameters. This will implicitly attempt to gather the
-	/// `OmittedValidationData` and `ValidationCode` from the runtime API of the chain,
-	/// based on the `relay_parent` of the `CandidateDescriptor`.
+/// Otherwise, they should return either `Ok(ValidationResult::Valid(_))`
+/// or `Ok(ValidationResult::Invalid)`.
+#[derive(Debug)]
+pub enum CandidateValidationMessage {
+	/// Validate a candidate with provided parameters using relay-chain state.
+	///
+	/// This will implicitly attempt to gather the `PersistedValidationData` and `ValidationCode`
+	/// from the runtime API of the chain, based on the `relay_parent`
+	/// of the `CandidateDescriptor`.
+	///
+	/// This will also perform checking of validation outputs against the acceptance criteria.
 	///
 	/// If there is no state available which can provide this data or the core for
 	/// the para is not free at the relay-parent, an error is returned.
-	ValidateFromChainState(CandidateDescriptor, PoV, ResponseChannel<Result<ValidationResult>>),
-
-	/// Validate a candidate with provided parameters. Explicitly provide the `PersistedValidationData`
-	/// and `ValidationCode` so this can do full validation without needing to access the state of
-	/// the relay-chain. Optionally provide the `TransientValidationData` which will lead to checks
-	/// on the output.
+	ValidateFromChainState(
+		CandidateDescriptor,
+		Arc<PoV>,
+		oneshot::Sender<Result<ValidationResult, ValidationFailed>>,
+	),
+	/// Validate a candidate with provided, exhaustive parameters for validation.
+	///
+	/// Explicitly provide the `PersistedValidationData` and `ValidationCode` so this can do full
+	/// validation without needing to access the state of the relay-chain.
+	///
+	/// This request doesn't involve acceptance criteria checking, therefore only useful for the
+	/// cases where the validity of the candidate is established. This is the case for the typical
+	/// use-case: secondary checkers would use this request relying on the full prior checks
+	/// performed by the relay-chain.
 	ValidateFromExhaustive(
 		PersistedValidationData,
-		Option<TransientValidationData>,
 		ValidationCode,
 		CandidateDescriptor,
-		PoV,
-		ResponseChannel<Result<ValidationResult>>,
+		Arc<PoV>,
+		oneshot::Sender<Result<ValidationResult, ValidationFailed>>,
 	),
 }
 ```