Ease parachain candidate code fetching (#2593)

* code stored in para + modify CandidateDescriptor.

* WIP: digest + some more impl

* validation_code_hash in payload + check in inclusion

* check in client + refator

* tests

* fix encoding indices

* remove old todos

* fix test

* fix test

* add test

* fetch validation code inside collation-generation from the relay-chain

* HashMismatch -> PoVHashMismatch + miscompilation

* refactor, store hash when needed

* storage rename: more specific but slightly too verbose

* do not hash on candidate validation, fetch hash instead

* better test

* fix test

* guide updates

* don't panic in runtime

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

polkadot/roadmap/implementers-guide/src/runtime/paras.md

@@ -116,10 +116,10 @@ Parachains: Vec<ParaId>,
 ParaLifecycle: map ParaId => Option<ParaLifecycle>,
 /// The head-data of every registered para.
 Heads: map ParaId => Option<HeadData>;
-/// The validation code of every live para.
-ValidationCode: map ParaId => Option<ValidationCode>;
-/// Actual past code, indicated by the para id as well as the block number at which it became outdated.
-PastCode: map (ParaId, BlockNumber) => Option<ValidationCode>;
+/// The validation code hash of every live para.
+CurrentCodeHash: map ParaId => Option<Hash>;
+/// Actual past code hash, indicated by the para id as well as the block number at which it became outdated.
+PastCodeHash: map (ParaId, BlockNumber) => Option<Hash>;
 /// Past code of parachains. The parachains themselves may not be registered anymore,
 /// but we also keep their code on-chain for the same amount of time as outdated code
 /// to keep it available for secondary checkers.
@@ -136,24 +136,28 @@ PastCodePruning: Vec<(ParaId, BlockNumber)>;
 /// in the context of a relay chain block with a number >= `expected_at`.
 FutureCodeUpgrades: map ParaId => Option<BlockNumber>;
 /// The actual future code of a para.
-FutureCode: map ParaId => Option<ValidationCode>;
+FutureCodeHash: map ParaId => Option<Hash>;
 /// The actions to perform during the start of a specific session index.
 ActionsQueue: map SessionIndex => Vec<ParaId>;
 /// Upcoming paras instantiation arguments.
 UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
+/// The number of references on the validation code in `CodeByHash` storage.
+CodeByHashRefs: map Hash => u32;
+/// Validation code stored by its hash.
+CoeByHash: map Hash => Option<ValidationCode>
 ```
 
 ## Session Change
 
 1. Execute all queued actions for paralifecycle changes:
   1. Clean up outgoing paras.
-     1. This means removing the entries under `Heads`, `ValidationCode`, `FutureCodeUpgrades`, and
+     1. This means removing the entries under `Heads`, `CurrentCode`, `FutureCodeUpgrades`, and
         `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and
-        `PastCodePruning` using the outgoing `ParaId` and removed `ValidationCode` value. This is
+        `PastCodePruning` using the outgoing `ParaId` and removed `CurrentCode` value. This is
         because any outdated validation code must remain available on-chain for a determined amount
         of blocks, and validation code outdated by de-registering the para is still subject to that
         invariant.
-  1. Apply all incoming paras by initializing the `Heads` and `ValidationCode` using the genesis
+  1. Apply all incoming paras by initializing the `Heads` and `CurrentCode` using the genesis
      parameters.
   1. Amend the `Parachains` list and `ParaLifecycle` to reflect changes in registered parachains.
   1. Amend the `ParaLifecycle` set to reflect changes in registered parathreads.
@@ -175,7 +179,7 @@ UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
 * `schedule_para_cleanup(ParaId)`: Schedule a para to be cleaned up after the next full session.
 * `schedule_parathread_upgrade(ParaId)`: Schedule a parathread to be upgraded to a parachain.
 * `schedule_parachain_downgrade(ParaId)`: Schedule a parachain to be downgraded to a parathread.
-* `schedule_code_upgrade(ParaId, ValidationCode, expected_at: BlockNumber)`: Schedule a future code
+* `schedule_code_upgrade(ParaId, CurrentCode, expected_at: BlockNumber)`: Schedule a future code
   upgrade of the given parachain, to be applied after inclusion of a block of the same parachain
   executed in the context of a relay-chain block with number >= `expected_at`.
 * `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head,
@@ -187,6 +191,7 @@ UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
   intermediate parablock has been included at the given relay-chain height. This may return past,
   current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if
   provided, must be before `at`. If the validation code has been pruned, this will return `None`.
+* `validation_code_hash_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Just like `validation_code_at`, but returns the code hash.
 * `lifecycle(ParaId) -> Option<ParaLifecycle>`: Return the `ParaLifecycle` of a para.
 * `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live parachain,
   including those which may be transitioning to a parathread in the future.
@@ -197,9 +202,6 @@ UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
 * `last_code_upgrade(id: ParaId, include_future: bool) -> Option<BlockNumber>`: The block number of
   the last scheduled upgrade of the requested para. Includes future upgrades if the flag is set.
   This is the `expected_at` number, not the `activated_at` number.
-* `persisted_validation_data(id: ParaId) -> Option<PersistedValidationData>`: Get the
-  PersistedValidationData of the given para, assuming the context is the parent block. Returns
-  `None` if the para is not known.
 
 ## Finalization