Implement Runtime APIs (#1411)

* create a README on Runtime APIs

* add ParaId type

* write up runtime APIs

* more preamble

* rename

* rejig runtime APIs

* add occupied_since to `BlockNumber`

* skeleton crate for runtime API subsystem

* improve group_for_core

* improve docs on availability cores runtime API

* guide: freed -> free

* add primitives for runtime APIs

* create a v1 ParachainHost API trait

* guide: make validation code return `Option`al.

* skeleton runtime API helpers

* make parachain-host runtime-generic

* skeleton for most runtime API implementation functions

* guide: add runtime API helper methods

* implement new helpers of the inclusion module

* guide: remove retries check, as it is unneeded

* implement helpers for scheduler module for Runtime APIs

* clean up `validator_groups` implementation

* implement next_rotation_at and last_rotation_at

* guide: more helpers on GroupRotationInfo

* almost finish implementing runtime APIs

* add explicit block parameter to runtime API fns

* guide: generalize number parameter

* guide: add group_responsible to occupied-core

* update primitives due to guide changes

* finishing touches on runtime API implementation; squash warnings

* break out runtime API impl to separate file

* add tests for next_up logic

* test group rotation info

* point to filed TODO

* remove unused TODO [now]

* indentation

* guide: para -> para_id

* rename para field to para_id for core meta

* remove reference to outdated AvailabilityCores type

* add an event in `inclusion` for candidates being included or timing out

* guide: candidate events

* guide: adjust language

* Candidate events type from guide and adjust inclusion event

* implement `candidate_events` runtime API

* fix runtime test compilation

* max -> min

* fix typos

* guide: add `RuntimeAPIRequest::CandidateEvents`

polkadot/roadmap/implementers-guide/src/runtime-api/README.md

@@ -59,7 +59,19 @@ struct GroupRotationInfo {
 impl GroupRotationInfo {
 	/// Returns the index of the group needed to validate the core at the given index,
 	/// assuming the given amount of cores/groups.
-	fn group_for_core(core_index: usize, cores: usize) -> usize;
+	fn group_for_core(&self, core_index, cores) -> GroupIndex;
+
+	/// Returns the block number of the next rotation after the current block. If the current block
+	/// is 10 and the rotation frequency is 5, this should return 15.
+	///
+	/// If the group rotation frequency is 0, returns 0.
+	fn next_rotation_at(&self) -> BlockNumber;
+
+	/// Returns the block number of the last rotation before or including the current block. If the
+	/// current block is 10 and the rotation frequency is 5, this should return 10.
+	///
+	/// If the group rotation frequency is 0, returns 0.
+	fn last_rotation_at(&self) -> BlockNumber;
 }
 
 /// Returns the validator groups and rotation info localized based on the block whose state
@@ -81,7 +93,7 @@ This is all the information that a validator needs about scheduling for the curr
 ```rust
 struct OccupiedCore {
 	/// The ID of the para occupying the core.
-	para: ParaId,
+	para_id: ParaId,
 	/// If this core is freed by availability, this is the assignment that is next up on this
 	/// core, if any. None if there is nothing queued for this core.
 	next_up_on_available: Option<ScheduledCore>,
@@ -97,11 +109,13 @@ struct OccupiedCore {
 	/// validators has attested to availability on-chain. A 2/3+ majority of `1` bits means that
 	/// this will be available.
 	availability: Bitfield,
+	/// The group assigned to distribute availability pieces of this candidate.
+	group_responsible: GroupIndex,
 }
 
 struct ScheduledCore {
 	/// The ID of a para scheduled.
-	para: ParaId,
+	para_id: ParaId,
 	/// The collator required to author the block, if any.
 	collator: Option<CollatorId>,
 }
@@ -171,7 +185,7 @@ fn session_index_for_child(at: Block) -> SessionIndex;
 Fetch the validation code used by a para, making the given `OccupiedCoreAssumption`.
 
 ```rust
-fn validation_code(at: Block, ParaId, OccupiedCoreAssumption) -> ValidationCode;
+fn validation_code(at: Block, ParaId, OccupiedCoreAssumption) -> Option<ValidationCode>;
 ```
 
 ## Candidate Pending Availability
@@ -181,3 +195,20 @@ Get the receipt of a candidate pending availability. This returns `Some` for any
 ```rust
 fn candidate_pending_availability(at: Block, ParaId) -> Option<CommittedCandidateReceipt>;
 ```
+
+## Candidate Events
+
+Yields a vector of events concerning candidates that occurred within the given block.
+
+```rust
+enum CandidateEvent {
+	/// This candidate receipt was backed in the most recent block.
+	CandidateBacked(CandidateReceipt, HeadData),
+	/// This candidate receipt was included and became a parablock at the most recent block.
+	CandidateIncluded(CandidateReceipt, HeadData),
+	/// This candidate receipt was not made available in time and timed out.
+	CandidateTimedOut(CandidateReceipt, HeadData),
+}
+
+fn candidate_events(at: Block) -> Vec<CandidateEvent>;
+```

polkadot/roadmap/implementers-guide/src/runtime/inclusion.md

@@ -84,3 +84,6 @@ All failed checks should lead to an unrecoverable error making the block invalid
       // return a vector of cleaned-up core IDs.
     }
   ```
+* `force_enact(ParaId)`: Forcibly enact the candidate with the given ID as though it had been deemed available by bitfields. Is a no-op if there is no candidate pending availability for this para-id. This should generally not be used but it is useful during execution of Runtime APIs, where the changes to the state are expected to be discarded directly after.
+* `candidate_pending_availability(ParaId) -> Option<CommittedCandidateReceipt>`: returns the `CommittedCandidateReceipt` pending availability for the para provided, if any.
+* `pending_availability(ParaId) -> Option<CandidatePendingAvailability>`: returns the metadata around the candidate pending availability for the para, if any.

polkadot/roadmap/implementers-guide/src/runtime/scheduler.md

@@ -198,3 +198,6 @@ Actions:
 - `core_para(CoreIndex) -> ParaId`: return the currently-scheduled or occupied ParaId for the given core.
 - `group_validators(GroupIndex) -> Option<Vec<ValidatorIndex>>`: return all validators in a given group, if the group index is valid for this session.
 - `availability_timeout_predicate() -> Option<impl Fn(CoreIndex, BlockNumber) -> bool>`: returns an optional predicate that should be used for timing out occupied cores. if `None`, no timing-out should be done. The predicate accepts the index of the core, and the block number since which it has been occupied. The predicate should be implemented based on the time since the last validator group rotation, and the respective parachain and parathread timeouts, i.e. only within `max(config.chain_availability_period, config.thread_availability_period)` of the last rotation would this return `Some`.
+- `group_rotation_info() -> GroupRotationInfo`: Returns a helper for determining group rotation.
+- `next_up_on_available(CoreIndex) -> Option<ScheduledCore>`: Return the next thing that will be scheduled on this core assuming it is currently occupied and the candidate occupying it became available. Returns in `ScheduledCore` format (todo: link to Runtime APIs page; linkcheck doesn't allow this right now). For parachains, this is always the ID of the parachain and no specified collator. For parathreads, this is based on the next item in the `ParathreadQueue` assigned to that core, and is `None` if there isn't one.
+- `next_up_on_time_out(CoreIndex) -> Option<ScheduledCore>`: Return the next thing that will be scheduled on this core assuming it is currently occupied and the candidate occupying it timed out. Returns in `ScheduledCore` format (todo: link to Runtime APIs page; linkcheck doesn't allow this right now). For parachains, this is always the ID of the parachain and no specified collator. For parathreads, this is based on the next item in the `ParathreadQueue` assigned to that core, or if there isn't one, the claim that is currently occupying the core. Otherwise `None`.

polkadot/roadmap/implementers-guide/src/types/overseer-protocol.md

@@ -252,9 +252,11 @@ enum RuntimeApiRequest {
 		ResponseChannel<Option<LocalValidationData>>,
 	),
 	/// Get information about all availability cores.
-	AvailabilityCores(ResponseChannel<AvailabilityCores>),
+	AvailabilityCores(ResponseChannel<Vec<CoreState>>),
 	/// Get a committed candidate receipt for all candidates pending availability.
 	CandidatePendingAvailability(ParaId, ResponseChannel<Option<CommittedCandidateReceipt>>),
+	/// Get all events concerning candidates in the last block.
+	CandidateEvents(ResponseChannel<Vec<CandidateEvent>>),
 }
 
 enum RuntimeApiMessage {