Implement Approval Voting Subsystem (#2112)

* skeleton * skeleton aux-schema module * start approval types * start aux schema with aux store * doc * finish basic types * start approval types * doc * finish basic types * write out schema types * add debug and codec impls to approval types * add debug and codec impls to approval types also add some key computation * add debug and codec impls to approval types * getters for block and candidate entries * grumbles * remove unused AssignmentId * load_decode utility * implement DB clearing * function for adding new block entry to aux store * start `canonicalize` implementation * more skeleton * finish implementing canonicalize * tag TODO * implement a test AuxStore * add allow(unused) * basic loading and deleting test * block_entry test function * add a test for `add_block_entry` * ensure range is exclusive at end * test clear() * test that add_block sets children * add a test for canonicalize * extract Pre-digest from header * utilities for extracting RelayVRFStory from the header-chain * add approval voting message types * approval distribution message type * subsystem skeleton * state struct * add futures-timer * prepare service for babe slot duration * more skeleton * better integrate AuxStore * RelayVRF -> RelayVRFStory * canonicalize * implement some tick functionality * guide: tweaks * check_approval * more tweaks and helpers * guide: add core index to candidate event * primitives: add core index to candidate event * runtime: add core index to candidate events * head handling (session window) * implement `determine_new_blocks` * add TODO * change error type on functions * compute RelayVRFModulo assignments * compute RelayVRFDelay assignments * fix delay tranche calc * assignment checking * pluralize * some dummy code for fetching assignments * guide: add babe epoch runtime API * implement a current_epoch() runtime API * compute assignments * candidate events get backing group * import blocks and assignments into DB * push block approval meta * add message types, no overseer integration yet * notify approval distribution of new blocks * refactor import into separate functions * impl tranches_to_approve * guide: improve function signatures * guide: remove Tick from ApprovalEntry * trigger and broadcast assignment * most of approval launching * remove byteorder crate * load blocks back to finality, except on startup * check unchecked assignments * add claimed core to approval voting message * fix checks * assign only to backing group * remove import_checked_assignment from guide * newline * import assignments * abstract out a bit * check and import approvals * check full approvals from assignment import too * comment * create a Transaction utility * must_use * use transaction in `check_full_approvals` * wire up wakeups * add Ord to CandidateHash * wakeup refactoring * return candidate info from add_block_entry * schedule wakeups * background task: do candidate validation * forward candidate validation requests * issue approval votes when requested * clean up a couple TODOs * fix up session caching * clean up last unimplemented!() items * fix remaining warnings * remove TODO * implement handle_approved_ancestor * update Cargo.lock * fix runtime API tests * guide: cleanup assignment checking * use claimed candidate index instead of core * extract time to a trait * tests module * write a mock clock for testing * allow swapping out the clock * make abstract over assignment criteria * add some skeleton tests and simplify params * fix backing group check * do backing group check inside check_assignment_cert * write some empty test functions to implement * add a test for non-backing * test that produced checks pass * some empty test ideas * runtime/inclusion: remove outdated TODO * fix compilation * av-store: fix tests * dummy cert * criteria tests * move `TestStore` to main tests file * fix unused warning * test harness beginnings * resolve slots renaming fallout * more compilation fixes * wip: extract pure data into a separate module * wip: extract pure data into a separate module * move types completely to v1 * add persisted_entries * add conversion trait impls * clean up some warnings * extract import logic to own module * schedule wakeups * experiment with Actions * uncomment approval-checking * separate module for approval checking utilities * port more code to use actions * get approval pipeline using actions * all logic is uncommented * main loop processes actions * all loop logic uncommented * separate function for handling actions * remove last unimplemented item * clean up warnings * State gives read-only access to underlying DB * tests for approval checking * tests for approval criteria * skeleton test module for import * list of import tests to do * some test glue code * test reject bad assignment * test slot too far in future * test reject assignment with unknown candidate * remove loads_blocks tests * determine_new_blocks back to finalized & harness * more coverage for determining new blocks * make `imported_block_info` have less reliance on State * candidate_info tests * tests for session caching * remove println * extricate DB and main TestStores * rewrite approval checking logic to counteract early delays * move state out of function * update approval-checking tests * tweak wakeups & scheduling logic * rename check_full_approvals * test that assignment import updates candidate * some approval import tests * some tests for check_and_apply_approval * add 'full' qualifier to avoid confusion * extract should-trigger logic to separate function * some tests for all triggering * tests for when we trigger assignments * test wakeups * add block utilities for testing * some more tests for approval updates * approved_ancestor tests * new action type for launch approval * process-wakeup tests * clean up some warnings * fix in_future test * approval checking tests * tighten up too-far-in-future * special-case genesis when caching sessions * fix bitfield len Co-authored-by: Andronik Ordian <write@reusable.software>
2026-06-13 10:31:04 +00:00 · 2021-02-11 10:21:47 -06:00
parent 09eadfc979
commit e48c687504
29 changed files with 7505 additions and 132 deletions
@@ -16,29 +16,44 @@

 //! Types relevant for approval.

-pub use sp_consensus_vrf::schnorrkel::{VRFOutput, VRFProof};
-pub use sp_consensus_slots::Slot;
+pub use sp_consensus_vrf::schnorrkel::{VRFOutput, VRFProof, Randomness};
+pub use sp_consensus_babe::Slot;

 use polkadot_primitives::v1::{
-	CandidateHash, Hash, ValidatorIndex, Signed, ValidatorSignature, CoreIndex,
-	BlockNumber, CandidateIndex,
+	CandidateHash, Hash, ValidatorIndex, ValidatorSignature, CoreIndex,
+	Header, BlockNumber, CandidateIndex,
 };
 use parity_scale_codec::{Encode, Decode};
+use sp_consensus_babe as babe_primitives;
+use sp_application_crypto::Public;

 /// Validators assigning to check a particular candidate are split up into tranches.
 /// Earlier tranches of validators check first, with later tranches serving as backup.
 pub type DelayTranche = u32;

+/// A static context used to compute the Relay VRF story based on the
+/// VRF output included in the header-chain.
+pub const RELAY_VRF_STORY_CONTEXT: &[u8] = b"A&V RC-VRF";
+
 /// A static context used for all relay-vrf-modulo VRFs.
 pub const RELAY_VRF_MODULO_CONTEXT: &[u8] = b"A&V MOD";

-/// A static context used for all relay-vrf-delay VRFs.
-pub const RELAY_VRF_DELAY_CONTEXT: &[u8] = b"A&V TRANCHE";
+/// A static context used for all relay-vrf-modulo VRFs.
+pub const RELAY_VRF_DELAY_CONTEXT: &[u8] = b"A&V DELAY";
+
+/// A static context used for transcripts indicating assigned availability core.
+pub const ASSIGNED_CORE_CONTEXT: &[u8] = b"A&V ASSIGNED";
+
+/// A static context associated with producing randomness for a core.
+pub const CORE_RANDOMNESS_CONTEXT: &[u8] = b"A&V CORE";
+
+/// A static context associated with producing randomness for a tranche.
+pub const TRANCHE_RANDOMNESS_CONTEXT: &[u8] = b"A&V TRANCHE";

 /// random bytes derived from the VRF submitted within the block by the
 /// block author as a credential and used as input to approval assignment criteria.
 #[derive(Debug, Clone, Encode, Decode, PartialEq)]
-pub struct RelayVRF(pub [u8; 32]);
+pub struct RelayVRFStory(pub [u8; 32]);

 /// Different kinds of input data or criteria that can prove a validator's assignment
 /// to check a particular parachain.
@@ -87,9 +102,6 @@ pub struct IndirectAssignmentCert {
 #[derive(Debug, Clone, Encode, Decode)]
 pub struct ApprovalVote(pub CandidateHash);

-/// An approval vote signed by some validator.
-pub type SignedApprovalVote = Signed<ApprovalVote>;
-
 /// A signed approval vote which references the candidate indirectly via the block.
 ///
 /// In practice, we have a look-up from block hash and candidate index to candidate hash,
@@ -121,3 +133,84 @@ pub struct BlockApprovalMeta {
 	/// The consensus slot of the block.
 	pub slot: Slot,
 }
+
+/// Errors that can occur during the approvals protocol.
+#[derive(Debug, thiserror::Error)]
+#[allow(missing_docs)]
+pub enum ApprovalError {
+	#[error("Schnorrkel signature error")]
+	SchnorrkelSignature(schnorrkel::errors::SignatureError),
+	#[error("Authority index {0} out of bounds")]
+	AuthorityOutOfBounds(usize),
+}
+
+/// An unsafe VRF output. Provide BABE Epoch info to create a `RelayVRFStory`.
+pub struct UnsafeVRFOutput {
+	vrf_output: VRFOutput,
+	slot: Slot,
+	authority_index: u32,
+}
+
+impl UnsafeVRFOutput {
+	/// Get the slot.
+	pub fn slot(&self) -> Slot {
+		self.slot
+	}
+
+	/// Compute the randomness associated with this VRF output.
+	pub fn compute_randomness(
+		self,
+		authorities: &[(babe_primitives::AuthorityId, babe_primitives::BabeAuthorityWeight)],
+		randomness: &babe_primitives::Randomness,
+		epoch_index: u64,
+	) -> Result<RelayVRFStory, ApprovalError> {
+		let author = match authorities.get(self.authority_index as usize) {
+			None => return Err(ApprovalError::AuthorityOutOfBounds(self.authority_index as _)),
+			Some(x) => &x.0,
+		};
+
+		let pubkey = schnorrkel::PublicKey::from_bytes(author.as_slice())
+			.map_err(ApprovalError::SchnorrkelSignature)?;
+
+		let transcript = babe_primitives::make_transcript(
+			randomness,
+			self.slot,
+			epoch_index,
+		);
+
+		let inout = self.vrf_output.0.attach_input_hash(&pubkey, transcript)
+			.map_err(ApprovalError::SchnorrkelSignature)?;
+		Ok(RelayVRFStory(inout.make_bytes(RELAY_VRF_STORY_CONTEXT)))
+	}
+}
+
+/// Extract the slot number and relay VRF from a header.
+///
+/// This fails if either there is no BABE `PreRuntime` digest or
+/// the digest has type `SecondaryPlain`, which Substrate nodes do
+/// not produce or accept anymore.
+pub fn babe_unsafe_vrf_info(header: &Header) -> Option<UnsafeVRFOutput> {
+	use babe_primitives::digests::{CompatibleDigestItem, PreDigest};
+
+	for digest in &header.digest.logs {
+		if let Some(pre) = digest.as_babe_pre_digest() {
+			let slot = pre.slot();
+			let authority_index = pre.authority_index();
+
+			// exhaustive match to defend against upstream variant changes.
+			let vrf_output = match pre {
+				PreDigest::Primary(primary) => primary.vrf_output,
+				PreDigest::SecondaryVRF(secondary) => secondary.vrf_output,
+				PreDigest::SecondaryPlain(_) => return None,
+			};
+
+			return Some(UnsafeVRFOutput {
+				vrf_output,
+				slot,
+				authority_index,
+			});
+		}
+	}
+
+	None
+}
@@ -32,6 +32,7 @@ use polkadot_primitives::v1::{
 use std::pin::Pin;

 pub use sp_core::traits::SpawnNamed;
+pub use sp_consensus_babe::Epoch as BabeEpoch;

 pub mod approval;