BABE Epochs (#3028)

* Add `epoch` field to `SlotInfo` * Add slot calculations * More work on epochs in BABE * Apply suggestions from code review Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com> * Typo: `/` not `%` for division * Delete useless `LastSlotInEpoch::put(false)` * Bump `spec_version` * Make test suite pass again * Implement BABE epoch randomness signing * Try to fix compilation Currently causes a stack overflow in the compiler * Fix rustc stack overflow * Add missing `PartialEq` and `Eq` implementations * Fix compile errors in test suite * Another silly compile error * Clone `epoch` * Fix compile error in benchmarks * Implement `clone` for `Epoch` * Merge master * AUTHORING TEST PASSES!!! * Fix compilation * Bump `spec_version` * Fix compilation * Fix compilation (again) * Remove an outdated FIXME * Fix run.sh and move it to scripts/ * Delete commented-out code * Fix documentation Co-Authored-By: André Silva <andre.beat@gmail.com> * Fix BABE initialization and refactor * Respond to review * typo * Remove useless data in `CheckedHeader::Deferred` * Remove `slot_number` from Epoch It is not needed, and only served to waste space and cause confusion. * Remove epoch from BABE digests * Move digest.rs to primitives * Fix incorrect warning names * Fix compile error * Consistent field naming for BABE digests * More compiler error fixex * Unbound variable * more compile errors * another compile error * Fix compile errors in runtime * another compile error * Another compile error * Fix wasm build * missing import * Fix more compile errors * yet another compile error * compile fix in test runtime * Fix and simplify the BABE runtime The BABE runtime was massively overcomplicated and also wrong. It assumed it needed to: 1. delay new authorities taking effect until the next epoch 2. not delay emitting `Consensus` digests to mark epoch changes However, the first is handled by the `srml_session` crate, and the second is flat-out incorrect: `Consensus` digests take effect immediately. Furthermore, `srml_babe` tried to duplicate the functionality of `srml_session::PeriodicSession`, but did it both clumsily and incorrectly. Fortunately, the new code is simpler and far more likely to be correct. * Use `system` to get the test authorities The genesis block used by tests defines no authorities. Only the test suite is affected. * Fix test runtime impl for BabeApi::epoch() with std * Fix compilation * Cached authorities are in the form of an epoch not a `Vec<AuthorityId>`. * `slots_per_epoch` is not fixed in general The BABE code previously assumed `slots_per_epoch` to be a constant, but that assumption is false in general. Furthermore, removing this assumption also allows a lot of code to go away. * fix compile error * Implement epoch checker * Fix runtime compilation * fork-tree: add method for finding a node in the tree * babe: register epoch transitions in fork tree and validate them * fork-tree: add method for arbitrary pruning * Expose the queued validator set to SRML modules BABE needs to know not only what the current validator set is, but also what the next validator set will be. Expose this to clients of the session module. * Bump hex-literal Hopefully this will fix the panic * babe: prune epoch change fork tree on finality * babe: validate epoch index on transition * babe: persist epoch changes tree * Fix compile error in tests * Fix compile error in tests * Another compile error in tests * Fix compilation of tests * core: move grandpa::is_descendent_of to client utils * babe: use is_descendent_of from client utils * babe: extract slot_number from pre_digest in import_block * Move BABE testsuite to its own file * Initial part of test code * Missing `WeightMultiplierUpdate` in test-runtime * bump `spec_version` * Add a test that a very bogus is rejected * Run the tests again * Fix compiler diagnostics * Bump `spec_version` * Initial infrastructure for mutation testing * Mutation testing of block import * babe: revert epoch changes in case of block import error * babe: fix logging target * babe: BabeBlockImport doesn't box inner BlockImport * babe: fix epoch check in block import * babe: populate authorities cache on block authorship * babe: remove unused functions * babe: use RANDOMNESS_LENGTH const * babe: remove unneeded config parameters * core: revert change to hex dependency version * cleanup gitignore * babe: add docs to aux_schema * babe: remove useless drops in tests * babe: remove annoying macos smart quotes * fork-tree: docs * fork-tree: add tests * babe: style * babe: rename randomness config variable * babe: remove randomness helper function * babe: style fixes * babe: add docs * babe: fix tests * node: bump spec_version * babe: fix tests
2026-06-13 21:01:05 +00:00 · 2019-07-23 04:36:16 -04:00
parent 78bc5edc14
commit f78a780790
30 changed files with 1368 additions and 571 deletions
@@ -0,0 +1,69 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see <http://www.gnu.org/licenses/>.
+
+//! Schema for BABE epoch changes in the aux-db.
+
+use log::info;
+use parity_codec::{Decode, Encode};
+
+use client::backend::AuxStore;
+use client::error::{Result as ClientResult, Error as ClientError};
+use runtime_primitives::traits::Block as BlockT;
+
+use super::{EpochChanges, SharedEpochChanges};
+
+const BABE_EPOCH_CHANGES: &[u8] = b"babe_epoch_changes";
+
+fn load_decode<B, T>(backend: &B, key: &[u8]) -> ClientResult<Option<T>>
+	where
+		B: AuxStore,
+		T: Decode,
+{
+	let corrupt = || ClientError::Backend(format!("BABE DB is corrupted.")).into();
+	match backend.get_aux(key)? {
+		None => Ok(None),
+		Some(t) => T::decode(&mut &t[..]).ok_or_else(corrupt).map(Some)
+	}
+}
+
+/// Load or initialize persistent epoch change data from backend.
+pub(crate) fn load_epoch_changes<Block: BlockT, B: AuxStore>(
+	backend: &B,
+) -> ClientResult<SharedEpochChanges<Block>> {
+	let epoch_changes = load_decode::<_, EpochChanges<Block>>(backend, BABE_EPOCH_CHANGES)?
+		.map(Into::into)
+		.unwrap_or_else(|| {
+			info!(target: "babe",
+				"Creating empty BABE epoch changes on what appears to be first startup."
+			);
+			SharedEpochChanges::new()
+		});
+
+	Ok(epoch_changes)
+}
+
+/// Update the epoch changes on disk after a change.
+pub(crate) fn write_epoch_changes<Block: BlockT, F, R>(
+	epoch_changes: &EpochChanges<Block>,
+	write_aux: F,
+) -> R where
+	F: FnOnce(&[(&'static [u8], &[u8])]) -> R,
+{
+	let encoded_epoch_changes = epoch_changes.encode();
+	write_aux(
+		&[(BABE_EPOCH_CHANGES, encoded_epoch_changes.as_slice())],
+	)
+}
@@ -1,111 +0,0 @@
-// Copyright 2019 Parity Technologies (UK) Ltd.
-// This file is part of Substrate.
-
-// Substrate is free software: you can redistribute it and/or modify
-// it under the terms of the GNU General Public License as published by
-// the Free Software Foundation, either version 3 of the License, or
-// (at your option) any later version.
-
-// Substrate is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU General Public License for more details.
-
-// You should have received a copy of the GNU General Public License
-// along with Substrate.  If not, see <http://www.gnu.org/licenses/>.
-
-//! Private implementation details of BABE digests.
-
-use primitives::sr25519::Signature;
-use babe_primitives::{self, BABE_ENGINE_ID, SlotNumber};
-use runtime_primitives::{DigestItem, generic::OpaqueDigestItemId};
-use std::fmt::Debug;
-use parity_codec::{Decode, Encode, Codec, Input};
-use schnorrkel::{vrf::{VRFProof, VRFOutput, VRF_OUTPUT_LENGTH, VRF_PROOF_LENGTH}};
-
-/// A BABE pre-digest.  It includes:
-///
-/// * The public key of the author.
-/// * The VRF proof.
-/// * The VRF output.
-/// * The slot number.
-#[derive(Clone, Debug, PartialEq, Eq)]
-pub struct BabePreDigest {
-	pub(super) vrf_output: VRFOutput,
-	pub(super) proof: VRFProof,
-	pub(super) index: babe_primitives::AuthorityIndex,
-	pub(super) slot_num: SlotNumber,
-}
-
-/// The prefix used by BABE for its VRF keys.
-pub const BABE_VRF_PREFIX: &'static [u8] = b"substrate-babe-vrf";
-
-type RawBabePreDigest = (
-	[u8; VRF_OUTPUT_LENGTH],
-	[u8; VRF_PROOF_LENGTH],
-	u64,
-	u64,
-);
-
-impl Encode for BabePreDigest {
-	fn encode(&self) -> Vec<u8> {
-		let tmp: RawBabePreDigest = (
-			*self.vrf_output.as_bytes(),
-			self.proof.to_bytes(),
-			self.index,
-			self.slot_num,
-		);
-		parity_codec::Encode::encode(&tmp)
-	}
-}
-
-impl Decode for BabePreDigest {
-	fn decode<R: Input>(i: &mut R) -> Option<Self> {
-		let (output, proof, index, slot_num): RawBabePreDigest = Decode::decode(i)?;
-
-		// Verify (at compile time) that the sizes in babe_primitives are correct
-		let _: [u8; babe_primitives::VRF_OUTPUT_LENGTH] = output;
-		let _: [u8; babe_primitives::VRF_PROOF_LENGTH] = proof;
-		Some(BabePreDigest {
-			proof: VRFProof::from_bytes(&proof).ok()?,
-			vrf_output: VRFOutput::from_bytes(&output).ok()?,
-			index,
-			slot_num,
-		})
-	}
-}
-
-/// A digest item which is usable with BABE consensus.
-pub trait CompatibleDigestItem: Sized {
-	/// Construct a digest item which contains a BABE pre-digest.
-	fn babe_pre_digest(seal: BabePreDigest) -> Self;
-
-	/// If this item is an BABE pre-digest, return it.
-	fn as_babe_pre_digest(&self) -> Option<BabePreDigest>;
-
-	/// Construct a digest item which contains a BABE seal.
-	fn babe_seal(signature: Signature) -> Self;
-
-	/// If this item is a BABE signature, return the signature.
-	fn as_babe_seal(&self) -> Option<Signature>;
-}
-
-impl<Hash> CompatibleDigestItem for DigestItem<Hash> where
-	Hash: Debug + Send + Sync + Eq + Clone + Codec + 'static
-{
-	fn babe_pre_digest(digest: BabePreDigest) -> Self {
-		DigestItem::PreRuntime(BABE_ENGINE_ID, digest.encode())
-	}
-
-	fn as_babe_pre_digest(&self) -> Option<BabePreDigest> {
-		self.try_to(OpaqueDigestItemId::PreRuntime(&BABE_ENGINE_ID))
-	}
-
-	fn babe_seal(signature: Signature) -> Self {
-		DigestItem::Seal(BABE_ENGINE_ID, signature.encode())
-	}
-
-	fn as_babe_seal(&self) -> Option<Signature> {
-		self.try_to(OpaqueDigestItemId::Seal(&BABE_ENGINE_ID))
-	}
-}
@@ -0,0 +1,340 @@
+// Copyright 2019 Parity Technologies (UK) Ltd.
+// This file is part of Substrate.
+
+// Substrate is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Substrate is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Substrate.  If not, see <http://www.gnu.org/licenses/>.
+
+//! BABE testsuite
+
+// FIXME #2532: need to allow deprecated until refactor is done
+// https://github.com/paritytech/substrate/issues/2532
+#![allow(deprecated)]
+use super::*;
+
+use client::{LongestChain, block_builder::BlockBuilder};
+use consensus_common::NoNetwork as DummyOracle;
+use network::test::*;
+use network::test::{Block as TestBlock, PeersClient};
+use runtime_primitives::traits::{Block as BlockT, DigestFor};
+use network::config::ProtocolConfig;
+use tokio::runtime::current_thread;
+use keyring::sr25519::Keyring;
+use super::generic::DigestItem;
+use client::BlockchainEvents;
+use test_client;
+use futures::Async;
+use log::debug;
+use std::{time::Duration, borrow::Borrow, cell::RefCell};
+type Item = generic::DigestItem<Hash>;
+
+type Error = client::error::Error;
+
+type TestClient = client::Client<
+	test_client::Backend,
+	test_client::Executor,
+	TestBlock,
+	test_client::runtime::RuntimeApi,
+>;
+
+struct DummyFactory(Arc<TestClient>);
+struct DummyProposer(u64, Arc<TestClient>);
+
+impl Environment<TestBlock> for DummyFactory {
+	type Proposer = DummyProposer;
+	type Error = Error;
+
+	fn init(&self, parent_header: &<TestBlock as BlockT>::Header)
+		-> Result<DummyProposer, Error>
+	{
+		Ok(DummyProposer(parent_header.number + 1, self.0.clone()))
+	}
+}
+
+impl Proposer<TestBlock> for DummyProposer {
+	type Error = Error;
+	type Create = Result<TestBlock, Error>;
+
+	fn propose(&self, _: InherentData, digests: DigestFor<TestBlock>, _: Duration) -> Result<TestBlock, Error> {
+		self.1.new_block(digests).unwrap().bake().map_err(|e| e.into())
+	}
+}
+
+type Mutator = Arc<dyn for<'r> Fn(&'r mut TestHeader) + Send + Sync>;
+
+thread_local! {
+	static MUTATOR: RefCell<Mutator> = RefCell::new(Arc::new(|_|()));
+}
+
+pub struct BabeTestNet {
+	peers: Vec<Peer<(), DummySpecialization>>,
+}
+
+type TestHeader = <TestBlock as BlockT>::Header;
+type TestExtrinsic = <TestBlock as BlockT>::Extrinsic;
+
+pub struct TestVerifier {
+	inner: BabeVerifier<PeersFullClient>,
+	mutator: Mutator,
+}
+
+impl Verifier<TestBlock> for TestVerifier {
+	/// Verify the given data and return the BlockImportParams and an optional
+	/// new set of validators to import. If not, err with an Error-Message
+	/// presented to the User in the logs.
+	fn verify(
+		&self,
+		origin: BlockOrigin,
+		mut header: TestHeader,
+		justification: Option<Justification>,
+		body: Option<Vec<TestExtrinsic>>,
+	) -> Result<(BlockImportParams<TestBlock>, Option<Vec<(CacheKeyId, Vec<u8>)>>), String> {
+		let cb: &(dyn Fn(&mut TestHeader) + Send + Sync) = self.mutator.borrow();
+		cb(&mut header);
+		Ok(self.inner.verify(origin, header, justification, body).expect("verification failed!"))
+	}
+}
+
+impl TestNetFactory for BabeTestNet {
+	type Specialization = DummySpecialization;
+	type Verifier = TestVerifier;
+	type PeerData = ();
+
+	/// Create new test network with peers and given config.
+	fn from_config(_config: &ProtocolConfig) -> Self {
+		debug!(target: "babe", "Creating test network from config");
+		BabeTestNet {
+			peers: Vec::new(),
+		}
+	}
+
+	/// KLUDGE: this function gets the mutator from thread-local storage.
+	fn make_verifier(&self, client: PeersClient, _cfg: &ProtocolConfig)
+		-> Arc<Self::Verifier>
+	{
+		let api = client.as_full().expect("only full clients are used in test");
+		trace!(target: "babe", "Creating a verifier");
+		let config = Config::get_or_compute(&*api)
+			.expect("slot duration available");
+		let inherent_data_providers = InherentDataProviders::new();
+		register_babe_inherent_data_provider(
+			&inherent_data_providers,
+			config.get()
+		).expect("Registers babe inherent data provider");
+		trace!(target: "babe", "Provider registered");
+
+		Arc::new(TestVerifier {
+			inner: BabeVerifier {
+				api,
+				inherent_data_providers,
+				config,
+				time_source: Default::default(),
+			},
+			mutator: MUTATOR.with(|s| s.borrow().clone()),
+		})
+	}
+
+	fn peer(&mut self, i: usize) -> &mut Peer<Self::PeerData, DummySpecialization> {
+		trace!(target: "babe", "Retreiving a peer");
+		&mut self.peers[i]
+	}
+
+	fn peers(&self) -> &Vec<Peer<Self::PeerData, DummySpecialization>> {
+		trace!(target: "babe", "Retreiving peers");
+		&self.peers
+	}
+
+	fn mut_peers<F: FnOnce(&mut Vec<Peer<Self::PeerData, DummySpecialization>>)>(
+		&mut self,
+		closure: F,
+	) {
+		closure(&mut self.peers);
+	}
+}
+
+#[test]
+fn can_serialize_block() {
+	let _ = env_logger::try_init();
+	assert!(BabePreDigest::decode(&mut &b""[..]).is_none());
+}
+
+#[test]
+#[should_panic]
+fn rejects_empty_block() {
+	env_logger::try_init().unwrap();
+	let mut net = BabeTestNet::new(3);
+	let block_builder = |builder: BlockBuilder<_, _>| {
+		builder.bake().unwrap()
+	};
+	net.mut_peers(|peer| {
+		peer[0].generate_blocks(1, BlockOrigin::NetworkInitialSync, block_builder);
+	})
+}
+
+fn run_one_test() {
+	let _ = env_logger::try_init();
+	let net = BabeTestNet::new(3);
+
+	let peers = &[
+		(0, Keyring::Alice),
+		(1, Keyring::Bob),
+		(2, Keyring::Charlie),
+	];
+
+	let net = Arc::new(Mutex::new(net));
+	let mut import_notifications = Vec::new();
+	let mut runtime = current_thread::Runtime::new().unwrap();
+	for (peer_id, key) in peers {
+		let client = net.lock().peer(*peer_id).client().as_full().unwrap();
+		let environ = Arc::new(DummyFactory(client.clone()));
+		import_notifications.push(
+			client.import_notification_stream()
+				.map(|v| Ok::<_, ()>(v)).compat()
+				.take_while(|n| Ok(n.header.number() < &5))
+				.for_each(move |_| Ok(()))
+		);
+
+		let config = Config::get_or_compute(&*client)
+			.expect("slot duration available");
+
+		let inherent_data_providers = InherentDataProviders::new();
+		register_babe_inherent_data_provider(
+			&inherent_data_providers, config.get()
+		).expect("Registers babe inherent data provider");
+
+
+		#[allow(deprecated)]
+		let select_chain = LongestChain::new(client.backend().clone());
+
+		runtime.spawn(start_babe(BabeParams {
+			config,
+			local_key: Arc::new(key.clone().into()),
+			block_import: client.clone(),
+			select_chain,
+			client,
+			env: environ.clone(),
+			sync_oracle: DummyOracle,
+			inherent_data_providers,
+			force_authoring: false,
+			time_source: Default::default(),
+		}).expect("Starts babe"));
+	}
+
+	// wait for all finalized on each.
+	let wait_for = futures::future::join_all(import_notifications);
+
+	let drive_to_completion = futures::future::poll_fn(|| { net.lock().poll(); Ok(Async::NotReady) });
+	runtime.block_on(wait_for.select(drive_to_completion).map_err(|_| ())).unwrap();
+}
+
+#[test]
+fn authoring_blocks() { run_one_test() }
+
+#[test]
+#[should_panic]
+fn rejects_missing_inherent_digest() {
+	MUTATOR.with(|s| *s.borrow_mut() = Arc::new(move |header: &mut TestHeader| {
+		let v = std::mem::replace(&mut header.digest_mut().logs, vec![]);
+		header.digest_mut().logs = v.into_iter()
+			.filter(|v| v.as_babe_pre_digest().is_none())
+			.collect()
+	}));
+	run_one_test()
+}
+
+#[test]
+#[should_panic]
+fn rejects_missing_seals() {
+	MUTATOR.with(|s| *s.borrow_mut() = Arc::new(move |header: &mut TestHeader| {
+		let v = std::mem::replace(&mut header.digest_mut().logs, vec![]);
+		header.digest_mut().logs = v.into_iter()
+			.filter(|v| v.as_babe_seal().is_none())
+			.collect()
+	}));
+	run_one_test()
+}
+
+// TODO: this test assumes that the test runtime will trigger epoch changes
+// which isn't the case since it doesn't include the session module.
+#[test]
+#[should_panic]
+#[ignore]
+fn rejects_missing_consensus_digests() {
+	MUTATOR.with(|s| *s.borrow_mut() = Arc::new(move |header: &mut TestHeader| {
+		let v = std::mem::replace(&mut header.digest_mut().logs, vec![]);
+		header.digest_mut().logs = v.into_iter()
+			.filter(|v| v.as_babe_epoch().is_none())
+			.collect()
+	}));
+	run_one_test()
+}
+
+#[test]
+fn wrong_consensus_engine_id_rejected() {
+	let _ = env_logger::try_init();
+	let sig = sr25519::Pair::generate().0.sign(b"");
+	let bad_seal: Item = DigestItem::Seal([0; 4], sig.0.to_vec());
+	assert!(bad_seal.as_babe_pre_digest().is_none());
+	assert!(bad_seal.as_babe_seal().is_none())
+}
+
+#[test]
+fn malformed_pre_digest_rejected() {
+	let _ = env_logger::try_init();
+	let bad_seal: Item = DigestItem::Seal(BABE_ENGINE_ID, [0; 64].to_vec());
+	assert!(bad_seal.as_babe_pre_digest().is_none());
+}
+
+#[test]
+fn sig_is_not_pre_digest() {
+	let _ = env_logger::try_init();
+	let sig = sr25519::Pair::generate().0.sign(b"");
+	let bad_seal: Item = DigestItem::Seal(BABE_ENGINE_ID, sig.0.to_vec());
+	assert!(bad_seal.as_babe_pre_digest().is_none());
+	assert!(bad_seal.as_babe_seal().is_some())
+}
+
+#[test]
+fn can_author_block() {
+	let _ = env_logger::try_init();
+	let randomness = &[];
+	let (pair, _) = sr25519::Pair::generate();
+	let mut i = 0;
+	let epoch = Epoch {
+		authorities: vec![(pair.public(), 0)],
+		randomness: [0; 32],
+		epoch_index: 1,
+		duration: 100,
+	};
+	loop {
+		match claim_slot(randomness, i, 0, epoch.clone(), &pair, u64::MAX / 10) {
+			None => i += 1,
+			Some(s) => {
+				debug!(target: "babe", "Authored block {:?}", s);
+				break
+			}
+		}
+	}
+}
+
+#[test]
+fn authorities_call_works() {
+	let _ = env_logger::try_init();
+	let client = test_client::new();
+
+	assert_eq!(client.info().chain.best_number, 0);
+	assert_eq!(epoch(&client, &BlockId::Number(0)).unwrap().authorities, vec![
+		(Keyring::Alice.into(), 1),
+		(Keyring::Bob.into(), 1),
+		(Keyring::Charlie.into(), 1),
+	]);
+}