BABE signing and verification (#2270)

* Add basic BABE consensus type

* Update core/consensus/babe/slots/Cargo.toml

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Fix parameterization and run `rustfmt`

* Respond to review comments

* Update various Cargo.lock files

* Revert "Update various Cargo.lock files"

This reverts commit af53d7624752a744320e9cbb25749fdd8e6f46d2.

* `BabeSealSignature` → `BabeSeal`

* Move slot code to its own crate

This was highly non-trivial, due to cyclic dependencies.

* Remove redundancy between AuRa and BABE

Some of the code duplication was removed using a macro.

* Fix build error

* Avoid non-`#[doc(hidden)]` re-exports

Also, bump some library versions in `Cargo.toml`.

* Remove dead code in AuRa

* Remove impl_slot macro

It was more trouble than it was worth.

Also, delete useless dependencies on Serde.

* AuRa and BABE need different DB keys

* Bring back `aura::Network`, but deprecate it.

* Improve docs and add `slot_duration` inherent method

* Add docs to `substrate_consensus_aura::SlotDuration`

* Add missing documentation and #![forbid(missing_docs, unsafe_code)]

* Add a #![forbid(missing_docs)]

* Remove dependency of `test-runtime` on `slots`

* Update core/consensus/babe/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Remove wrongly added file

* Fix copyright notice

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Bump `impl_version` and `spec_version`

* Add more code to BABE

Most of it is copied from AuRa code, but at least the initial core is
there.

* Stuck on horrible compiler error message

* add missing files

* Spaces → tabs

* Simplify code

* Fix compilation

This involved fixing dependencies and adding a `Mutex`.

* More work on BABE

* Fix deprecation version

* Fix deprecation version; remove spurious carets

* Fix Cargo.toml

* Implement VRF signing logic

* The import queue code compiles, though it probably doesn’t work.

* Add VRF verification

* Update Cargo.lock

* Update dependencies

* Move test network to sr25519 authority keys

* Fix accidental build bustage

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Working testsuite at last!

The problem was with serialization and deserialization.  Normally,
those functions are generated automatically, but those for `BabeSeal`
had to be written manually.  The hand-written versions were not
correct, however, as shown by the decoder not being able to decode the
output of the encoder.

* Enable BabeSeal::Encode asserts in --release tests

* Bump runtime and dependency versions

* Fix wasm compilation

The wasm build was broken because of a typo in
`core/test-runtime/src/lib.rs`, and missing gates on the `std` feature
in `core/consensus/{aura,babe}/primitives/Cargo.toml`.  Additionally,
improve the quotation in the build scripts.

* Merge Cargo.lock

* Change expected JSON string

The test was also broken on `master`, so I suspect that the test was
incorrect.

* Responded to review

* Remove hard-coded threshold from production code

A hard-coded threshold is now only used in tests.

* Fix swapped doc comments

* Fix unused import warnings

* fix ci error

* fix typo

* Fix spacing in docs

* Minor changes suggested by @joepetrowski

on https://github.com/paritytech/substrate/pull/2372

* Remove unnecessary getters

* fix compile error

* Fix silly unused-variable error

* Improve documentation formatting

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Add issue links

* Revert excess verbosity and #![forbid(warnings)]

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Reformat some comments

* Threshold should depend on number of validators

Also, respond to code review

* Fix silly compilation errors

* Reduce logging verbosity

* Fix missing import
This commit is contained in:
DemiMarie-parity
2019-05-02 11:16:54 -04:00
committed by GitHub
parent d3f59a0712
commit fb19684358
40 changed files with 1570 additions and 263 deletions
@@ -17,12 +17,13 @@
//! Import Queue primitive: something which can verify and import blocks.
//!
//! This serves as an intermediate and abstracted step between synchronization
//! and import. Each mode of consensus will have its own requirements for block verification.
//! Some algorithms can verify in parallel, while others only sequentially.
//! and import. Each mode of consensus will have its own requirements for block
//! verification. Some algorithms can verify in parallel, while others only
//! sequentially.
//!
//! The `ImportQueue` trait allows such verification strategies to be instantiated.
//! The `BasicQueue` and `BasicVerifier` traits allow serial queues to be
//! instantiated simply.
//! The `ImportQueue` trait allows such verification strategies to be
//! instantiated. The `BasicQueue` and `BasicVerifier` traits allow serial
//! queues to be instantiated simply.
use crate::block_import::{
BlockImport, BlockOrigin, ImportBlock, ImportedAux, ImportResult, JustificationImport,
@@ -106,8 +107,8 @@ impl<B: BlockT> Clone for Box<ImportQueue<B>> {
}
}
/// Interface to a basic block import queue that is importing blocks sequentially in a separate thread,
/// with pluggable verification.
/// Interface to a basic block import queue that is importing blocks
/// sequentially in a separate thread, with pluggable verification.
#[derive(Clone)]
pub struct BasicQueue<B: BlockT> {
sender: Sender<BlockImportMsg<B>>,
@@ -120,20 +121,24 @@ impl<B: BlockT> ImportQueueClone<B> for BasicQueue<B> {
}
/// "BasicQueue" is a wrapper around a channel sender to the "BlockImporter".
/// "BasicQueue" itself does not keep any state or do any importing work, and can therefore be send to other threads.
/// "BasicQueue" itself does not keep any state or do any importing work, and
/// can therefore be send to other threads.
///
/// "BasicQueue" implements "ImportQueue" by sending messages to the "BlockImporter", which runs in it's own thread.
/// "BasicQueue" implements "ImportQueue" by sending messages to the
/// "BlockImporter", which runs in it's own thread.
///
/// The "BlockImporter" is responsible for handling incoming requests from the "BasicQueue",
/// some of these requests are handled by the "BlockImporter" itself, such as "is_importing" or "status",
/// and justifications are also imported by the "BlockImporter".
/// The "BlockImporter" is responsible for handling incoming requests from the
/// "BasicQueue". Some of these requests are handled by the "BlockImporter"
/// itself, such as "is_importing", "status", and justifications.
///
/// The "import block" work will be offloaded to a single "BlockImportWorker", running in another thread.
/// Offloading the work is done via a channel,
/// ensuring blocks in this implementation are imported sequentially and in order(as received by the "BlockImporter")
/// The "import block" work will be offloaded to a single "BlockImportWorker",
/// running in another thread. Offloading the work is done via a channel,
/// ensuring blocks in this implementation are imported sequentially and in
/// order (as received by the "BlockImporter").
///
/// As long as the "BasicQueue" is not dropped, the "BlockImporter" will keep running.
/// The "BlockImporter" owns a sender to the "BlockImportWorker", ensuring that the worker is kept alive until that sender is dropped.
/// As long as the "BasicQueue" is not dropped, the "BlockImporter" will keep
/// running. The "BlockImporter" owns a sender to the "BlockImportWorker",
/// ensuring that the worker is kept alive until that sender is dropped.
impl<B: BlockT> BasicQueue<B> {
/// Instantiate a new basic queue, with given verifier.
pub fn new<V: 'static + Verifier<B>>(
@@ -152,8 +157,8 @@ impl<B: BlockT> BasicQueue<B> {
/// Send synchronization request to the block import channel.
///
/// The caller should wait for Link::synchronized() call to ensure that it has synchronized
/// with ImportQueue.
/// The caller should wait for Link::synchronized() call to ensure that it
/// has synchronized with ImportQueue.
#[cfg(any(test, feature = "test-helpers"))]
pub fn synchronize(&self) {
self
@@ -238,6 +243,7 @@ impl<B: BlockT> BlockImporter<B> {
worker_sender: Sender<BlockImportWorkerMsg<B>>,
justification_import: Option<SharedJustificationImport<B>>,
) -> Sender<BlockImportMsg<B>> {
trace!(target: "block_import", "Creating new Block Importer!");
let (sender, port) = channel::bounded(4);
let _ = thread::Builder::new()
.name("ImportQueue".into())
@@ -258,6 +264,7 @@ impl<B: BlockT> BlockImporter<B> {
}
fn run(&mut self) -> bool {
trace!(target: "import_queue", "Running import queue");
let msg = select! {
recv(self.port) -> msg => {
match msg {
@@ -297,6 +304,7 @@ impl<B: BlockT> BlockImporter<B> {
BlockImportMsg::Stop => return false,
#[cfg(any(test, feature = "test-helpers"))]
BlockImportMsg::Synchronize => {
trace!(target: "sync", "Received synchronization message");
self.worker_sender
.send(BlockImportWorkerMsg::Synchronize)
.expect("1. This is holding a sender to the worker, 2. the worker should not quit while a sender is still held; qed");
@@ -318,6 +326,7 @@ impl<B: BlockT> BlockImporter<B> {
BlockImportWorkerMsg::Imported(results) => (results),
#[cfg(any(test, feature = "test-helpers"))]
BlockImportWorkerMsg::Synchronize => {
trace!(target: "sync", "Synchronizing link");
link.synchronized();
return true;
},
@@ -434,6 +443,7 @@ impl<B: BlockT, V: 'static + Verifier<B>> BlockImportWorker<B, V> {
},
#[cfg(any(test, feature = "test-helpers"))]
BlockImportWorkerMsg::Synchronize => {
trace!(target: "sync", "Sending sync message");
let _ = worker.result_sender.send(BlockImportWorkerMsg::Synchronize);
},
_ => unreachable!("Import Worker does not receive the Imported message; qed"),
@@ -118,6 +118,20 @@ impl<T: SyncOracle> SyncOracle for Arc<T> {
}
}
/// Extra verification for blocks.
pub trait ExtraVerification<B: Block>: Send + Sync {
/// Future that resolves when the block is verified, or fails with error if
/// not.
type Verified: IntoFuture<Item=(),Error=String>;
/// Do additional verification for this block.
fn verify(
&self,
header: &B::Header,
body: Option<&[B::Extrinsic]>,
) -> Self::Verified;
}
/// A list of all well known keys in the cache.
pub mod well_known_cache_keys {
/// The type representing cache keys.