Storing multiple Justifications per block (#7640)

* primitives/runtime: initial changes on supporting multiple Justifications

* primitives/runtime: make Justifications strongly typed

* Encode/decode Justifications

* primitives/runtime: add Justification type

* backend: apply_finality and finalize_block takes a single Justification

* manual-seal: create engine id and let rpc take encoded justification

* backend: skeleton functions for appending justifications

* backend: initial implementation append_justification

Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k

* backend: guard against duplicate consensus engine id

* client/db: add check for block finality

* client/api: add append_justification to in_mem db

* client/light: add no-op append_justification

* network: fix decode call for Justification

* network: only send a single Justification in BlockData

* network: minor comment update

* protocol: update field names to distinguish single justification

* client: further field renames to plural

* client: update function names to plural justifications

* client/db: upgrade existing database for new format

* network: remove dependency on grandpa crate

* db: fix check for finalized block

* grandpa: check for multiple grandpa justifications hwne importing

* backend: update Finalizer trait to take multiple Justifications

* db: remove debugging statements in migration code

* manual-seal: update note about engine id

* db: fix check for finalized block

* client: update variable name to reflect it is now plural

* grandpa: fix incorrect empty Justications in test

* primitives: make Justifications opaque to avoid being empty

* network: fix detecting empty Justification

* runtime: doc strings for Justifications functions

* runtime: add into_justifications

* primitives: check for duplicates in when adding to Justifications

* network/test: use real grandpa engine id in test

* client: fix reviewer comments

* primitives: rename Justifications::push to append

* backend: revert changes to Finalizer trait

* backend: revert mark_finalized

* backend: revert changes to finalize_block

* backend: revert finalized_blocks

* db: add a quick early return for performance

* client: minor reviewer comments

* service/test: use local ConsensusEngineId

* network: add link to issue for sending multiple Justifications

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* network: tweaks to review suggestions

* network: revert change to BlockData for backwards compatibility

* Apply suggestion from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* primitives: update doc comment for Justifications

* client/db/upgrade: avoid grandpa crate dependency

* consensus: revert to single Justification for import_justification

* primitives: improve justifications docs

* style cleanups

* use and_then

* client: rename JUSTIFICATIONS db column

* network: revert to using FRNK in network-test

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>

substrate/primitives/runtime/src/generic/block.rs

@@ -30,7 +30,7 @@ use crate::traits::{
 	self, Member, Block as BlockT, Header as HeaderT, MaybeSerialize, MaybeMallocSizeOf,
 	NumberFor,
 };
-use crate::Justification;
+use crate::Justifications;
 
 /// Something to identify a block.
 #[derive(PartialEq, Eq, Clone, RuntimeDebug)]
@@ -112,5 +112,5 @@ pub struct SignedBlock<Block> {
 	/// Full block.
 	pub block: Block,
 	/// Block justification.
-	pub justification: Option<Justification>,
+	pub justifications: Option<Justifications>,
 }

substrate/primitives/runtime/src/lib.rs

@@ -96,7 +96,65 @@ pub use either::Either;
 /// the block itself would allow swapping justifications to change the block's hash
 /// (and thus fork the chain). Sending a `Justification` alongside a block instead
 /// bypasses this problem.
-pub type Justification = Vec<u8>;
+///
+/// Each justification is provided as an encoded blob, and is tagged with an ID
+/// to identify the consensus engine that generated the proof (we might have
+/// multiple justifications from different engines for the same block).
+pub type Justification = (ConsensusEngineId, EncodedJustification);
+
+/// The encoded justification specific to a consensus engine.
+pub type EncodedJustification = Vec<u8>;
+
+/// Collection of justifications for a given block, multiple justifications may
+/// be provided by different consensus engines for the same block.
+#[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
+#[derive(Debug, Clone, PartialEq, Eq, Encode, Decode)]
+pub struct Justifications(Vec<Justification>);
+
+impl Justifications {
+	/// Return an iterator over the justifications.
+	pub fn iter(&self) -> impl Iterator<Item = &Justification> {
+		self.0.iter()
+	}
+
+	/// Append a justification. Returns false if a justification with the same
+	/// `ConsensusEngineId` already exists, in which case the justification is
+	/// not inserted.
+	pub fn append(&mut self, justification: Justification) -> bool {
+		if self.get(justification.0).is_some() {
+			return false;
+		}
+		self.0.push(justification);
+		true
+	}
+
+	/// Return the encoded justification for the given consensus engine, if it
+	/// exists.
+	pub fn get(&self, engine_id: ConsensusEngineId) -> Option<&EncodedJustification> {
+		self.iter().find(|j| j.0 == engine_id).map(|j| &j.1)
+	}
+
+	/// Return a copy of the encoded justification for the given consensus
+	/// engine, if it exists.
+	pub fn into_justification(self, engine_id: ConsensusEngineId) -> Option<EncodedJustification> {
+		self.into_iter().find(|j| j.0 == engine_id).map(|j| j.1)
+	}
+}
+
+impl IntoIterator for Justifications {
+	type Item = Justification;
+	type IntoIter = sp_std::vec::IntoIter<Self::Item>;
+
+	fn into_iter(self) -> Self::IntoIter {
+		self.0.into_iter()
+	}
+}
+
+impl From<Justification> for Justifications {
+	fn from(justification: Justification) -> Self {
+		Self(vec![justification])
+	}
+}
 
 use traits::{Verify, Lazy};