Approve multiple candidates with a single signature (#1191)

Initial implementation for the plan discussed here: https://github.com/paritytech/polkadot-sdk/issues/701
Built on top of https://github.com/paritytech/polkadot-sdk/pull/1178
v0: https://github.com/paritytech/polkadot/pull/7554,

## Overall idea

When approval-voting checks a candidate and is ready to advertise the
approval, defer it in a per-relay chain block until we either have
MAX_APPROVAL_COALESCE_COUNT candidates to sign or a candidate has stayed
MAX_APPROVALS_COALESCE_TICKS in the queue, in both cases we sign what
candidates we have available.

This should allow us to reduce the number of approvals messages we have
to create/send/verify. The parameters are configurable, so we should
find some values that balance:

- Security of the network: Delaying broadcasting of an approval
shouldn't but the finality at risk and to make sure that never happens
we won't delay sending a vote if we are past 2/3 from the no-show time.
- Scalability of the network: MAX_APPROVAL_COALESCE_COUNT = 1 &
MAX_APPROVALS_COALESCE_TICKS =0, is what we have now and we know from
the measurements we did on versi, it bottlenecks
approval-distribution/approval-voting when increase significantly the
number of validators and parachains
- Block storage: In case of disputes we have to import this votes on
chain and that increase the necessary storage with
MAX_APPROVAL_COALESCE_COUNT * CandidateHash per vote. Given that
disputes are not the normal way of the network functioning and we will
limit MAX_APPROVAL_COALESCE_COUNT in the single digits numbers, this
should be good enough. Alternatively, we could try to create a better
way to store this on-chain through indirection, if that's needed.

## Other fixes:
- Fixed the fact that we were sending random assignments to
non-validators, that was wrong because those won't do anything with it
and they won't gossip it either because they do not have a grid topology
set, so we would waste the random assignments.
- Added metrics to be able to debug potential no-shows and
mis-processing of approvals/assignments.

## TODO:
- [x] Get feedback, that this is moving in the right direction. @ordian
@sandreim @eskimor @burdges, let me know what you think.
- [x] More and more testing.
- [x]  Test in versi.
- [x] Make MAX_APPROVAL_COALESCE_COUNT &
MAX_APPROVAL_COALESCE_WAIT_MILLIS a parachain host configuration.
- [x] Make sure the backwards compatibility works correctly
- [x] Make sure this direction is compatible with other streams of work:
https://github.com/paritytech/polkadot-sdk/issues/635 &
https://github.com/paritytech/polkadot-sdk/issues/742
- [x] Final versi burn-in before merging

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

polkadot/primitives/src/v6/mod.rs

@@ -1070,6 +1070,26 @@ impl ApprovalVote {
 	}
 }
 
+/// A vote of approval for multiple candidates.
+#[derive(Clone, RuntimeDebug)]
+pub struct ApprovalVoteMultipleCandidates<'a>(pub &'a [CandidateHash]);
+
+impl<'a> ApprovalVoteMultipleCandidates<'a> {
+	/// Yields the signing payload for this approval vote.
+	pub fn signing_payload(&self, session_index: SessionIndex) -> Vec<u8> {
+		const MAGIC: [u8; 4] = *b"APPR";
+		// Make this backwards compatible with `ApprovalVote` so if we have just on candidate the
+		// signature will look the same.
+		// This gives us the nice benefit that old nodes can still check signatures when len is 1
+		// and the new node can check the signature coming from old nodes.
+		if self.0.len() == 1 {
+			(MAGIC, self.0.first().expect("QED: we just checked"), session_index).encode()
+		} else {
+			(MAGIC, &self.0, session_index).encode()
+		}
+	}
+}
+
 /// Custom validity errors used in Polkadot while validating transactions.
 #[repr(u8)]
 pub enum ValidityError {
@@ -1246,25 +1266,42 @@ pub enum DisputeStatement {
 
 impl DisputeStatement {
 	/// Get the payload data for this type of dispute statement.
-	pub fn payload_data(&self, candidate_hash: CandidateHash, session: SessionIndex) -> Vec<u8> {
-		match *self {
+	///
+	/// Returns Error if the candidate_hash is not included in the list of signed
+	/// candidate from ApprovalCheckingMultipleCandidate.
+	pub fn payload_data(
+		&self,
+		candidate_hash: CandidateHash,
+		session: SessionIndex,
+	) -> Result<Vec<u8>, ()> {
+		match self {
 			DisputeStatement::Valid(ValidDisputeStatementKind::Explicit) =>
-				ExplicitDisputeStatement { valid: true, candidate_hash, session }.signing_payload(),
+				Ok(ExplicitDisputeStatement { valid: true, candidate_hash, session }
+					.signing_payload()),
 			DisputeStatement::Valid(ValidDisputeStatementKind::BackingSeconded(
 				inclusion_parent,
-			)) => CompactStatement::Seconded(candidate_hash).signing_payload(&SigningContext {
+			)) => Ok(CompactStatement::Seconded(candidate_hash).signing_payload(&SigningContext {
 				session_index: session,
-				parent_hash: inclusion_parent,
-			}),
+				parent_hash: *inclusion_parent,
+			})),
 			DisputeStatement::Valid(ValidDisputeStatementKind::BackingValid(inclusion_parent)) =>
-				CompactStatement::Valid(candidate_hash).signing_payload(&SigningContext {
+				Ok(CompactStatement::Valid(candidate_hash).signing_payload(&SigningContext {
 					session_index: session,
-					parent_hash: inclusion_parent,
-				}),
+					parent_hash: *inclusion_parent,
+				})),
 			DisputeStatement::Valid(ValidDisputeStatementKind::ApprovalChecking) =>
-				ApprovalVote(candidate_hash).signing_payload(session),
+				Ok(ApprovalVote(candidate_hash).signing_payload(session)),
+			DisputeStatement::Valid(
+				ValidDisputeStatementKind::ApprovalCheckingMultipleCandidates(candidate_hashes),
+			) =>
+				if candidate_hashes.contains(&candidate_hash) {
+					Ok(ApprovalVoteMultipleCandidates(candidate_hashes).signing_payload(session))
+				} else {
+					Err(())
+				},
 			DisputeStatement::Invalid(InvalidDisputeStatementKind::Explicit) =>
-				ExplicitDisputeStatement { valid: false, candidate_hash, session }.signing_payload(),
+				Ok(ExplicitDisputeStatement { valid: false, candidate_hash, session }
+					.signing_payload()),
 		}
 	}
 
@@ -1276,7 +1313,7 @@ impl DisputeStatement {
 		session: SessionIndex,
 		validator_signature: &ValidatorSignature,
 	) -> Result<(), ()> {
-		let payload = self.payload_data(candidate_hash, session);
+		let payload = self.payload_data(candidate_hash, session)?;
 
 		if validator_signature.verify(&payload[..], &validator_public) {
 			Ok(())
@@ -1308,13 +1345,14 @@ impl DisputeStatement {
 			Self::Valid(ValidDisputeStatementKind::BackingValid(_)) => true,
 			Self::Valid(ValidDisputeStatementKind::Explicit) |
 			Self::Valid(ValidDisputeStatementKind::ApprovalChecking) |
+			Self::Valid(ValidDisputeStatementKind::ApprovalCheckingMultipleCandidates(_)) |
 			Self::Invalid(_) => false,
 		}
 	}
 }
 
 /// Different kinds of statements of validity on  a candidate.
-#[derive(Encode, Decode, Copy, Clone, PartialEq, RuntimeDebug, TypeInfo)]
+#[derive(Encode, Decode, Clone, PartialEq, RuntimeDebug, TypeInfo)]
 pub enum ValidDisputeStatementKind {
 	/// An explicit statement issued as part of a dispute.
 	#[codec(index = 0)]
@@ -1328,6 +1366,12 @@ pub enum ValidDisputeStatementKind {
 	/// An approval vote from the approval checking phase.
 	#[codec(index = 3)]
 	ApprovalChecking,
+	/// An approval vote from the new version.
+	/// We can't create this version untill all nodes
+	/// have been updated to support it and max_approval_coalesce_count
+	/// is set to more than 1.
+	#[codec(index = 4)]
+	ApprovalCheckingMultipleCandidates(Vec<CandidateHash>),
 }
 
 /// Different kinds of statements of invalidity on a candidate.