Chain Selection Subsystem Logic (#3277)

* crate skeleton and type definitions

* add ChainSelectionMessage

* add error type

* run loop

* fix overseer

* simplify determine_new_blocks API

* write an overlay struct and fetch new blocks

* add new function to overlay

* more flow

* add leaves to overlay and add a strong type around leaves-set

* add is_parent_viable

* implement block import, ignoring reversions

* add stagnant-at to overlay

* add stagnant

* add revert consensus log

* flow for reversions

* extract and import block reversions

* recursively update viability

* remove redundant parameter from WriteBlockEntry

* do some removal of viable leaves

* address grumbles

* refactor

* address grumbles

* add comment about non-monotonicity

* extract backend to submodule

* begin the hunt for viable leaves

* viability pivots for updating the active leaves

* remove LeafSearchFrontier

* partially -> explicitly viable and untwist some booleans

* extract tree to submodule

* implement block finality update

* Implement block approval routine

* implement stagnant detection

* ensure blocks pruned on finality are removed from the active leaves set

* write down some planned test cases

* floww

* leaf loading

* implement best_leaf_containing

* write down a few more tests to do

* remove dependence of tree on header

* guide: ChainApiMessage::BlockWeight

* node: BlockWeight ChainAPI

* fix compile issue

* note a few TODOs for the future

* fetch block weight using new BlockWeight ChainAPI

* implement unimplemented

* sort leaves by block number after weight

* remove warnings and add more TODOs

* create test module

* storage for test backend

* wrap inner in mutex

* add write waker query to test backend

* Add OverseerSignal -> FromOverseer conversion

* add test harnes

* add no-op test

* add some more test helpers

* the first test

* more progress on tests

* test two subtrees

* determine-new-blocks: cleaner genesis avoidance and tighter ancestry requests

* don't make ancestry requests when asking for one block

* add a couple more tests

* add to AllMessages in guide

* remove bad spaces from bridge

* compact iterator

* test import with gaps

* more reversion tests

* test finalization pruning subtrees

* fixups

* test clobbering and fix bug in overlay

* exhaustive backend state after finalizaiton tested

* more finality tests

* leaf tests

* test approval

* test ChainSelectionMessage::Leaves thoroughly

* remove TODO

* avoid Ordering::is_ne so CI can build

* comment algorithmic complexity

* Update node/core/chain-selection/src/lib.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

polkadot/node/subsystem/src/lib.rs

@@ -175,6 +175,11 @@ pub enum FromOverseer<M> {
 	},
 }
 
+impl<M> From<OverseerSignal> for FromOverseer<M> {
+	fn from(signal: OverseerSignal) -> Self {
+		FromOverseer::Signal(signal)
+	}
+}
 
 /// An error type that describes faults that may happen
 ///

polkadot/node/subsystem/src/messages.rs

@@ -506,6 +506,32 @@ impl ChainApiMessage {
 	}
 }
 
+/// Chain selection subsystem messages
+#[derive(Debug)]
+pub enum ChainSelectionMessage {
+	/// Signal to the chain selection subsystem that a specific block has been approved.
+	Approved(Hash),
+	/// Request the leaves in descending order by score.
+	Leaves(oneshot::Sender<Vec<Hash>>),
+	/// Request the best leaf containing the given block in its ancestry. Return `None` if
+	/// there is no such leaf.
+	BestLeafContaining(Hash, oneshot::Sender<Option<Hash>>),
+}
+
+impl ChainSelectionMessage {
+	/// If the current variant contains the relay parent hash, return it.
+	pub fn relay_parent(&self) -> Option<Hash> {
+		// None of the messages, even the ones containing specific
+		// block hashes, can be considered to have those blocks as
+		// a relay parent.
+		match *self {
+			ChainSelectionMessage::Approved(_) => None,
+			ChainSelectionMessage::Leaves(_) => None,
+			ChainSelectionMessage::BestLeafContaining(..) => None,
+		}
+	}
+}
+
 /// A sender for the result of a runtime API request.
 pub type RuntimeApiSender<T> = oneshot::Sender<Result<T, crate::errors::RuntimeApiError>>;
 
@@ -832,6 +858,9 @@ pub enum AllMessages {
 	/// Message for the dispute participation subsystem.
 	#[skip]
 	DisputeParticipation(DisputeParticipationMessage),
+	/// Message for the chain selection subsystem.
+	#[skip]
+	ChainSelection(ChainSelectionMessage),
 }
 
 impl From<IncomingRequest<req_res_v1::PoVFetchingRequest>> for AvailabilityDistributionMessage {