Chain Selection Subsystem Logic (#3277)

* crate skeleton and type definitions

* add ChainSelectionMessage

* add error type

* run loop

* fix overseer

* simplify determine_new_blocks API

* write an overlay struct and fetch new blocks

* add new function to overlay

* more flow

* add leaves to overlay and add a strong type around leaves-set

* add is_parent_viable

* implement block import, ignoring reversions

* add stagnant-at to overlay

* add stagnant

* add revert consensus log

* flow for reversions

* extract and import block reversions

* recursively update viability

* remove redundant parameter from WriteBlockEntry

* do some removal of viable leaves

* address grumbles

* refactor

* address grumbles

* add comment about non-monotonicity

* extract backend to submodule

* begin the hunt for viable leaves

* viability pivots for updating the active leaves

* remove LeafSearchFrontier

* partially -> explicitly viable and untwist some booleans

* extract tree to submodule

* implement block finality update

* Implement block approval routine

* implement stagnant detection

* ensure blocks pruned on finality are removed from the active leaves set

* write down some planned test cases

* floww

* leaf loading

* implement best_leaf_containing

* write down a few more tests to do

* remove dependence of tree on header

* guide: ChainApiMessage::BlockWeight

* node: BlockWeight ChainAPI

* fix compile issue

* note a few TODOs for the future

* fetch block weight using new BlockWeight ChainAPI

* implement unimplemented

* sort leaves by block number after weight

* remove warnings and add more TODOs

* create test module

* storage for test backend

* wrap inner in mutex

* add write waker query to test backend

* Add OverseerSignal -> FromOverseer conversion

* add test harnes

* add no-op test

* add some more test helpers

* the first test

* more progress on tests

* test two subtrees

* determine-new-blocks: cleaner genesis avoidance and tighter ancestry requests

* don't make ancestry requests when asking for one block

* add a couple more tests

* add to AllMessages in guide

* remove bad spaces from bridge

* compact iterator

* test import with gaps

* more reversion tests

* test finalization pruning subtrees

* fixups

* test clobbering and fix bug in overlay

* exhaustive backend state after finalizaiton tested

* more finality tests

* leaf tests

* test approval

* test ChainSelectionMessage::Leaves thoroughly

* remove TODO

* avoid Ordering::is_ne so CI can build

* comment algorithmic complexity

* Update node/core/chain-selection/src/lib.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

polkadot/node/subsystem-util/src/determine_new_blocks.rs

@@ -17,8 +17,7 @@
 //! A utility for fetching all unknown blocks based on a new chain-head hash.
 
 use polkadot_node_subsystem::{
-	messages::ChainApiMessage,
-	SubsystemSender, SubsystemError, SubsystemResult,
+	messages::ChainApiMessage, SubsystemSender,
 };
 use polkadot_primitives::v1::{Hash, Header, BlockNumber};
 use futures::prelude::*;
@@ -41,9 +40,7 @@ pub async fn determine_new_blocks<E>(
 	head: Hash,
 	header: &Header,
 	lower_bound_number: BlockNumber,
-) -> SubsystemResult<Vec<(Hash, Header)>>
-	where SubsystemError: From<E>
-{
+) -> Result<Vec<(Hash, Header)>, E> {
 	const ANCESTRY_STEP: usize = 4;
 
 	let min_block_needed = lower_bound_number + 1;
@@ -173,7 +170,7 @@ mod tests {
 			self.blocks.insert(hash);
 		}
 
-		fn is_known(&self, hash: &Hash) -> Result<bool, SubsystemError> {
+		fn is_known(&self, hash: &Hash) -> Result<bool, ()> {
 			Ok(self.blocks.contains(hash))
 		}
 	}