Request based availability distribution (#2423)

* WIP

* availability distribution, still very wip.

Work on the requesting side of things.

* Some docs on what I intend to do.

* Checkpoint of session cache implementation

as I will likely replace it with something smarter.

* More work, mostly on cache

and getting things to type check.

* Only derive MallocSizeOf and Debug for std.

* availability-distribution: Cache feature complete.

* Sketch out logic in `FetchTask` for actual fetching.

- Compile fixes.
- Cleanup.

* Format cleanup.

* More format fixes.

* Almost feature complete `fetch_task`.

Missing:

- Check for cancel
- Actual querying of peer ids.

* Finish FetchTask so far.

* Directly use AuthorityDiscoveryId in protocol and cache.

* Resolve `AuthorityDiscoveryId` on sending requests.

* Rework fetch_task

- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.

* From<u32> implementation for `ValidatorIndex`.

* Fixes and more integration work.

* Make session cache proper lru cache.

* Use proper lru cache.

* Requester finished.

* ProtocolState -> Requester

Also make sure to not fetch our own chunk.

* Cleanup + fixes.

* Remove unused functions

- FetchTask::is_finished
- SessionCache::fetch_session_info

* availability-distribution responding side.

* Cleanup + Fixes.

* More fixes.

* More fixes.

adder-collator is running!

* Some docs.

* Docs.

* Fix reporting of bad guys.

* Fix tests

* Make all tests compile.

* Fix test.

* Cleanup + get rid of some warnings.

* state -> requester

* Mostly doc fixes.

* Fix test suite.

* Get rid of now redundant message types.

* WIP

* Rob's review remarks.

* Fix test suite.

* core.relay_parent -> leaf for session request.

* Style fix.

* Decrease request timeout.

* Cleanup obsolete errors.

* Metrics + don't fail on non fatal errors.

* requester.rs -> requester/mod.rs

* Panic on invalid BadValidator report.

* Fix indentation.

* Use typed default timeout constant.

* Make channel size 0, as each sender gets one slot anyways.

* Fix incorrect metrics initialization.

* Fix build after merge.

* More fixes.

* Hopefully valid metrics names.

* Better metrics names.

* Some tests that already work.

* Slightly better docs.

* Some more tests.

* Fix network bridge test.

polkadot/node/core/approval-voting/src/criteria.rs

@@ -253,7 +253,7 @@ pub(crate) fn compute_assignments(
 	let (index, assignments_key): (ValidatorIndex, AssignmentPair) = {
 		let key = config.assignment_keys.iter().enumerate()
 			.find_map(|(i, p)| match keystore.key_pair(p) {
-				Ok(Some(pair)) => Some((i as ValidatorIndex, pair)),
+				Ok(Some(pair)) => Some((ValidatorIndex(i as _), pair)),
 				Ok(None) => None,
 				Err(sc_keystore::Error::Unavailable) => None,
 				Err(sc_keystore::Error::Io(e)) if e.kind() == std::io::ErrorKind::NotFound => None,
@@ -422,7 +422,7 @@ pub(crate) fn check_assignment_cert(
 	backing_group: GroupIndex,
 ) -> Result<DelayTranche, InvalidAssignment> {
 	let validator_public = config.assignment_keys
-		.get(validator_index as usize)
+		.get(validator_index.0 as usize)
 		.ok_or(InvalidAssignment)?;
 
 	let public = schnorrkel::PublicKey::from_bytes(validator_public.as_slice())
@@ -540,7 +540,7 @@ mod tests {
 		(0..n_groups).map(|i| {
 			(i * size .. (i + 1) *size)
 				.chain(if i < big_groups { Some(scraps + i) } else { None })
-				.map(|j| j as ValidatorIndex)
+				.map(|j| ValidatorIndex(j as _))
 				.collect::<Vec<_>>()
 		}).collect()
 	}
@@ -570,7 +570,7 @@ mod tests {
 					Sr25519Keyring::Bob,
 					Sr25519Keyring::Charlie,
 				]),
-				validator_groups: vec![vec![0], vec![1, 2]],
+				validator_groups: vec![vec![ValidatorIndex(0)], vec![ValidatorIndex(1), ValidatorIndex(2)]],
 				n_cores: 2,
 				zeroth_delay_tranche_width: 10,
 				relay_vrf_modulo_samples: 3,
@@ -601,7 +601,7 @@ mod tests {
 					Sr25519Keyring::Bob,
 					Sr25519Keyring::Charlie,
 				]),
-				validator_groups: vec![vec![0], vec![1, 2]],
+				validator_groups: vec![vec![ValidatorIndex(0)], vec![ValidatorIndex(1), ValidatorIndex(2)]],
 				n_cores: 2,
 				zeroth_delay_tranche_width: 10,
 				relay_vrf_modulo_samples: 3,
@@ -693,7 +693,7 @@ mod tests {
 				group: group_for_core(core.0 as _),
 				cert: assignment.cert,
 				own_group: GroupIndex(0),
-				val_index: 0,
+				val_index: ValidatorIndex(0),
 				config: config.clone(),
 			};
 
@@ -743,7 +743,7 @@ mod tests {
 	#[test]
 	fn check_rejects_nonexistent_key() {
 		check_mutated_assignments(200, 100, 25, |m| {
-			m.val_index += 200;
+			m.val_index.0 += 200;
 			Some(false)
 		});
 	}