Request based PoV distribution (#2640)

* Indentation fix.

* Prepare request-response for PoV fetching.

* Drop old PoV distribution.

* WIP: Fetch PoV directly from backing.

* Backing compiles.

* Runtime access and connection management for PoV distribution.

* Get rid of seemingly dead code.

* Implement PoV fetching.

Backing does not yet use it.

* Don't send `ConnectToValidators` for empty list.

* Even better - no need to check over and over again.

* PoV fetching implemented.

+ Typechecks
+ Should work

Missing:

- Guide
- Tests
- Do fallback fetching in case fetching from seconding validator fails.

* Check PoV hash upon reception.

* Implement retry of PoV fetching in backing.

* Avoid pointless validation spawning.

* Add jaeger span to pov requesting.

* Add back tracing.

* Review remarks.

* Whitespace.

* Whitespace again.

* Cleanup + fix tests.

* Log to log target in overseer.

* Fix more tests.

* Don't fail if group cannot be found.

* Simple test for PoV fetcher.

* Handle missing group membership better.

* Add test for retry functionality.

* Fix flaky test.

* Spaces again.

* Guide updates.

* Spaces.

polkadot/node/network/availability-distribution/src/session_cache.rs

@@ -33,8 +33,7 @@ use polkadot_primitives::v1::{
 use polkadot_subsystem::SubsystemContext;
 
 use super::{
-	error::{recv_runtime, NonFatalError},
-	Error,
+	error::{recv_runtime, Error},
 	LOG_TARGET,
 };
 
@@ -82,7 +81,9 @@ pub struct SessionInfo {
 
 	/// Remember to which group we belong, so we won't start fetching chunks for candidates with
 	/// our group being responsible. (We should have that chunk already.)
-	pub our_group: GroupIndex,
+	///
+	/// `None`, if we are not in fact part of any group.
+	pub our_group: Option<GroupIndex>,
 }
 
 /// Report of bad validators.
@@ -122,7 +123,7 @@ impl SessionCache {
 		ctx: &mut Context,
 		parent: Hash,
 		with_info: F,
-	) -> Result<Option<R>, NonFatalError>
+	) -> Result<Option<R>, Error>
 	where
 		Context: SubsystemContext,
 		F: FnOnce(&SessionInfo) -> R,
@@ -219,7 +220,7 @@ impl SessionCache {
 		ctx: &mut Context,
 		parent: Hash,
 		session_index: SessionIndex,
-	) -> Result<Option<SessionInfo>, NonFatalError>
+	) -> Result<Option<SessionInfo>, Error>
 	where
 		Context: SubsystemContext,
 	{
@@ -230,7 +231,7 @@ impl SessionCache {
 			..
 		} = recv_runtime(request_session_info_ctx(parent, session_index, ctx).await)
 			.await?
-			.ok_or(NonFatalError::NoSuchSession(session_index))?;
+			.ok_or(Error::NoSuchSession(session_index))?;
 
 		if let Some(our_index) = self.get_our_index(validators).await {
 			// Get our group index:
@@ -245,8 +246,8 @@ impl SessionCache {
 							None
 						}
 					})
-				})
-				.expect("Every validator should be in a validator group. qed.");
+				}
+			);
 
 			// Shuffle validators in groups:
 			let mut rng = thread_rng();
@@ -274,9 +275,9 @@ impl SessionCache {
 				session_index,
 				our_group,
 			};
-			return Ok(Some(info));
+			return Ok(Some(info))
 		}
-		return Ok(None);
+		return Ok(None)
 	}
 
 	/// Get our `ValidatorIndex`.