Generic request/response infrastructure for Polkadot (#2352)

* Move NetworkBridgeEvent to subsystem::messages. It is not protocol related at all, it is in fact only part of the subsystem communication as it gets wrapped into messages of each subsystem. * Request/response infrastructure is taking shape. WIP: Does not compile. * Multiplexer variant not supported by Rusts type system. * request_response::request type checks. * Cleanup. * Minor fixes for request_response. * Implement request sending + move multiplexer. Request multiplexer is moved to bridge, as there the implementation is more straight forward as we can specialize on `AllMessages` for the multiplexing target. Sending of requests is mostly complete, apart from a few `From` instances. Receiving is also almost done, initializtion needs to be fixed and the multiplexer needs to be invoked. * Remove obsolete multiplexer. * Initialize bridge with multiplexer. * Finish generic request sending/receiving. Subsystems are now able to receive and send requests and responses via the overseer. * Doc update. * Fixes. * Link issue for not yet implemented code. * Fixes suggested by @ordian - thanks! - start encoding at 0 - don't crash on zero protocols - don't panic on not yet implemented request handling * Update node/network/protocol/src/request_response/v1.rs Use index 0 instead of 1. Co-authored-by: Andronik Ordian <write@reusable.software> * Update node/network/protocol/src/request_response.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Fix existing tests. * Better avoidance of division by zoro errors. * Doc fixes. * send_request -> start_request. * Fix missing renamings. * Update substrate. * Pass TryConnect instead of true. * Actually import `IfDisconnected`. * Fix wrong import. * Update node/network/bridge/src/lib.rs typo Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Update node/network/bridge/src/multiplexer.rs Remove redundant import. Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Stop doing tracing from within `From` instance. Thanks for the catch @tomaka! * Get rid of redundant import. * Formatting cleanup. * Fix tests. * Add link to issue. * Clarify comments some more. * Fix tests. * Formatting fix. * tabs * Fix link Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * Use map_err. Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * Improvements inspired by suggestions by @drahnr. - Channel size is now determined by function. - Explicitely scope NetworkService::start_request. Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
2026-04-27 03:27:58 +00:00 · 2021-02-03 21:21:09 +01:00
parent 41eeb5716f
commit 0cb1ccd122
25 changed files with 964 additions and 260 deletions
@@ -0,0 +1,209 @@
+// Copyright 2021 Parity Technologies (UK) Ltd.
+// This file is part of Polkadot.
+
+// Polkadot is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Polkadot is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Polkadot.  If not, see <http://www.gnu.org/licenses/>.
+
+use futures::channel::oneshot;
+use futures::prelude::Future;
+
+use parity_scale_codec::{Decode, Encode, Error as DecodingError};
+use sc_network as network;
+use sc_network::config as netconfig;
+use sc_network::PeerId;
+
+use super::{v1, Protocol};
+
+/// Common properties of any `Request`.
+pub trait IsRequest {
+	/// Each request has a corresponding `Response`.
+	type Response;
+
+	/// What protocol this `Request` implements.
+	const PROTOCOL: Protocol;
+}
+
+/// All requests that can be sent to the network bridge via `NetworkBridgeMessage::SendRequest`.
+#[derive(Debug)]
+pub enum Requests {
+	/// Request an availability chunk from a node.
+	AvailabilityFetching(OutgoingRequest<v1::AvailabilityFetchingRequest>),
+}
+
+impl Requests {
+	/// Get the protocol this request conforms to.
+	pub fn get_protocol(&self) -> Protocol {
+		match self {
+			Self::AvailabilityFetching(_) => Protocol::AvailabilityFetching,
+		}
+	}
+
+	/// Encode the request.
+	///
+	/// The corresponding protocol is returned as well, as we are now leaving typed territory.
+	///
+	/// Note: `Requests` is just an enum collecting all supported requests supported by network
+	/// bridge, it is never sent over the wire. This function just encodes the individual requests
+	/// contained in the enum.
+	pub fn encode_request(self) -> (Protocol, OutgoingRequest<Vec<u8>>) {
+		match self {
+			Self::AvailabilityFetching(r) => r.encode_request(),
+		}
+	}
+}
+
+/// A request to be sent to the network bridge, including a sender for sending responses/failures.
+///
+/// The network implementation will make use of that sender for informing the requesting subsystem
+/// about responses/errors.
+#[derive(Debug)]
+pub struct OutgoingRequest<Req> {
+	/// Intendent recipient of this request.
+	pub peer: PeerId,
+	/// The actual request to send over the wire.
+	pub payload: Req,
+	/// Sender which is used by networking to get us back a response.
+	pub pending_response: oneshot::Sender<Result<Vec<u8>, network::RequestFailure>>,
+}
+
+/// Any error that can occur when sending a request.
+pub enum RequestError {
+	/// Response could not be decoded.
+	InvalidResponse(DecodingError),
+
+	/// Some error in substrate/libp2p happened.
+	NetworkError(network::RequestFailure),
+
+	/// Response got canceled by networking.
+	Canceled(oneshot::Canceled),
+}
+
+impl<Req> OutgoingRequest<Req>
+where
+	Req: IsRequest + Encode,
+	Req::Response: Decode,
+{
+	/// Create a new `OutgoingRequest`.
+	///
+	/// It will contain a sender that is used by the networking for sending back responses. The
+	/// connected receiver is returned as the second element in the returned tuple.
+	pub fn new(
+		peer: PeerId,
+		payload: Req,
+	) -> (
+		Self,
+		impl Future<Output = Result<Req::Response, RequestError>>,
+	) {
+		let (tx, rx) = oneshot::channel();
+		let r = Self {
+			peer,
+			payload,
+			pending_response: tx,
+		};
+		(r, receive_response::<Req>(rx))
+	}
+
+	/// Encode a request into a `Vec<u8>`.
+	///
+	/// As this throws away type information, we also return the `Protocol` this encoded request
+	/// adheres to.
+	pub fn encode_request(self) -> (Protocol, OutgoingRequest<Vec<u8>>) {
+		let OutgoingRequest {
+			peer,
+			payload,
+			pending_response,
+		} = self;
+		let encoded = OutgoingRequest {
+			peer,
+			payload: payload.encode(),
+			pending_response,
+		};
+		(Req::PROTOCOL, encoded)
+	}
+}
+
+impl From<DecodingError> for RequestError {
+	fn from(err: DecodingError) -> Self {
+		Self::InvalidResponse(err)
+	}
+}
+
+impl From<network::RequestFailure> for RequestError {
+	fn from(err: network::RequestFailure) -> Self {
+		Self::NetworkError(err)
+	}
+}
+
+impl From<oneshot::Canceled> for RequestError {
+	fn from(err: oneshot::Canceled) -> Self {
+		Self::Canceled(err)
+	}
+}
+
+/// A request coming in, including a sender for sending responses.
+///
+/// `IncomingRequest`s are produced by `RequestMultiplexer` on behalf of the network bridge.
+#[derive(Debug)]
+pub struct IncomingRequest<Req> {
+	/// PeerId of sending peer.
+	pub peer: PeerId,
+	/// The sent request.
+	pub payload: Req,
+	pending_response: oneshot::Sender<netconfig::OutgoingResponse>,
+}
+
+impl<Req> IncomingRequest<Req>
+where
+	Req: IsRequest,
+	Req::Response: Encode,
+{
+	/// Create new `IncomingRequest`.
+	pub fn new(
+		peer: PeerId,
+		payload: Req,
+		pending_response: oneshot::Sender<netconfig::OutgoingResponse>,
+	) -> Self {
+		Self {
+			peer,
+			payload,
+			pending_response,
+		}
+	}
+
+	/// Send the response back.
+	///
+	/// On success we return Ok(()), on error we return the not sent `Response`.
+	///
+	/// netconfig::OutgoingResponse exposes a way of modifying the peer's reputation. If needed we
+	/// can change this function to expose this feature as well.
+	pub fn send_response(self, resp: Req::Response) -> Result<(), Req::Response> {
+		self.pending_response
+			.send(netconfig::OutgoingResponse {
+				result: Ok(resp.encode()),
+				reputation_changes: Vec::new(),
+			})
+			.map_err(|_| resp)
+	}
+}
+
+/// Future for actually receiving a typed response for an OutgoingRequest.
+async fn receive_response<Req>(
+	rec: oneshot::Receiver<Result<Vec<u8>, network::RequestFailure>>,
+) -> Result<Req::Response, RequestError>
+where
+	Req: IsRequest,
+	Req::Response: Decode,
+{
+	let raw = rec.await??;
+	Ok(Decode::decode(&mut raw.as_ref())?)
+}
@@ -0,0 +1,44 @@
+// Copyright 2021 Parity Technologies (UK) Ltd.
+// This file is part of Polkadot.
+
+// Polkadot is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Polkadot is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Polkadot.  If not, see <http://www.gnu.org/licenses/>.
+
+//! Requests and responses as sent over the wire for the individual protocols.
+
+use parity_scale_codec::{Decode, Encode};
+
+use polkadot_primitives::v1::{CandidateHash, ErasureChunk, ValidatorIndex};
+
+use super::request::IsRequest;
+use super::Protocol;
+
+/// Request an availability chunk.
+#[derive(Debug, Clone, Encode, Decode)]
+pub struct AvailabilityFetchingRequest {
+	candidate_hash: CandidateHash,
+	index: ValidatorIndex,
+}
+
+/// Receive a rqeuested erasure chunk.
+#[derive(Debug, Clone, Encode, Decode)]
+pub enum AvailabilityFetchingResponse {
+	/// The requested chunk.
+	#[codec(index = 0)]
+	Chunk(ErasureChunk),
+}
+
+impl IsRequest for AvailabilityFetchingRequest {
+	type Response = AvailabilityFetchingResponse;
+	const PROTOCOL: Protocol = Protocol::AvailabilityFetching;
+}