pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-12 18:11:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Integrate litep2p into Polkadot SDK (#2944)

Browse Source 
[litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P
networking library. It supports all of the features of `rust-libp2p`
that are currently being utilized by Polkadot SDK.

Compared to `rust-libp2p`, `litep2p` has a quite different architecture
which is why the new `litep2p` network backend is only able to use a
little of the existing code in `sc-network`. The design has been mainly
influenced by how we'd wish to structure our networking-related code in
Polkadot SDK: independent higher-levels protocols directly communicating
with the network over links that support bidirectional backpressure. A
good example would be `NotificationHandle`/`RequestResponseHandle`
abstractions which allow, e.g., `SyncingEngine` to directly communicate
with peers to announce/request blocks.

I've tried running `polkadot --network-backend litep2p` with a few
different peer configurations and there is a noticeable reduction in
networking CPU usage. For high load (`--out-peers 200`), networking CPU
usage goes down from ~110% to ~30% (80 pp) and for normal load
(`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp).

These should not be taken as final numbers because:

a) there are still some low-hanging optimization fruits, such as
enabling [receive window
auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating
`Peerset` more closely with `litep2p` or improving memory usage of the
WebSocket transport
b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less
work will increase the networking CPU usage
c) verification in a more diverse set of tests/conditions is needed

Nevertheless, these numbers should give an early estimate for CPU usage
of the new networking backend.

This PR consists of three separate changes:
* introduce a generic `PeerId` (wrapper around `Multihash`) so that we
don't have use `NetworkService::PeerId` in every part of the code that
uses a `PeerId`
* introduce `NetworkBackend` trait, implement it for the libp2p network
stack and make Polkadot SDK generic over `NetworkBackend`
  * implement `NetworkBackend` for litep2p

The new library should be considered experimental which is why
`rust-libp2p` will remain as the default option for the time being. This
PR currently depends on the master branch of `litep2p` but I'll cut a
new release for the library once all review comments have been
addresses.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
This commit is contained in:
Aaro Altonen
2024-04-08 19:44:13 +03:00
committed by GitHub
parent 9543d31474
commit 80616f6d03
181 changed files with 11055 additions and 1862 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Cargo.lock
Generated
+589 -140
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+1 -1
View File
@@ -260,13 +260,13 @@ members = [
"substrate/client/mixnet",
"substrate/client/network",
"substrate/client/network-gossip",
"substrate/client/network/bitswap",
"substrate/client/network/common",
"substrate/client/network/light",
"substrate/client/network/statement",
"substrate/client/network/sync",
"substrate/client/network/test",
"substrate/client/network/transactions",
"substrate/client/network/types",
"substrate/client/offchain",
"substrate/client/proposer-metrics",
"substrate/client/rpc",
cumulus/client/relay-chain-minimal-node/src/lib.rs
+56 -26
View File
@@ -27,12 +27,16 @@ use polkadot_node_network_protocol::{
},
};
use polkadot_core_primitives::{Block as RelayBlock, Hash as RelayHash};
use polkadot_node_subsystem_util::metrics::prometheus::Registry;
use polkadot_primitives::CollatorPair;
use polkadot_service::{overseer::OverseerGenArgs, IsParachainNode};
use sc_authority_discovery::Service as AuthorityDiscoveryService;
use sc_network::{config::FullNetworkConfiguration, Event, NetworkEventStream, NetworkService};
use sc_network::{
config::FullNetworkConfiguration, service::traits::NetworkService, Event, NetworkBackend,
NetworkEventStream,
};
use sc_service::{config::PrometheusConfig, Configuration, TaskManager};
use sp_runtime::{app_crypto::Pair, traits::Block as BlockT};
@@ -51,7 +55,7 @@ fn build_authority_discovery_service<Block: BlockT>(
task_manager: &TaskManager,
client: Arc<BlockChainRpcClient>,
config: &Configuration,
network: Arc<NetworkService<Block, <Block as BlockT>::Hash>>,
network: Arc<dyn NetworkService>,
prometheus_registry: Option<Registry>,
) -> AuthorityDiscoveryService {
let auth_disc_publish_non_global_ips = config.network.allow_non_globals_in_dht;
@@ -72,7 +76,7 @@ fn build_authority_discovery_service<Block: BlockT>(
..Default::default()
},
client,
network,
Arc::new(network.clone()),
Box::pin(dht_event_stream),
authority_discovery_role,
prometheus_registry,
@@ -92,12 +96,22 @@ async fn build_interface(
client: RelayChainRpcClient,
) -> RelayChainResult<(Arc<(dyn RelayChainInterface + 'static)>, Option<CollatorPair>)> {
let collator_pair = CollatorPair::generate().0;
let collator_node = new_minimal_relay_chain(
polkadot_config,
collator_pair.clone(),
Arc::new(BlockChainRpcClient::new(client.clone())),
)
.await?;
let collator_node = match polkadot_config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p =>
new_minimal_relay_chain::<RelayBlock, sc_network::NetworkWorker<RelayBlock, RelayHash>>(
polkadot_config,
collator_pair.clone(),
Arc::new(BlockChainRpcClient::new(client.clone())),
)
.await?,
sc_network::config::NetworkBackendType::Litep2p =>
new_minimal_relay_chain::<RelayBlock, sc_network::Litep2pNetworkBackend>(
polkadot_config,
collator_pair.clone(),
Arc::new(BlockChainRpcClient::new(client.clone())),
)
.await?,
};
task_manager.add_child(collator_node.task_manager);
Ok((
Arc::new(RelayChainRpcInterface::new(client, collator_node.overseer_handle)),
@@ -143,6 +157,7 @@ pub async fn build_minimal_relay_chain_node_light_client(
build_interface(polkadot_config, task_manager, client).await
}
/// Builds a minimal relay chain node. Chain data is fetched
/// via [`BlockChainRpcClient`] and fed into the overseer and its subsystems.
///
@@ -155,13 +170,18 @@ pub async fn build_minimal_relay_chain_node_light_client(
/// - NetworkBridgeTx
/// - RuntimeApi
#[sc_tracing::logging::prefix_logs_with("Relaychain")]
async fn new_minimal_relay_chain(
async fn new_minimal_relay_chain<Block: BlockT, Network: NetworkBackend<RelayBlock, RelayHash>>(
config: Configuration,
collator_pair: CollatorPair,
relay_chain_rpc_client: Arc<BlockChainRpcClient>,
) -> Result<NewMinimalNode, RelayChainError> {
let role = config.role.clone();
let mut net_config = sc_network::config::FullNetworkConfiguration::new(&config.network);
let mut net_config =
sc_network::config::FullNetworkConfiguration::<_, _, Network>::new(&config.network);
let metrics = Network::register_notification_metrics(
config.prometheus_config.as_ref().map(|cfg| &cfg.registry),
);
let peer_store_handle = net_config.peer_store_handle();
let prometheus_registry = config.prometheus_registry();
let task_manager = TaskManager::new(config.tokio_handle.clone(), prometheus_registry)?;
@@ -178,13 +198,18 @@ async fn new_minimal_relay_chain(
let peerset_protocol_names =
PeerSetProtocolNames::new(genesis_hash, config.chain_spec.fork_id());
let is_authority = if role.is_authority() { IsAuthority::Yes } else { IsAuthority::No };
let notification_services = peer_sets_info(is_authority, &peerset_protocol_names)
.into_iter()
.map(|(config, (peerset, service))| {
net_config.add_notification_protocol(config);
(peerset, service)
})
.collect::<std::collections::HashMap<PeerSet, Box<dyn sc_network::NotificationService>>>();
let notification_services = peer_sets_info::<_, Network>(
is_authority,
&peerset_protocol_names,
metrics.clone(),
Arc::clone(&peer_store_handle),
)
.into_iter()
.map(|(config, (peerset, service))| {
net_config.add_notification_protocol(config);
(peerset, service)
})
.collect::<std::collections::HashMap<PeerSet, Box<dyn sc_network::NotificationService>>>();
let request_protocol_names = ReqProtocolNames::new(genesis_hash, config.chain_spec.fork_id());
let (collation_req_v1_receiver, collation_req_v2_receiver, available_data_req_receiver) =
@@ -194,16 +219,17 @@ async fn new_minimal_relay_chain(
.chain_get_header(None)
.await?
.ok_or_else(|| RelayChainError::RpcCallError("Unable to fetch best header".to_string()))?;
let (network, network_starter, sync_service) = build_collator_network(
let (network, network_starter, sync_service) = build_collator_network::<Network>(
&config,
net_config,
task_manager.spawn_handle(),
genesis_hash,
best_header,
metrics,
)
.map_err(|e| RelayChainError::Application(Box::new(e) as Box<_>))?;
let authority_discovery_service = build_authority_discovery_service(
let authority_discovery_service = build_authority_discovery_service::<Block>(
&task_manager,
relay_chain_rpc_client.clone(),
&config,
@@ -236,24 +262,28 @@ async fn new_minimal_relay_chain(
Ok(NewMinimalNode { task_manager, overseer_handle })
}
fn build_request_response_protocol_receivers(
fn build_request_response_protocol_receivers<
Block: BlockT,
Network: NetworkBackend<Block, <Block as BlockT>::Hash>,
>(
request_protocol_names: &ReqProtocolNames,
config: &mut FullNetworkConfiguration,
config: &mut FullNetworkConfiguration<Block, <Block as BlockT>::Hash, Network>,
) -> (
IncomingRequestReceiver<v1::CollationFetchingRequest>,
IncomingRequestReceiver<v2::CollationFetchingRequest>,
IncomingRequestReceiver<v1::AvailableDataFetchingRequest>,
) {
let (collation_req_v1_receiver, cfg) =
IncomingRequest::get_config_receiver(request_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(request_protocol_names);
config.add_request_response_protocol(cfg);
let (collation_req_v2_receiver, cfg) =
IncomingRequest::get_config_receiver(request_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(request_protocol_names);
config.add_request_response_protocol(cfg);
let (available_data_req_receiver, cfg) =
IncomingRequest::get_config_receiver(request_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(request_protocol_names);
config.add_request_response_protocol(cfg);
let cfg = Protocol::ChunkFetchingV1.get_outbound_only_config(request_protocol_names);
let cfg =
Protocol::ChunkFetchingV1.get_outbound_only_config::<_, Network>(request_protocol_names);
config.add_request_response_protocol(cfg);
(collation_req_v1_receiver, collation_req_v2_receiver, available_data_req_receiver)
}
cumulus/client/relay-chain-minimal-node/src/network.rs
+33 -36
View File
@@ -15,64 +15,56 @@
// along with Cumulus. If not, see <http://www.gnu.org/licenses/>.
use polkadot_core_primitives::{Block, Hash, Header};
use sp_runtime::traits::{Block as BlockT, NumberFor};
use sp_runtime::traits::NumberFor;
use sc_network::{
config::{
NetworkConfiguration, NonDefaultSetConfig, NonReservedPeerMode, NotificationHandshake,
ProtocolId, SetConfig,
NetworkConfiguration, NonReservedPeerMode, NotificationHandshake, PeerStore, ProtocolId,
SetConfig,
},
peer_store::PeerStore,
NetworkService,
peer_store::PeerStoreProvider,
service::traits::NetworkService,
NotificationMetrics,
};
use sc_network::{config::FullNetworkConfiguration, NotificationService};
use sc_network::{config::FullNetworkConfiguration, NetworkBackend, NotificationService};
use sc_network_common::{role::Roles, sync::message::BlockAnnouncesHandshake};
use sc_service::{error::Error, Configuration, NetworkStarter, SpawnTaskHandle};
use std::{iter, sync::Arc};
/// Build the network service, the network status sinks and an RPC sender.
pub(crate) fn build_collator_network(
pub(crate) fn build_collator_network<Network: NetworkBackend<Block, Hash>>(
config: &Configuration,
mut full_network_config: FullNetworkConfiguration,
mut network_config: FullNetworkConfiguration<Block, Hash, Network>,
spawn_handle: SpawnTaskHandle,
genesis_hash: Hash,
best_header: Header,
notification_metrics: NotificationMetrics,
) -> Result<
(
Arc<NetworkService<Block, Hash>>,
NetworkStarter,
Arc<dyn sp_consensus::SyncOracle + Send + Sync>,
),
(Arc<dyn NetworkService>, NetworkStarter, Arc<dyn sp_consensus::SyncOracle + Send + Sync>),
Error,
> {
let protocol_id = config.protocol_id();
let (block_announce_config, _notification_service) = get_block_announce_proto_config::<Block>(
let (block_announce_config, _notification_service) = get_block_announce_proto_config::<Network>(
protocol_id.clone(),
&None,
Roles::from(&config.role),
best_header.number,
best_header.hash(),
genesis_hash,
notification_metrics.clone(),
network_config.peer_store_handle(),
);
// Since this node has no syncing, we do not want light-clients to connect to it.
// Here we set any potential light-client slots to 0.
adjust_network_config_light_in_peers(&mut full_network_config.network_config);
adjust_network_config_light_in_peers(&mut network_config.network_config);
let peer_store = PeerStore::new(
full_network_config
.network_config
.boot_nodes
.iter()
.map(|bootnode| bootnode.peer_id)
.collect(),
);
let peer_store_handle = peer_store.handle();
let peer_store = network_config.take_peer_store();
spawn_handle.spawn("peer-store", Some("networking"), peer_store.run());
let network_params = sc_network::config::Params::<Block> {
let network_params = sc_network::config::Params::<Block, Hash, Network> {
role: config.role.clone(),
executor: {
let spawn_handle = Clone::clone(&spawn_handle);
@@ -81,16 +73,17 @@ pub(crate) fn build_collator_network(
})
},
fork_id: None,
network_config: full_network_config,
peer_store: peer_store_handle,
network_config,
genesis_hash,
protocol_id,
metrics_registry: config.prometheus_config.as_ref().map(|config| config.registry.clone()),
block_announce_config,
bitswap_config: None,
notification_metrics,
};
let network_worker = sc_network::NetworkWorker::new(network_params)?;
let network_service = network_worker.service().clone();
let network_worker = Network::new(network_params)?;
let network_service = network_worker.network_service();
let (network_start_tx, network_start_rx) = futures::channel::oneshot::channel();
@@ -143,14 +136,16 @@ impl sp_consensus::SyncOracle for SyncOracle {
}
}
fn get_block_announce_proto_config<B: BlockT>(
fn get_block_announce_proto_config<Network: NetworkBackend<Block, Hash>>(
protocol_id: ProtocolId,
fork_id: &Option<String>,
roles: Roles,
best_number: NumberFor<B>,
best_hash: B::Hash,
genesis_hash: B::Hash,
) -> (NonDefaultSetConfig, Box<dyn NotificationService>) {
best_number: NumberFor<Block>,
best_hash: Hash,
genesis_hash: Hash,
metrics: NotificationMetrics,
peer_store_handle: Arc<dyn PeerStoreProvider>,
) -> (Network::NotificationProtocolConfig, Box<dyn NotificationService>) {
let block_announces_protocol = {
let genesis_hash = genesis_hash.as_ref();
if let Some(ref fork_id) = fork_id {
@@ -160,11 +155,11 @@ fn get_block_announce_proto_config<B: BlockT>(
}
};
NonDefaultSetConfig::new(
Network::notification_config(
block_announces_protocol.into(),
iter::once(format!("/{}/block-announces/1", protocol_id.as_ref()).into()).collect(),
1024 * 1024,
Some(NotificationHandshake::new(BlockAnnouncesHandshake::<B>::build(
Some(NotificationHandshake::new(BlockAnnouncesHandshake::<Block>::build(
roles,
best_number,
best_hash,
@@ -178,5 +173,7 @@ fn get_block_announce_proto_config<B: BlockT>(
reserved_nodes: Vec::new(),
non_reserved_mode: NonReservedPeerMode::Deny,
},
metrics,
peer_store_handle,
)
}
cumulus/client/service/src/lib.rs
+12 -5
View File
@@ -40,7 +40,7 @@ use sc_consensus::{
import_queue::{ImportQueue, ImportQueueService},
BlockImport,
};
use sc_network::{config::SyncMode, NetworkService};
use sc_network::{config::SyncMode, service::traits::NetworkService, NetworkBackend};
use sc_network_sync::SyncingService;
use sc_network_transactions::TransactionsHandlerController;
use sc_service::{Configuration, NetworkStarter, SpawnTaskHandle, TaskManager, WarpSyncParams};
@@ -406,13 +406,15 @@ pub struct BuildNetworkParams<
+ HeaderBackend<Block>
+ BlockIdTo<Block>
+ 'static,
Network: NetworkBackend<Block, <Block as BlockT>::Hash>,
RCInterface,
IQ,
> where
Client::Api: sp_transaction_pool::runtime_api::TaggedTransactionQueue<Block>,
{
pub parachain_config: &'a Configuration,
pub net_config: sc_network::config::FullNetworkConfiguration,
pub net_config:
sc_network::config::FullNetworkConfiguration<Block, <Block as BlockT>::Hash, Network>,
pub client: Arc<Client>,
pub transaction_pool: Arc<sc_transaction_pool::FullPool<Block, Client>>,
pub para_id: ParaId,
@@ -423,7 +425,7 @@ pub struct BuildNetworkParams<
}
/// Build the network service, the network status sinks and an RPC sender.
pub async fn build_network<'a, Block, Client, RCInterface, IQ>(
pub async fn build_network<'a, Block, Client, RCInterface, IQ, Network>(
BuildNetworkParams {
parachain_config,
net_config,
@@ -434,9 +436,9 @@ pub async fn build_network<'a, Block, Client, RCInterface, IQ>(
relay_chain_interface,
import_queue,
sybil_resistance_level,
}: BuildNetworkParams<'a, Block, Client, RCInterface, IQ>,
}: BuildNetworkParams<'a, Block, Client, Network, RCInterface, IQ>,
) -> sc_service::error::Result<(
Arc<NetworkService<Block, Block::Hash>>,
Arc<dyn NetworkService>,
TracingUnboundedSender<sc_rpc::system::Request<Block>>,
TransactionsHandlerController<Block::Hash>,
NetworkStarter,
@@ -461,6 +463,7 @@ where
for<'b> &'b Client: BlockImport<Block>,
RCInterface: RelayChainInterface + Clone + 'static,
IQ: ImportQueue<Block> + 'static,
Network: NetworkBackend<Block, <Block as BlockT>::Hash>,
{
let warp_sync_params = match parachain_config.network.sync_mode {
SyncMode::Warp => {
@@ -485,6 +488,9 @@ where
Box::new(block_announce_validator) as Box<_>
},
};
let metrics = Network::register_notification_metrics(
parachain_config.prometheus_config.as_ref().map(|cfg| &cfg.registry),
);
sc_service::build_network(sc_service::BuildNetworkParams {
config: parachain_config,
@@ -496,6 +502,7 @@ where
block_announce_validator_builder: Some(Box::new(move |_| block_announce_validator)),
warp_sync_params,
block_relay: None,
metrics,
})
}
cumulus/polkadot-parachain/src/command.rs
+213 -129
View File
@@ -21,7 +21,7 @@ use crate::{
fake_runtime_api::{
asset_hub_polkadot_aura::RuntimeApi as AssetHubPolkadotRuntimeApi, aura::RuntimeApi,
},
service::{new_partial, Block},
service::{new_partial, Block, Hash},
};
use cumulus_client_service::storage_proof_size::HostFunctions as ReclaimHostFunctions;
use cumulus_primitives_core::ParaId;
@@ -498,7 +498,6 @@ macro_rules! construct_async_run {
/// Parse command line arguments into service configuration.
pub fn run() -> Result<()> {
use Runtime::*;
let cli = Cli::from_args();
match &cli.subcommand {
@@ -670,151 +669,236 @@ pub fn run() -> Result<()> {
info!("Parachain Account: {}", parachain_account);
info!("Is collating: {}", if config.role.is_authority() { "yes" } else { "no" });
match config.chain_spec.runtime()? {
AssetHubPolkadot => crate::service::start_asset_hub_node::<
AssetHubPolkadotRuntimeApi,
AssetHubPolkadotAuraId,
>(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
AssetHubKusama =>
crate::service::start_asset_hub_node::<
RuntimeApi,
AuraId,
>(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
AssetHubRococo | AssetHubWestend =>
crate::service::start_asset_hub_lookahead_node::<
RuntimeApi,
AuraId,
>(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
CollectivesPolkadot =>
crate::service::start_generic_aura_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
CollectivesWestend =>
crate::service::start_generic_aura_lookahead_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
Seedling | Shell =>
crate::service::start_shell_node(
match polkadot_config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p =>
start_node::<sc_network::NetworkWorker<_, _>>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
ContractsRococo => crate::service::start_contracts_rococo_node(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
BridgeHub(bridge_hub_runtime_type) => match bridge_hub_runtime_type {
chain_spec::bridge_hubs::BridgeHubRuntimeType::Polkadot |
chain_spec::bridge_hubs::BridgeHubRuntimeType::PolkadotLocal =>
crate::service::start_generic_aura_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0),
chain_spec::bridge_hubs::BridgeHubRuntimeType::Kusama |
chain_spec::bridge_hubs::BridgeHubRuntimeType::KusamaLocal =>
crate::service::start_generic_aura_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0),
chain_spec::bridge_hubs::BridgeHubRuntimeType::Westend |
chain_spec::bridge_hubs::BridgeHubRuntimeType::WestendLocal |
chain_spec::bridge_hubs::BridgeHubRuntimeType::WestendDevelopment =>
crate::service::start_generic_aura_lookahead_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0),
chain_spec::bridge_hubs::BridgeHubRuntimeType::Rococo |
chain_spec::bridge_hubs::BridgeHubRuntimeType::RococoLocal |
chain_spec::bridge_hubs::BridgeHubRuntimeType::RococoDevelopment =>
crate::service::start_generic_aura_lookahead_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0),
}
.map_err(Into::into),
Coretime(coretime_runtime_type) => match coretime_runtime_type {
chain_spec::coretime::CoretimeRuntimeType::Kusama |
chain_spec::coretime::CoretimeRuntimeType::KusamaLocal |
chain_spec::coretime::CoretimeRuntimeType::Polkadot |
chain_spec::coretime::CoretimeRuntimeType::PolkadotLocal |
chain_spec::coretime::CoretimeRuntimeType::Rococo |
chain_spec::coretime::CoretimeRuntimeType::RococoLocal |
chain_spec::coretime::CoretimeRuntimeType::RococoDevelopment |
chain_spec::coretime::CoretimeRuntimeType::Westend |
chain_spec::coretime::CoretimeRuntimeType::WestendLocal |
chain_spec::coretime::CoretimeRuntimeType::WestendDevelopment =>
crate::service::start_generic_aura_lookahead_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0),
}
.map_err(Into::into),
Penpal(_) | Default =>
crate::service::start_rococo_parachain_node(
.await,
sc_network::config::NetworkBackendType::Litep2p =>
start_node::<sc_network::Litep2pNetworkBackend>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Glutton | GluttonWestend =>
crate::service::start_basic_lookahead_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
People(people_runtime_type) => match people_runtime_type {
chain_spec::people::PeopleRuntimeType::Kusama |
chain_spec::people::PeopleRuntimeType::KusamaLocal |
chain_spec::people::PeopleRuntimeType::Polkadot |
chain_spec::people::PeopleRuntimeType::PolkadotLocal |
chain_spec::people::PeopleRuntimeType::Rococo |
chain_spec::people::PeopleRuntimeType::RococoLocal |
chain_spec::people::PeopleRuntimeType::RococoDevelopment |
chain_spec::people::PeopleRuntimeType::Westend |
chain_spec::people::PeopleRuntimeType::WestendLocal |
chain_spec::people::PeopleRuntimeType::WestendDevelopment =>
crate::service::start_generic_aura_lookahead_node(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0),
}
.map_err(Into::into),
.await,
}
})
},
}
}
async fn start_node<Network: sc_network::NetworkBackend<Block, Hash>>(
config: sc_service::Configuration,
polkadot_config: sc_service::Configuration,
collator_options: cumulus_client_cli::CollatorOptions,
id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> Result<sc_service::TaskManager> {
match config.chain_spec.runtime()? {
Runtime::AssetHubPolkadot => crate::service::start_asset_hub_node::<
AssetHubPolkadotRuntimeApi,
AssetHubPolkadotAuraId,
Network,
>(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::AssetHubKusama => crate::service::start_asset_hub_node::<
RuntimeApi,
AuraId,
Network,
>(config, polkadot_config, collator_options, id, hwbench)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::AssetHubRococo | Runtime::AssetHubWestend =>
crate::service::start_asset_hub_lookahead_node::<RuntimeApi, AuraId, Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::CollectivesPolkadot => crate::service::start_generic_aura_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::CollectivesWestend =>
crate::service::start_generic_aura_lookahead_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::Seedling | Runtime::Shell => crate::service::start_shell_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::ContractsRococo => crate::service::start_contracts_rococo_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::BridgeHub(bridge_hub_runtime_type) => match bridge_hub_runtime_type {
chain_spec::bridge_hubs::BridgeHubRuntimeType::Polkadot |
chain_spec::bridge_hubs::BridgeHubRuntimeType::PolkadotLocal =>
crate::service::start_generic_aura_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0),
chain_spec::bridge_hubs::BridgeHubRuntimeType::Kusama |
chain_spec::bridge_hubs::BridgeHubRuntimeType::KusamaLocal =>
crate::service::start_generic_aura_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0),
chain_spec::bridge_hubs::BridgeHubRuntimeType::Westend |
chain_spec::bridge_hubs::BridgeHubRuntimeType::WestendLocal |
chain_spec::bridge_hubs::BridgeHubRuntimeType::WestendDevelopment =>
crate::service::start_generic_aura_lookahead_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0),
chain_spec::bridge_hubs::BridgeHubRuntimeType::Rococo |
chain_spec::bridge_hubs::BridgeHubRuntimeType::RococoLocal |
chain_spec::bridge_hubs::BridgeHubRuntimeType::RococoDevelopment =>
crate::service::start_generic_aura_lookahead_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0),
}
.map_err(Into::into),
Runtime::Coretime(coretime_runtime_type) => match coretime_runtime_type {
chain_spec::coretime::CoretimeRuntimeType::Kusama |
chain_spec::coretime::CoretimeRuntimeType::KusamaLocal |
chain_spec::coretime::CoretimeRuntimeType::Polkadot |
chain_spec::coretime::CoretimeRuntimeType::PolkadotLocal |
chain_spec::coretime::CoretimeRuntimeType::Rococo |
chain_spec::coretime::CoretimeRuntimeType::RococoLocal |
chain_spec::coretime::CoretimeRuntimeType::RococoDevelopment |
chain_spec::coretime::CoretimeRuntimeType::Westend |
chain_spec::coretime::CoretimeRuntimeType::WestendLocal |
chain_spec::coretime::CoretimeRuntimeType::WestendDevelopment =>
crate::service::start_generic_aura_lookahead_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0),
}
.map_err(Into::into),
Runtime::Penpal(_) | Runtime::Default =>
crate::service::start_rococo_parachain_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::Glutton | Runtime::GluttonWestend =>
crate::service::start_basic_lookahead_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into),
Runtime::People(people_runtime_type) => match people_runtime_type {
chain_spec::people::PeopleRuntimeType::Kusama |
chain_spec::people::PeopleRuntimeType::KusamaLocal |
chain_spec::people::PeopleRuntimeType::Polkadot |
chain_spec::people::PeopleRuntimeType::PolkadotLocal |
chain_spec::people::PeopleRuntimeType::Rococo |
chain_spec::people::PeopleRuntimeType::RococoLocal |
chain_spec::people::PeopleRuntimeType::RococoDevelopment |
chain_spec::people::PeopleRuntimeType::Westend |
chain_spec::people::PeopleRuntimeType::WestendLocal |
chain_spec::people::PeopleRuntimeType::WestendDevelopment =>
crate::service::start_generic_aura_lookahead_node::<Network>(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0),
}
.map_err(Into::into),
}
}
impl DefaultConfigurationValues for RelayChainCli {
fn p2p_listen_port() -> u16 {
30334
cumulus/polkadot-parachain/src/service.rs
+26 -19
View File
@@ -51,7 +51,7 @@ use sc_consensus::{
BlockImportParams, ImportQueue,
};
use sc_executor::{HeapAllocStrategy, WasmExecutor, DEFAULT_HEAP_ALLOC_STRATEGY};
use sc_network::{config::FullNetworkConfiguration, NetworkBlock};
use sc_network::{config::FullNetworkConfiguration, service::traits::NetworkBackend, NetworkBlock};
use sc_network_sync::SyncingService;
use sc_service::{Configuration, PartialComponents, TFullBackend, TFullClient, TaskManager};
use sc_telemetry::{Telemetry, TelemetryHandle, TelemetryWorker, TelemetryWorkerHandle};
@@ -191,7 +191,7 @@ where
///
/// This is the actual implementation that is abstract over the executor and the runtime api.
#[sc_tracing::logging::prefix_logs_with("Parachain")]
async fn start_node_impl<RuntimeApi, RB, BIQ, SC>(
async fn start_node_impl<RuntimeApi, RB, BIQ, SC, Net>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
@@ -244,6 +244,7 @@ where
Arc<dyn Fn(Hash, Option<Vec<u8>>) + Send + Sync>,
Arc<ParachainBackend>,
) -> Result<(), sc_service::Error>,
Net: NetworkBackend<Block, Hash>,
{
let parachain_config = prepare_node_config(parachain_config);
@@ -269,7 +270,7 @@ where
let prometheus_registry = parachain_config.prometheus_registry().cloned();
let transaction_pool = params.transaction_pool.clone();
let import_queue_service = params.import_queue.service();
let net_config = FullNetworkConfiguration::new(&parachain_config.network);
let net_config = FullNetworkConfiguration::<_, _, Net>::new(&parachain_config.network);
let (network, system_rpc_tx, tx_handler_controller, start_network, sync_service) =
build_network(BuildNetworkParams {
@@ -423,14 +424,14 @@ pub fn build_aura_import_queue(
}
/// Start a rococo parachain node.
pub async fn start_rococo_parachain_node(
pub async fn start_rococo_parachain_node<Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
para_id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(TaskManager, Arc<ParachainClient<FakeRuntimeApi>>)> {
start_node_impl::<FakeRuntimeApi, _, _, _>(
start_node_impl::<FakeRuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -492,14 +493,14 @@ fn build_contracts_rpc_extensions(
}
/// Start a polkadot-shell parachain node.
pub async fn start_shell_node(
pub async fn start_shell_node<Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
para_id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(TaskManager, Arc<ParachainClient<FakeRuntimeApi>>)> {
start_node_impl::<FakeRuntimeApi, _, _, _>(
start_node_impl::<FakeRuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -687,14 +688,14 @@ where
}
/// Start an aura powered parachain node. Some system chains use this.
pub async fn start_generic_aura_node(
pub async fn start_generic_aura_node<Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
para_id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(TaskManager, Arc<ParachainClient<FakeRuntimeApi>>)> {
start_node_impl::<FakeRuntimeApi, _, _, _>(
start_node_impl::<FakeRuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -768,14 +769,14 @@ pub async fn start_generic_aura_node(
/// Uses the lookahead collator to support async backing.
///
/// Start an aura powered parachain node. Some system chains use this.
pub async fn start_generic_aura_lookahead_node(
pub async fn start_generic_aura_lookahead_node<Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
para_id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(TaskManager, Arc<ParachainClient<FakeRuntimeApi>>)> {
start_node_impl::<FakeRuntimeApi, _, _, _>(
start_node_impl::<FakeRuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -792,7 +793,7 @@ pub async fn start_generic_aura_lookahead_node(
/// Start a shell node which should later transition into an Aura powered parachain node. Asset Hub
/// uses this because at genesis, Asset Hub was on the `shell` runtime which didn't have Aura and
/// needs to sync and upgrade before it can run `AuraApi` functions.
pub async fn start_asset_hub_node<RuntimeApi, AuraId: AppCrypto + Send + Codec + Sync>(
pub async fn start_asset_hub_node<RuntimeApi, AuraId: AppCrypto + Send + Codec + Sync, Net>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
@@ -813,8 +814,9 @@ where
+ frame_rpc_system::AccountNonceApi<Block, AccountId, Nonce>,
<<AuraId as AppCrypto>::Pair as Pair>::Signature:
TryFrom<Vec<u8>> + std::hash::Hash + sp_runtime::traits::Member + Codec,
Net: NetworkBackend<Block, Hash>,
{
start_node_impl::<RuntimeApi, _, _, _>(
start_node_impl::<RuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -940,7 +942,11 @@ where
///
/// Uses the lookahead collator to support async backing.
#[sc_tracing::logging::prefix_logs_with("Parachain")]
pub async fn start_asset_hub_lookahead_node<RuntimeApi, AuraId: AppCrypto + Send + Codec + Sync>(
pub async fn start_asset_hub_lookahead_node<
RuntimeApi,
AuraId: AppCrypto + Send + Codec + Sync,
Net,
>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
@@ -962,8 +968,9 @@ where
+ cumulus_primitives_aura::AuraUnincludedSegmentApi<Block>,
<<AuraId as AppCrypto>::Pair as Pair>::Signature:
TryFrom<Vec<u8>> + std::hash::Hash + sp_runtime::traits::Member + Codec,
Net: NetworkBackend<Block, Hash>,
{
start_node_impl::<RuntimeApi, _, _, _>(
start_node_impl::<RuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -1213,14 +1220,14 @@ fn start_lookahead_aura_consensus(
/// Start an aura powered parachain node which uses the lookahead collator to support async backing.
/// This node is basic in the sense that its runtime api doesn't include common contents such as
/// transaction payment. Used for aura glutton.
pub async fn start_basic_lookahead_node(
pub async fn start_basic_lookahead_node<Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
para_id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(TaskManager, Arc<ParachainClient<FakeRuntimeApi>>)> {
start_node_impl::<FakeRuntimeApi, _, _, _>(
start_node_impl::<FakeRuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
@@ -1235,14 +1242,14 @@ pub async fn start_basic_lookahead_node(
}
/// Start a parachain node for Rococo Contracts.
pub async fn start_contracts_rococo_node(
pub async fn start_contracts_rococo_node<Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
polkadot_config: Configuration,
collator_options: CollatorOptions,
para_id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(TaskManager, Arc<ParachainClient<FakeRuntimeApi>>)> {
start_node_impl::<FakeRuntimeApi, _, _, _>(
start_node_impl::<FakeRuntimeApi, _, _, _, Net>(
parachain_config,
polkadot_config,
collator_options,
cumulus/test/service/src/lib.rs
+40 -21
View File
@@ -62,7 +62,9 @@ use polkadot_service::ProvideRuntimeApi;
use sc_consensus::ImportQueue;
use sc_network::{
config::{FullNetworkConfiguration, TransportConfig},
multiaddr, NetworkBlock, NetworkService, NetworkStateInfo,
multiaddr,
service::traits::NetworkService,
NetworkBackend, NetworkBlock, NetworkStateInfo,
};
use sc_service::{
config::{
@@ -74,7 +76,7 @@ use sc_service::{
};
use sp_arithmetic::traits::SaturatedConversion;
use sp_blockchain::HeaderBackend;
use sp_core::{Pair, H256};
use sp_core::Pair;
use sp_keyring::Sr25519Keyring;
use sp_runtime::{codec::Encode, generic};
use sp_state_machine::BasicExternalities;
@@ -304,7 +306,7 @@ async fn build_relay_chain_interface(
///
/// This is the actual implementation that is abstract over the executor and the runtime api.
#[sc_tracing::logging::prefix_logs_with(parachain_config.network.node_name.as_str())]
pub async fn start_node_impl<RB>(
pub async fn start_node_impl<RB, Net: NetworkBackend<Block, Hash>>(
parachain_config: Configuration,
collator_key: Option<CollatorPair>,
relay_chain_config: Configuration,
@@ -318,7 +320,7 @@ pub async fn start_node_impl<RB>(
) -> sc_service::error::Result<(
TaskManager,
Arc<Client>,
Arc<NetworkService<Block, H256>>,
Arc<dyn NetworkService>,
RpcHandlers,
TransactionPool,
Arc<Backend>,
@@ -348,7 +350,7 @@ where
.map_err(|e| sc_service::Error::Application(Box::new(e) as Box<_>))?;
let import_queue_service = params.import_queue.service();
let net_config = FullNetworkConfiguration::new(&parachain_config.network);
let net_config = FullNetworkConfiguration::<Block, Hash, Net>::new(&parachain_config.network);
let (network, system_rpc_tx, tx_handler_controller, start_network, sync_service) =
build_network(BuildNetworkParams {
@@ -494,7 +496,7 @@ pub struct TestNode {
/// Client's instance.
pub client: Arc<Client>,
/// Node's network.
pub network: Arc<NetworkService<Block, H256>>,
pub network: Arc<dyn NetworkService>,
/// The `MultiaddrWithPeerId` to this node. This is useful if you want to pass it as "boot
/// node" to other nodes.
pub addr: MultiaddrWithPeerId,
@@ -702,21 +704,38 @@ impl TestNodeBuilder {
let multiaddr = parachain_config.network.listen_addresses[0].clone();
let (task_manager, client, network, rpc_handlers, transaction_pool, backend) =
start_node_impl(
parachain_config,
self.collator_key,
relay_chain_config,
self.para_id,
self.wrap_announce_block,
false,
|_| Ok(jsonrpsee::RpcModule::new(())),
self.consensus,
collator_options,
self.record_proof_during_import,
)
.await
.expect("could not create Cumulus test service");
match relay_chain_config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p =>
start_node_impl::<_, sc_network::NetworkWorker<_, _>>(
parachain_config,
self.collator_key,
relay_chain_config,
self.para_id,
self.wrap_announce_block,
false,
|_| Ok(jsonrpsee::RpcModule::new(())),
self.consensus,
collator_options,
self.record_proof_during_import,
)
.await
.expect("could not create Cumulus test service"),
sc_network::config::NetworkBackendType::Litep2p =>
start_node_impl::<_, sc_network::Litep2pNetworkBackend>(
parachain_config,
self.collator_key,
relay_chain_config,
self.para_id,
self.wrap_announce_block,
false,
|_| Ok(jsonrpsee::RpcModule::new(())),
self.consensus,
collator_options,
self.record_proof_during_import,
)
.await
.expect("could not create Cumulus test service"),
};
let peer_id = network.local_peer_id();
let addr = MultiaddrWithPeerId { multiaddr, peer_id };
cumulus/test/service/src/main.rs
+38 -12
View File
@@ -101,18 +101,44 @@ fn main() -> Result<(), sc_cli::Error> {
.unwrap_or(cumulus_test_service::Consensus::RelayChain);
let (mut task_manager, _, _, _, _, _) = tokio_runtime
.block_on(cumulus_test_service::start_node_impl(
config,
collator_key,
polkadot_config,
parachain_id.into(),
cli.disable_block_announcements.then(wrap_announce_block),
cli.fail_pov_recovery,
|_| Ok(jsonrpsee::RpcModule::new(())),
consensus,
collator_options,
true,
))
.block_on(async move {
match polkadot_config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p =>
cumulus_test_service::start_node_impl::<
_,
sc_network::NetworkWorker<_, _>,
>(
config,
collator_key,
polkadot_config,
parachain_id.into(),
cli.disable_block_announcements.then(wrap_announce_block),
cli.fail_pov_recovery,
|_| Ok(jsonrpsee::RpcModule::new(())),
consensus,
collator_options,
true,
)
.await,
sc_network::config::NetworkBackendType::Litep2p =>
cumulus_test_service::start_node_impl::<
_,
sc_network::Litep2pNetworkBackend,
>(
config,
collator_key,
polkadot_config,
parachain_id.into(),
cli.disable_block_announcements.then(wrap_announce_block),
cli.fail_pov_recovery,
|_| Ok(jsonrpsee::RpcModule::new(())),
consensus,
collator_options,
true,
)
.await,
}
})
.expect("could not create Cumulus test service");
tokio_runtime
polkadot/node/jaeger/Cargo.toml
+1
View File
@@ -16,6 +16,7 @@ parking_lot = "0.12.1"
polkadot-primitives = { path = "../../primitives" }
polkadot-node-primitives = { path = "../primitives" }
sc-network = { path = "../../../substrate/client/network" }
sc-network-types = { path = "../../../substrate/client/network/types" }
sp-core = { path = "../../../substrate/primitives/core" }
thiserror = { workspace = true }
tokio = "1.37"
polkadot/node/jaeger/src/spans.rs
+1 -1
View File
@@ -86,7 +86,7 @@
use parity_scale_codec::Encode;
use polkadot_node_primitives::PoV;
use polkadot_primitives::{BlakeTwo256, CandidateHash, Hash, HashT, Id as ParaId, ValidatorIndex};
use sc_network::PeerId;
use sc_network_types::PeerId;
use std::{fmt, sync::Arc};
polkadot/node/network/availability-distribution/src/tests/mod.rs
+9 -4
View File
@@ -19,7 +19,7 @@ use std::collections::HashSet;
use futures::{executor, future, Future};
use polkadot_node_network_protocol::request_response::{IncomingRequest, ReqProtocolNames};
use polkadot_primitives::{CoreState, Hash};
use polkadot_primitives::{Block, CoreState, Hash};
use sp_keystore::KeystorePtr;
use polkadot_node_subsystem_test_helpers as test_helpers;
@@ -44,9 +44,14 @@ fn test_harness<T: Future<Output = ()>>(
let genesis_hash = Hash::repeat_byte(0xff);
let req_protocol_names = ReqProtocolNames::new(&genesis_hash, None);
let (pov_req_receiver, pov_req_cfg) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (chunk_req_receiver, chunk_req_cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
let (pov_req_receiver, pov_req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (chunk_req_receiver, chunk_req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let subsystem = AvailabilityDistributionSubsystem::new(
keystore,
IncomingRequestReceivers { pov_req_receiver, chunk_req_receiver },
polkadot/node/network/availability-distribution/src/tests/state.rs
+1 -1
View File
@@ -337,7 +337,7 @@ fn to_incoming_req(
IncomingRequest::new(
// We don't really care:
network::PeerId::random(),
network::PeerId::random().into(),
payload,
tx,
)
polkadot/node/network/availability-recovery/src/tests.rs
+5 -2
View File
@@ -40,7 +40,7 @@ use polkadot_node_subsystem_test_helpers::{
};
use polkadot_node_subsystem_util::TimeoutExt;
use polkadot_primitives::{
AuthorityDiscoveryId, Hash, HeadData, IndexedVec, PersistedValidationData, ValidatorId,
AuthorityDiscoveryId, Block, Hash, HeadData, IndexedVec, PersistedValidationData, ValidatorId,
};
use polkadot_primitives_test_helpers::{dummy_candidate_receipt, dummy_hash};
@@ -52,7 +52,10 @@ const GENESIS_HASH: Hash = Hash::repeat_byte(0xff);
fn request_receiver(
req_protocol_names: &ReqProtocolNames,
) -> IncomingRequestReceiver<AvailableDataFetchingRequest> {
let receiver = IncomingRequest::get_config_receiver(req_protocol_names);
let receiver = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(req_protocol_names);
// Don't close the sending end of the request protocol. Otherwise, the subsystem will terminate.
std::mem::forget(receiver.1.inbound_queue);
receiver.0
polkadot/node/network/bridge/src/network.rs
+12 -13
View File
@@ -25,9 +25,8 @@ use parking_lot::Mutex;
use parity_scale_codec::Encode;
use sc_network::{
config::parse_addr, multiaddr::Multiaddr, types::ProtocolName, IfDisconnected, MessageSink,
NetworkPeers, NetworkRequest, NetworkService, OutboundFailure, ReputationChange,
RequestFailure,
config::parse_addr, multiaddr::Multiaddr, service::traits::NetworkService, types::ProtocolName,
IfDisconnected, MessageSink, OutboundFailure, ReputationChange, RequestFailure,
};
use polkadot_node_network_protocol::{
@@ -35,7 +34,7 @@ use polkadot_node_network_protocol::{
request_response::{OutgoingRequest, Recipient, ReqProtocolNames, Requests},
v1 as protocol_v1, v2 as protocol_v2, v3 as protocol_v3, PeerId,
};
use polkadot_primitives::{AuthorityDiscoveryId, Block, Hash};
use polkadot_primitives::AuthorityDiscoveryId;
use crate::{metrics::Metrics, validator_discovery::AuthorityDiscovery, WireMessage};
@@ -232,13 +231,13 @@ pub trait Network: Clone + Send + 'static {
}
#[async_trait]
impl Network for Arc<NetworkService<Block, Hash>> {
impl Network for Arc<dyn NetworkService> {
async fn set_reserved_peers(
&mut self,
protocol: ProtocolName,
multiaddresses: HashSet<Multiaddr>,
) -> Result<(), String> {
NetworkService::set_reserved_peers(&**self, protocol, multiaddresses)
<dyn NetworkService>::set_reserved_peers(&**self, protocol, multiaddresses)
}
async fn remove_from_peers_set(
@@ -246,15 +245,15 @@ impl Network for Arc<NetworkService<Block, Hash>> {
protocol: ProtocolName,
peers: Vec<PeerId>,
) -> Result<(), String> {
NetworkService::remove_peers_from_reserved_set(&**self, protocol, peers)
<dyn NetworkService>::remove_peers_from_reserved_set(&**self, protocol, peers)
}
fn report_peer(&self, who: PeerId, rep: ReputationChange) {
NetworkService::report_peer(&**self, who, rep);
<dyn NetworkService>::report_peer(&**self, who, rep);
}
fn disconnect_peer(&self, who: PeerId, protocol: ProtocolName) {
NetworkService::disconnect_peer(&**self, who, protocol);
<dyn NetworkService>::disconnect_peer(&**self, who, protocol);
}
async fn start_request<AD: AuthorityDiscovery>(
@@ -289,7 +288,7 @@ impl Network for Arc<NetworkService<Block, Hash>> {
Ok(v) => v,
Err(_) => continue,
};
NetworkService::add_known_address(self, peer_id, addr);
<dyn NetworkService>::add_known_address(&**self, peer_id, addr);
found_peer_id = Some(peer_id);
}
found_peer_id
@@ -321,8 +320,8 @@ impl Network for Arc<NetworkService<Block, Hash>> {
"Starting request",
);
NetworkService::start_request(
self,
<dyn NetworkService>::start_request(
&**self,
peer_id,
req_protocol_names.get_name(protocol),
payload,
@@ -333,7 +332,7 @@ impl Network for Arc<NetworkService<Block, Hash>> {
}
fn peer_role(&self, who: PeerId, handshake: Vec<u8>) -> Option<sc_network::ObservedRole> {
NetworkService::peer_role(self, who, handshake)
<dyn NetworkService>::peer_role(&**self, who, handshake)
}
}
polkadot/node/network/bridge/src/rx/tests.rs
+2 -2
View File
@@ -366,13 +366,13 @@ impl NotificationService for TestNotificationService {
}
/// Send synchronous `notification` to `peer`.
fn send_sync_notification(&self, _peer: &PeerId, _notification: Vec<u8>) {
fn send_sync_notification(&mut self, _peer: &PeerId, _notification: Vec<u8>) {
unimplemented!();
}
/// Send asynchronous `notification` to `peer`, allowing sender to exercise backpressure.
async fn send_async_notification(
&self,
&mut self,
_peer: &PeerId,
_notification: Vec<u8>,
) -> Result<(), sc_network::error::Error> {
polkadot/node/network/collator-protocol/src/collator_side/tests/mod.rs
+11 -6
View File
@@ -45,8 +45,9 @@ use polkadot_node_subsystem::{
use polkadot_node_subsystem_test_helpers as test_helpers;
use polkadot_node_subsystem_util::{reputation::add_reputation, TimeoutExt};
use polkadot_primitives::{
AuthorityDiscoveryId, CollatorPair, ExecutorParams, GroupIndex, GroupRotationInfo, IndexedVec,
NodeFeatures, ScheduledCore, SessionIndex, SessionInfo, ValidatorId, ValidatorIndex,
AuthorityDiscoveryId, Block, CollatorPair, ExecutorParams, GroupIndex, GroupRotationInfo,
IndexedVec, NodeFeatures, ScheduledCore, SessionIndex, SessionInfo, ValidatorId,
ValidatorIndex,
};
use polkadot_primitives_test_helpers::TestCandidateBuilder;
use test_helpers::mock::new_leaf;
@@ -249,10 +250,14 @@ fn test_harness<T: Future<Output = TestHarness>>(
let genesis_hash = Hash::repeat_byte(0xff);
let req_protocol_names = ReqProtocolNames::new(&genesis_hash, None);
let (collation_req_receiver, req_v1_cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
let (collation_req_v2_receiver, req_v2_cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
let (collation_req_receiver, req_v1_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (collation_req_v2_receiver, req_v2_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let subsystem = async {
run_inner(
context,
polkadot/node/network/dispute-distribution/src/tests/mod.rs
+6 -3
View File
@@ -57,8 +57,8 @@ use polkadot_node_subsystem_test_helpers::{
subsystem_test_harness, TestSubsystemContextHandle,
};
use polkadot_primitives::{
AuthorityDiscoveryId, CandidateHash, CandidateReceipt, ExecutorParams, Hash, NodeFeatures,
SessionIndex, SessionInfo,
AuthorityDiscoveryId, Block, CandidateHash, CandidateReceipt, ExecutorParams, Hash,
NodeFeatures, SessionIndex, SessionInfo,
};
use self::mock::{
@@ -879,7 +879,10 @@ where
let genesis_hash = Hash::repeat_byte(0xff);
let req_protocol_names = ReqProtocolNames::new(&genesis_hash, None);
let (req_receiver, req_cfg) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (req_receiver, req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let subsystem = DisputeDistributionSubsystem::new(
keystore,
req_receiver,
polkadot/node/network/protocol/Cargo.toml
+2
View File
@@ -18,7 +18,9 @@ polkadot-node-primitives = { path = "../../primitives" }
polkadot-node-jaeger = { path = "../../jaeger" }
parity-scale-codec = { version = "3.6.1", default-features = false, features = ["derive"] }
sc-network = { path = "../../../../substrate/client/network" }
sc-network-types = { path = "../../../../substrate/client/network/types" }
sc-authority-discovery = { path = "../../../../substrate/client/authority-discovery" }
sp-runtime = { path = "../../../../substrate/primitives/runtime" }
strum = { version = "0.26.2", features = ["derive"] }
futures = "0.3.30"
thiserror = { workspace = true }
polkadot/node/network/protocol/src/authority_discovery.rs
+2 -1
View File
@@ -23,7 +23,8 @@ use async_trait::async_trait;
use sc_authority_discovery::Service as AuthorityDiscoveryService;
use polkadot_primitives::AuthorityDiscoveryId;
use sc_network::{Multiaddr, PeerId};
use sc_network::Multiaddr;
use sc_network_types::PeerId;
/// An abstraction over the authority discovery service.
///
polkadot/node/network/protocol/src/lib.rs
+4 -3
View File
@@ -25,7 +25,8 @@ use std::{collections::HashMap, fmt};
#[doc(hidden)]
pub use polkadot_node_jaeger as jaeger;
pub use sc_network::{IfDisconnected, PeerId};
pub use sc_network::IfDisconnected;
pub use sc_network_types::PeerId;
#[doc(hidden)]
pub use std::sync::Arc;
@@ -610,7 +611,7 @@ pub mod v1 {
///
/// The payload is the local peer id of the node, which serves to prove that it
/// controls the collator key it is declaring an intention to collate under.
pub fn declare_signature_payload(peer_id: &sc_network::PeerId) -> Vec<u8> {
pub fn declare_signature_payload(peer_id: &sc_network_types::PeerId) -> Vec<u8> {
let mut payload = peer_id.to_bytes();
payload.extend_from_slice(b"COLL");
payload
@@ -863,7 +864,7 @@ pub mod v2 {
///
/// The payload is the local peer id of the node, which serves to prove that it
/// controls the collator key it is declaring an intention to collate under.
pub fn declare_signature_payload(peer_id: &sc_network::PeerId) -> Vec<u8> {
pub fn declare_signature_payload(peer_id: &sc_network_types::PeerId) -> Vec<u8> {
let mut payload = peer_id.to_bytes();
payload.extend_from_slice(b"COLL");
payload
polkadot/node/network/protocol/src/peer_set.rs
+26 -10
View File
@@ -19,13 +19,14 @@
use derive_more::Display;
use polkadot_primitives::Hash;
use sc_network::{
config::{NonDefaultSetConfig, SetConfig},
types::ProtocolName,
NotificationService,
config::SetConfig, peer_store::PeerStoreProvider, service::NotificationMetrics,
types::ProtocolName, NetworkBackend, NotificationService,
};
use sp_runtime::traits::Block;
use std::{
collections::{hash_map::Entry, HashMap},
ops::{Index, IndexMut},
sync::Arc,
};
use strum::{EnumIter, IntoEnumIterator};
@@ -65,11 +66,13 @@ impl PeerSet {
///
/// Those should be used in the network configuration to register the protocols with the
/// network service.
pub fn get_info(
pub fn get_info<B: Block, N: NetworkBackend<B, <B as Block>::Hash>>(
self,
is_authority: IsAuthority,
peerset_protocol_names: &PeerSetProtocolNames,
) -> (NonDefaultSetConfig, (PeerSet, Box<dyn NotificationService>)) {
metrics: NotificationMetrics,
peer_store_handle: Arc<dyn PeerStoreProvider>,
) -> (N::NotificationProtocolConfig, (PeerSet, Box<dyn NotificationService>)) {
// Networking layer relies on `get_main_name()` being the main name of the protocol
// for peersets and connection management.
let protocol = peerset_protocol_names.get_main_name(self);
@@ -82,7 +85,7 @@ impl PeerSet {
match self {
PeerSet::Validation => {
let (config, notification_service) = NonDefaultSetConfig::new(
let (config, notification_service) = N::notification_config(
protocol,
fallback_names,
max_notification_size,
@@ -97,12 +100,14 @@ impl PeerSet {
reserved_nodes: Vec::new(),
non_reserved_mode: sc_network::config::NonReservedPeerMode::Accept,
},
metrics,
peer_store_handle,
);
(config, (PeerSet::Validation, notification_service))
},
PeerSet::Collation => {
let (config, notification_service) = NonDefaultSetConfig::new(
let (config, notification_service) = N::notification_config(
protocol,
fallback_names,
max_notification_size,
@@ -119,6 +124,8 @@ impl PeerSet {
sc_network::config::NonReservedPeerMode::Deny
},
},
metrics,
peer_store_handle,
);
(config, (PeerSet::Collation, notification_service))
@@ -207,12 +214,21 @@ impl<T> IndexMut<PeerSet> for PerPeerSet<T> {
///
/// Should be used during network configuration (added to `NetworkConfiguration::extra_sets`)
/// or shortly after startup to register the protocols with the network service.
pub fn peer_sets_info(
pub fn peer_sets_info<B: Block, N: NetworkBackend<B, <B as Block>::Hash>>(
is_authority: IsAuthority,
peerset_protocol_names: &PeerSetProtocolNames,
) -> Vec<(NonDefaultSetConfig, (PeerSet, Box<dyn NotificationService>))> {
metrics: NotificationMetrics,
peer_store_handle: Arc<dyn PeerStoreProvider>,
) -> Vec<(N::NotificationProtocolConfig, (PeerSet, Box<dyn NotificationService>))> {
PeerSet::iter()
.map(|s| s.get_info(is_authority, &peerset_protocol_names))
.map(|s| {
s.get_info::<B, N>(
is_authority,
&peerset_protocol_names,
metrics.clone(),
Arc::clone(&peer_store_handle),
)
})
.collect()
}
polkadot/node/network/protocol/src/request_response/incoming/error.rs
+1 -1
View File
@@ -16,7 +16,7 @@
//! Error handling related code and Error/Result definitions.
use sc_network::PeerId;
use sc_network_types::PeerId;
use parity_scale_codec::Error as DecodingError;
polkadot/node/network/protocol/src/request_response/incoming/mod.rs
+6 -4
View File
@@ -20,7 +20,9 @@ use futures::{channel::oneshot, StreamExt};
use parity_scale_codec::{Decode, Encode};
use sc_network::{config as netconfig, config::RequestResponseConfig, PeerId};
use sc_network::{config as netconfig, NetworkBackend};
use sc_network_types::PeerId;
use sp_runtime::traits::Block;
use super::{IsRequest, ReqProtocolNames};
use crate::UnifiedReputationChange;
@@ -52,10 +54,10 @@ where
///
/// This Register that config with substrate networking and receive incoming requests via the
/// returned `IncomingRequestReceiver`.
pub fn get_config_receiver(
pub fn get_config_receiver<B: Block, N: NetworkBackend<B, <B as Block>::Hash>>(
req_protocol_names: &ReqProtocolNames,
) -> (IncomingRequestReceiver<Req>, RequestResponseConfig) {
let (raw, cfg) = Req::PROTOCOL.get_config(req_protocol_names);
) -> (IncomingRequestReceiver<Req>, N::RequestResponseProtocolConfig) {
let (raw, cfg) = Req::PROTOCOL.get_config::<B, N>(req_protocol_names);
(IncomingRequestReceiver { raw, phantom: PhantomData {} }, cfg)
}
polkadot/node/network/protocol/src/request_response/mod.rs
+59 -57
View File
@@ -52,6 +52,8 @@
use std::{collections::HashMap, time::Duration, u64};
use polkadot_primitives::{MAX_CODE_SIZE, MAX_POV_SIZE};
use sc_network::NetworkBackend;
use sp_runtime::traits::Block;
use strum::{EnumIter, IntoEnumIterator};
pub use sc_network::{config as network, config::RequestResponseConfig, ProtocolName};
@@ -179,76 +181,76 @@ impl Protocol {
///
/// Returns a `ProtocolConfig` for this protocol.
/// Use this if you plan only to send requests for this protocol.
pub fn get_outbound_only_config(
pub fn get_outbound_only_config<B: Block, N: NetworkBackend<B, <B as Block>::Hash>>(
self,
req_protocol_names: &ReqProtocolNames,
) -> RequestResponseConfig {
self.create_config(req_protocol_names, None)
) -> N::RequestResponseProtocolConfig {
self.create_config::<B, N>(req_protocol_names, None)
}
/// Get a configuration for a given Request response protocol.
///
/// Returns a receiver for messages received on this protocol and the requested
/// `ProtocolConfig`.
pub fn get_config(
pub fn get_config<B: Block, N: NetworkBackend<B, <B as Block>::Hash>>(
self,
req_protocol_names: &ReqProtocolNames,
) -> (async_channel::Receiver<network::IncomingRequest>, RequestResponseConfig) {
) -> (async_channel::Receiver<network::IncomingRequest>, N::RequestResponseProtocolConfig) {
let (tx, rx) = async_channel::bounded(self.get_channel_size());
let cfg = self.create_config(req_protocol_names, Some(tx));
let cfg = self.create_config::<B, N>(req_protocol_names, Some(tx));
(rx, cfg)
}
fn create_config(
fn create_config<B: Block, N: NetworkBackend<B, <B as Block>::Hash>>(
self,
req_protocol_names: &ReqProtocolNames,
tx: Option<async_channel::Sender<network::IncomingRequest>>,
) -> RequestResponseConfig {
) -> N::RequestResponseProtocolConfig {
let name = req_protocol_names.get_name(self);
let legacy_names = self.get_legacy_name().into_iter().map(Into::into).collect();
match self {
Protocol::ChunkFetchingV1 => RequestResponseConfig {
Protocol::ChunkFetchingV1 => N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
max_response_size: POV_RESPONSE_SIZE as u64 * 3,
legacy_names,
1_000,
POV_RESPONSE_SIZE as u64 * 3,
// We are connected to all validators:
request_timeout: CHUNK_REQUEST_TIMEOUT,
inbound_queue: tx,
},
CHUNK_REQUEST_TIMEOUT,
tx,
),
Protocol::CollationFetchingV1 | Protocol::CollationFetchingV2 =>
RequestResponseConfig {
N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
max_response_size: POV_RESPONSE_SIZE,
legacy_names,
1_000,
POV_RESPONSE_SIZE,
// Taken from initial implementation in collator protocol:
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
inbound_queue: tx,
},
Protocol::PoVFetchingV1 => RequestResponseConfig {
POV_REQUEST_TIMEOUT_CONNECTED,
tx,
),
Protocol::PoVFetchingV1 => N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
max_response_size: POV_RESPONSE_SIZE,
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
inbound_queue: tx,
},
Protocol::AvailableDataFetchingV1 => RequestResponseConfig {
legacy_names,
1_000,
POV_RESPONSE_SIZE,
POV_REQUEST_TIMEOUT_CONNECTED,
tx,
),
Protocol::AvailableDataFetchingV1 => N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
legacy_names,
1_000,
// Available data size is dominated by the PoV size.
max_response_size: POV_RESPONSE_SIZE,
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
inbound_queue: tx,
},
Protocol::StatementFetchingV1 => RequestResponseConfig {
POV_RESPONSE_SIZE,
POV_REQUEST_TIMEOUT_CONNECTED,
tx,
),
Protocol::StatementFetchingV1 => N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
legacy_names,
1_000,
// Available data size is dominated code size.
max_response_size: STATEMENT_RESPONSE_SIZE,
STATEMENT_RESPONSE_SIZE,
// We need statement fetching to be fast and will try our best at the responding
// side to answer requests within that timeout, assuming a bandwidth of 500Mbit/s
// - which is the recommended minimum bandwidth for nodes on Kusama as of April
@@ -258,27 +260,27 @@ impl Protocol {
// waiting for timeout on an overloaded node. Fetches from slow nodes will likely
// fail, but this is desired, so we can quickly move on to a faster one - we should
// also decrease its reputation.
request_timeout: Duration::from_secs(1),
inbound_queue: tx,
},
Protocol::DisputeSendingV1 => RequestResponseConfig {
Duration::from_secs(1),
tx,
),
Protocol::DisputeSendingV1 => N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
legacy_names,
1_000,
// Responses are just confirmation, in essence not even a bit. So 100 seems
// plenty.
max_response_size: 100,
request_timeout: DISPUTE_REQUEST_TIMEOUT,
inbound_queue: tx,
},
Protocol::AttestedCandidateV2 => RequestResponseConfig {
100,
DISPUTE_REQUEST_TIMEOUT,
tx,
),
Protocol::AttestedCandidateV2 => N::request_response_config(
name,
fallback_names: legacy_names,
max_request_size: 1_000,
max_response_size: ATTESTED_CANDIDATE_RESPONSE_SIZE,
request_timeout: ATTESTED_CANDIDATE_TIMEOUT,
inbound_queue: tx,
},
legacy_names,
1_000,
ATTESTED_CANDIDATE_RESPONSE_SIZE,
ATTESTED_CANDIDATE_TIMEOUT,
tx,
),
}
}
polkadot/node/network/protocol/src/request_response/outgoing.rs
+1 -1
View File
@@ -20,7 +20,7 @@ use network::ProtocolName;
use parity_scale_codec::{Decode, Encode, Error as DecodingError};
use sc_network as network;
use sc_network::PeerId;
use sc_network_types::PeerId;
use polkadot_primitives::AuthorityDiscoveryId;
polkadot/node/network/statement-distribution/src/legacy_v1/tests.rs
+49 -15
View File
@@ -43,7 +43,7 @@ use polkadot_node_subsystem::{
};
use polkadot_node_subsystem_test_helpers::mock::{make_ferdie_keystore, new_leaf};
use polkadot_primitives::{
ExecutorParams, GroupIndex, Hash, HeadData, Id as ParaId, IndexedVec, NodeFeatures,
Block, ExecutorParams, GroupIndex, Hash, HeadData, Id as ParaId, IndexedVec, NodeFeatures,
SessionInfo, ValidationCode,
};
use polkadot_primitives_test_helpers::{
@@ -768,8 +768,14 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
let (ctx, mut handle) = polkadot_node_subsystem_test_helpers::make_subsystem_context(pool);
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let bg = async move {
let s = StatementDistributionSubsystem {
@@ -1016,9 +1022,14 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
let (ctx, mut handle) = polkadot_node_subsystem_test_helpers::make_subsystem_context(pool);
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, mut req_cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, mut req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let bg = async move {
let s = StatementDistributionSubsystem {
@@ -1554,8 +1565,14 @@ fn delay_reputation_changes() {
let (ctx, mut handle) = polkadot_node_subsystem_test_helpers::make_subsystem_context(pool);
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let reputation_interval = Duration::from_millis(100);
@@ -2044,9 +2061,14 @@ fn share_prioritizes_backing_group() {
let (ctx, mut handle) = polkadot_node_subsystem_test_helpers::make_subsystem_context(pool);
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, mut req_cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, mut req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let bg = async move {
let s = StatementDistributionSubsystem {
@@ -2377,8 +2399,14 @@ fn peer_cant_flood_with_large_statements() {
let (ctx, mut handle) = polkadot_node_subsystem_test_helpers::make_subsystem_context(pool);
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let bg = async move {
let s = StatementDistributionSubsystem {
keystore: make_ferdie_keystore(),
@@ -2610,8 +2638,14 @@ fn handle_multiple_seconded_statements() {
let (ctx, mut handle) = polkadot_node_subsystem_test_helpers::make_subsystem_context(pool);
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let virtual_overseer_fut = async move {
let s = StatementDistributionSubsystem {
polkadot/node/network/statement-distribution/src/v2/tests/mod.rs
+9 -4
View File
@@ -33,7 +33,7 @@ use polkadot_node_subsystem::messages::{
use polkadot_node_subsystem_test_helpers as test_helpers;
use polkadot_node_subsystem_util::TimeoutExt;
use polkadot_primitives::{
AssignmentPair, AsyncBackingParams, BlockNumber, CommittedCandidateReceipt, CoreState,
AssignmentPair, AsyncBackingParams, Block, BlockNumber, CommittedCandidateReceipt, CoreState,
GroupRotationInfo, HeadData, Header, IndexedVec, PersistedValidationData, ScheduledCore,
SessionIndex, SessionInfo, ValidatorPair,
};
@@ -359,9 +359,14 @@ fn test_harness<T: Future<Output = VirtualOverseer>>(
Arc::new(LocalKeystore::in_memory()) as KeystorePtr
};
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (candidate_req_receiver, req_cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let (candidate_req_receiver, req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&req_protocol_names);
let mut rng = rand_chacha::ChaCha8Rng::seed_from_u64(0);
let test_state = TestState::from_config(config, req_cfg.inbound_queue.unwrap(), &mut rng);
polkadot/node/service/src/lib.rs
+56 -25
View File
@@ -655,7 +655,7 @@ pub struct NewFull {
pub task_manager: TaskManager,
pub client: Arc<FullClient>,
pub overseer_handle: Option<Handle>,
pub network: Arc<sc_network::NetworkService<Block, <Block as BlockT>::Hash>>,
pub network: Arc<dyn sc_network::service::traits::NetworkService>,
pub sync_service: Arc<sc_network_sync::SyncingService<Block>>,
pub rpc_handlers: RpcHandlers,
pub backend: Arc<FullBackend>,
@@ -719,7 +719,10 @@ pub const AVAILABILITY_CONFIG: AvailabilityConfig = AvailabilityConfig {
/// searched. If the path points to an executable rather then directory, that executable is used
/// both as preparation and execution worker (supposed to be used for tests only).
#[cfg(feature = "full-node")]
pub fn new_full<OverseerGenerator: OverseerGen>(
pub fn new_full<
OverseerGenerator: OverseerGen,
Network: sc_network::NetworkBackend<Block, <Block as BlockT>::Hash>,
>(
mut config: Configuration,
NewFullParams {
is_parachain_node,
@@ -805,19 +808,29 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
other: (rpc_extensions_builder, import_setup, rpc_setup, slot_duration, mut telemetry),
} = new_partial::<SelectRelayChain<_>>(&mut config, basics, select_chain)?;
let metrics = Network::register_notification_metrics(
config.prometheus_config.as_ref().map(|cfg| &cfg.registry),
);
let shared_voter_state = rpc_setup;
let auth_disc_publish_non_global_ips = config.network.allow_non_globals_in_dht;
let auth_disc_public_addresses = config.network.public_addresses.clone();
let mut net_config = sc_network::config::FullNetworkConfiguration::new(&config.network);
let mut net_config =
sc_network::config::FullNetworkConfiguration::<_, _, Network>::new(&config.network);
let genesis_hash = client.block_hash(0).ok().flatten().expect("Genesis block exists; qed");
let peer_store_handle = net_config.peer_store_handle();
// Note: GrandPa is pushed before the Polkadot-specific protocols. This doesn't change
// anything in terms of behaviour, but makes the logs more consistent with the other
// Substrate nodes.
let grandpa_protocol_name = grandpa::protocol_standard_name(&genesis_hash, &config.chain_spec);
let (grandpa_protocol_config, grandpa_notification_service) =
grandpa::grandpa_peers_set_config(grandpa_protocol_name.clone());
grandpa::grandpa_peers_set_config::<_, Network>(
grandpa_protocol_name.clone(),
metrics.clone(),
Arc::clone(&peer_store_handle),
);
net_config.add_notification_protocol(grandpa_protocol_config);
let beefy_gossip_proto_name =
@@ -825,7 +838,7 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
// `beefy_on_demand_justifications_handler` is given to `beefy-gadget` task to be run,
// while `beefy_req_resp_cfg` is added to `config.network.request_response_protocols`.
let (beefy_on_demand_justifications_handler, beefy_req_resp_cfg) =
beefy::communication::request_response::BeefyJustifsRequestHandler::new(
beefy::communication::request_response::BeefyJustifsRequestHandler::new::<_, Network>(
&genesis_hash,
config.chain_spec.fork_id(),
client.clone(),
@@ -835,7 +848,11 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
false => None,
true => {
let (beefy_notification_config, beefy_notification_service) =
beefy::communication::beefy_peers_set_config(beefy_gossip_proto_name.clone());
beefy::communication::beefy_peers_set_config::<_, Network>(
beefy_gossip_proto_name.clone(),
metrics.clone(),
Arc::clone(&peer_store_handle),
);
net_config.add_notification_protocol(beefy_notification_config);
net_config.add_request_response_protocol(beefy_req_resp_cfg);
@@ -857,13 +874,18 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
use polkadot_network_bridge::{peer_sets_info, IsAuthority};
let is_authority = if role.is_authority() { IsAuthority::Yes } else { IsAuthority::No };
peer_sets_info(is_authority, &peerset_protocol_names)
.into_iter()
.map(|(config, (peerset, service))| {
net_config.add_notification_protocol(config);
(peerset, service)
})
.collect::<HashMap<PeerSet, Box<dyn sc_network::NotificationService>>>()
peer_sets_info::<_, Network>(
is_authority,
&peerset_protocol_names,
metrics.clone(),
Arc::clone(&peer_store_handle),
)
.into_iter()
.map(|(config, (peerset, service))| {
net_config.add_notification_protocol(config);
(peerset, service)
})
.collect::<HashMap<PeerSet, Box<dyn sc_network::NotificationService>>>()
} else {
std::collections::HashMap::new()
};
@@ -871,17 +893,19 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
let req_protocol_names = ReqProtocolNames::new(&genesis_hash, config.chain_spec.fork_id());
let (collation_req_v1_receiver, cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let (collation_req_v2_receiver, cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let (available_data_req_receiver, cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let (pov_req_receiver, cfg) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (pov_req_receiver, cfg) =
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let (chunk_req_receiver, cfg) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (chunk_req_receiver, cfg) =
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let grandpa_hard_forks = if config.chain_spec.is_kusama() {
@@ -924,12 +948,13 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
None
};
let (statement_req_receiver, cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let (candidate_req_v2_receiver, cfg) =
IncomingRequest::get_config_receiver(&req_protocol_names);
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let (dispute_req_receiver, cfg) = IncomingRequest::get_config_receiver(&req_protocol_names);
let (dispute_req_receiver, cfg) =
IncomingRequest::get_config_receiver::<_, Network>(&req_protocol_names);
net_config.add_request_response_protocol(cfg);
let approval_voting_config = ApprovalVotingConfig {
col_approval_data: parachains_db::REAL_COLUMNS.col_approval_data,
@@ -970,6 +995,7 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
block_announce_validator_builder: None,
warp_sync_params: Some(WarpSyncParams::WithProvider(warp_sync)),
block_relay: None,
metrics,
})?;
if config.offchain_worker.enabled {
@@ -985,7 +1011,7 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
transaction_pool: Some(OffchainTransactionPoolFactory::new(
transaction_pool.clone(),
)),
network_provider: network.clone(),
network_provider: Arc::new(network.clone()),
is_validator: role.is_authority(),
enable_http_requests: false,
custom_extensions: move |_| vec![],
@@ -1068,7 +1094,7 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
..Default::default()
},
client.clone(),
network.clone(),
Arc::new(network.clone()),
Box::pin(dht_event_stream),
authority_discovery_role,
prometheus_registry.clone(),
@@ -1214,7 +1240,7 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
if let Some(notification_service) = beefy_notification_service {
let justifications_protocol_name = beefy_on_demand_justifications_handler.protocol_name();
let network_params = beefy::BeefyNetworkParams {
network: network.clone(),
network: Arc::new(network.clone()),
sync: sync_service.clone(),
gossip_protocol_name: beefy_gossip_proto_name,
justifications_protocol_name,
@@ -1383,7 +1409,12 @@ pub fn build_full<OverseerGenerator: OverseerGen>(
capacity
});
new_full(config, params)
match config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p =>
new_full::<_, sc_network::NetworkWorker<Block, Hash>>(config, params),
sc_network::config::NetworkBackendType::Litep2p =>
new_full::<_, sc_network::Litep2pNetworkBackend>(config, params),
}
}
/// Reverts the node state down to at most the last finalized block.
polkadot/node/service/src/overseer.rs
+6 -6
View File
@@ -14,7 +14,7 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
use super::{Block, Error, Hash, IsParachainNode, Registry};
use super::{Error, IsParachainNode, Registry};
use polkadot_node_subsystem_types::{ChainApiBackend, RuntimeApiSubsystemClient};
use polkadot_overseer::{DummySubsystem, InitializedOverseerBuilder, SubsystemError};
use sp_core::traits::SpawnNamed;
@@ -80,7 +80,7 @@ where
/// Runtime client generic, providing the `ProvideRuntimeApi` trait besides others.
pub runtime_client: Arc<RuntimeClient>,
/// Underlying network service implementation.
pub network_service: Arc<sc_network::NetworkService<Block, Hash>>,
pub network_service: Arc<dyn sc_network::service::traits::NetworkService>,
/// Underlying syncing service implementation.
pub sync_service: Arc<dyn consensus_common::SyncOracle + Send + Sync>,
/// Underlying authority discovery service.
@@ -183,11 +183,11 @@ pub fn validator_overseer_builder<Spawner, RuntimeClient>(
RuntimeApiSubsystem<RuntimeClient>,
AvailabilityStoreSubsystem,
NetworkBridgeRxSubsystem<
Arc<sc_network::NetworkService<Block, Hash>>,
Arc<dyn sc_network::service::traits::NetworkService>,
AuthorityDiscoveryService,
>,
NetworkBridgeTxSubsystem<
Arc<sc_network::NetworkService<Block, Hash>>,
Arc<dyn sc_network::service::traits::NetworkService>,
AuthorityDiscoveryService,
>,
ChainApiSubsystem<RuntimeClient>,
@@ -369,11 +369,11 @@ pub fn collator_overseer_builder<Spawner, RuntimeClient>(
RuntimeApiSubsystem<RuntimeClient>,
DummySubsystem,
NetworkBridgeRxSubsystem<
Arc<sc_network::NetworkService<Block, Hash>>,
Arc<dyn sc_network::service::traits::NetworkService>,
AuthorityDiscoveryService,
>,
NetworkBridgeTxSubsystem<
Arc<sc_network::NetworkService<Block, Hash>>,
Arc<dyn sc_network::service::traits::NetworkService>,
AuthorityDiscoveryService,
>,
ChainApiSubsystem<RuntimeClient>,
polkadot/node/subsystem-bench/Cargo.toml
+1
View File
@@ -62,6 +62,7 @@ polkadot-node-subsystem-test-helpers = { path = "../subsystem-test-helpers" }
sp-keyring = { path = "../../../substrate/primitives/keyring" }
sp-application-crypto = { path = "../../../substrate/primitives/application-crypto" }
sc-network = { path = "../../../substrate/client/network" }
sc-network-types = { path = "../../../substrate/client/network/types" }
sc-service = { path = "../../../substrate/client/service" }
sp-consensus = { path = "../../../substrate/primitives/consensus/common" }
polkadot-node-metrics = { path = "../metrics" }
polkadot/node/subsystem-bench/src/lib/approval/helpers.rs
+1 -1
View File
@@ -32,7 +32,7 @@ use polkadot_primitives::{
use polkadot_primitives_test_helpers::dummy_candidate_receipt_bad_sig;
use rand::{seq::SliceRandom, SeedableRng};
use rand_chacha::ChaCha20Rng;
use sc_network::PeerId;
use sc_network_types::PeerId;
use sp_consensus_babe::{
digests::{CompatibleDigestItem, PreDigest, SecondaryVRFPreDigest},
AllowedSlots, BabeEpochConfiguration, Epoch as BabeEpoch, VrfSignature, VrfTranscript,
polkadot/node/subsystem-bench/src/lib/approval/message_generator.rs
+1 -1
View File
@@ -48,7 +48,7 @@ use rand::{seq::SliceRandom, RngCore, SeedableRng};
use rand_chacha::ChaCha20Rng;
use rand_distr::{Distribution, Normal};
use sc_keystore::LocalKeystore;
use sc_network::PeerId;
use sc_network_types::PeerId;
use sc_service::SpawnTaskHandle;
use sha1::Digest;
use sp_application_crypto::AppCrypto;
polkadot/node/subsystem-bench/src/lib/approval/test_message.rs
+1 -1
View File
@@ -22,7 +22,7 @@ use itertools::Itertools;
use parity_scale_codec::{Decode, Encode};
use polkadot_node_network_protocol::v3 as protocol_v3;
use polkadot_primitives::{CandidateIndex, Hash, ValidatorIndex};
use sc_network::PeerId;
use sc_network_types::PeerId;
use std::collections::{HashMap, HashSet};
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
polkadot/node/subsystem-bench/src/lib/availability/mod.rs
+22 -9
View File
@@ -51,8 +51,9 @@ use polkadot_node_subsystem_types::{
Span,
};
use polkadot_overseer::{metrics::Metrics as OverseerMetrics, Handle as OverseerHandle};
use polkadot_primitives::GroupIndex;
use polkadot_primitives::{Block, GroupIndex, Hash};
use sc_network::request_responses::{IncomingRequest as RawIncomingRequest, ProtocolConfig};
use sc_service::SpawnTaskHandle;
use serde::{Deserialize, Serialize};
use std::{ops::Sub, sync::Arc, time::Instant};
@@ -140,20 +141,32 @@ pub fn prepare_test(
mode: TestDataAvailability,
with_prometheus_endpoint: bool,
) -> (TestEnvironment, Vec<ProtocolConfig>) {
let (collation_req_receiver, collation_req_cfg) =
IncomingRequest::get_config_receiver(&ReqProtocolNames::new(GENESIS_HASH, None));
let (pov_req_receiver, pov_req_cfg) =
IncomingRequest::get_config_receiver(&ReqProtocolNames::new(GENESIS_HASH, None));
let (chunk_req_receiver, chunk_req_cfg) =
IncomingRequest::get_config_receiver(&ReqProtocolNames::new(GENESIS_HASH, None));
let req_cfgs = vec![collation_req_cfg, pov_req_cfg];
let dependencies = TestEnvironmentDependencies::default();
let availability_state = NetworkAvailabilityState {
candidate_hashes: state.candidate_hashes.clone(),
available_data: state.available_data.clone(),
chunks: state.chunks.clone(),
};
let mut req_cfgs = Vec::new();
let (collation_req_receiver, collation_req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&ReqProtocolNames::new(GENESIS_HASH, None));
req_cfgs.push(collation_req_cfg);
let (pov_req_receiver, pov_req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&ReqProtocolNames::new(GENESIS_HASH, None));
let (chunk_req_receiver, chunk_req_cfg) = IncomingRequest::get_config_receiver::<
Block,
sc_network::NetworkWorker<Block, Hash>,
>(&ReqProtocolNames::new(GENESIS_HASH, None));
req_cfgs.push(pov_req_cfg);
let (network, network_interface, network_receiver) = new_network(
&state.config,
&dependencies,
polkadot/node/subsystem-bench/src/lib/configuration.rs
+1 -1
View File
@@ -21,7 +21,7 @@ use itertools::Itertools;
use polkadot_primitives::{AssignmentId, AuthorityDiscoveryId, ValidatorId};
use rand::thread_rng;
use rand_distr::{Distribution, Normal, Uniform};
use sc_network::PeerId;
use sc_network_types::PeerId;
use serde::{Deserialize, Serialize};
use sp_consensus_babe::AuthorityId;
use std::collections::HashMap;
polkadot/node/subsystem-bench/src/lib/network.rs
+2 -1
View File
@@ -67,8 +67,9 @@ use prometheus_endpoint::U64;
use rand::{seq::SliceRandom, thread_rng};
use sc_network::{
request_responses::{IncomingRequest, OutgoingResponse},
PeerId, RequestFailure,
RequestFailure,
};
use sc_network_types::PeerId;
use sc_service::SpawnTaskHandle;
use std::{
collections::HashMap,
polkadot/node/subsystem-types/Cargo.toml
+1
View File
@@ -19,6 +19,7 @@ polkadot-statement-table = { path = "../../statement-table" }
polkadot-node-jaeger = { path = "../jaeger" }
orchestra = { version = "0.3.5", default-features = false, features = ["futures_channel"] }
sc-network = { path = "../../../substrate/client/network" }
sc-network-types = { path = "../../../substrate/client/network/types" }
sp-api = { path = "../../../substrate/primitives/api" }
sp-blockchain = { path = "../../../substrate/primitives/blockchain" }
sp-consensus-babe = { path = "../../../substrate/primitives/consensus/babe" }
polkadot/node/subsystem-types/src/messages/network_bridge_event.rs
+2 -1
View File
@@ -16,7 +16,8 @@
use std::{collections::HashSet, convert::TryFrom};
pub use sc_network::{PeerId, ReputationChange};
pub use sc_network::ReputationChange;
pub use sc_network_types::PeerId;
use polkadot_node_network_protocol::{
grid_topology::SessionGridTopology, peer_set::ProtocolVersion, ObservedRole, OurView, View,
polkadot/node/test/service/src/lib.rs
+40 -18
View File
@@ -79,24 +79,46 @@ pub fn new_full<OverseerGenerator: OverseerGen>(
) -> Result<NewFull, Error> {
let workers_path = Some(workers_path.unwrap_or_else(get_relative_workers_path_for_test));
polkadot_service::new_full(
config,
polkadot_service::NewFullParams {
is_parachain_node,
enable_beefy: true,
force_authoring_backoff: false,
jaeger_agent: None,
telemetry_worker_handle: None,
node_version: None,
secure_validator_mode: false,
workers_path,
workers_names: None,
overseer_gen,
overseer_message_channel_capacity_override: None,
malus_finality_delay: None,
hwbench: None,
},
)
match config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p =>
polkadot_service::new_full::<_, sc_network::NetworkWorker<_, _>>(
config,
polkadot_service::NewFullParams {
is_parachain_node,
enable_beefy: true,
force_authoring_backoff: false,
jaeger_agent: None,
telemetry_worker_handle: None,
node_version: None,
secure_validator_mode: false,
workers_path,
workers_names: None,
overseer_gen,
overseer_message_channel_capacity_override: None,
malus_finality_delay: None,
hwbench: None,
},
),
sc_network::config::NetworkBackendType::Litep2p =>
polkadot_service::new_full::<_, sc_network::Litep2pNetworkBackend>(
config,
polkadot_service::NewFullParams {
is_parachain_node,
enable_beefy: true,
force_authoring_backoff: false,
jaeger_agent: None,
telemetry_worker_handle: None,
node_version: None,
secure_validator_mode: false,
workers_path,
workers_names: None,
overseer_gen,
overseer_message_channel_capacity_override: None,
malus_finality_delay: None,
hwbench: None,
},
),
}
}
fn get_relative_workers_path_for_test() -> PathBuf {
prdoc/pr_2944.prdoc
+28
View File
@@ -0,0 +1,28 @@
# Schema: Polkadot SDK PRDoc Schema (prdoc) v1.0.0
# See doc at https://raw.githubusercontent.com/paritytech/polkadot-sdk/master/prdoc/schema_user.json
title: "Integrate litep2p into Polkadot SDK"
doc:
- audience: Node Dev
description: |
litep2p is a libp2p-compatible P2P networking library. It supports all of the features of rust-libp2p
that are currently being utilized by Polkadot SDK and is a drop-in replacement for any node operator.
For node developers, introduction of litep2p implies specifying the networking backend that Polkadot SDK
shall use for P2P networking. A new trait called `NetworkBackend` is introduced which is implemented
by both the libp2p and litep2p backends and which is used to initialize any networking-related code.
- audience: Node Operator
description: |
litep2p is considered experimental and rust-libp2p will remain as the default networking backend
for Polkadot SDK but litep2p can be selected with `--network-backend litep2p`.
crates:
- name: "sc-network"
- name: "sc-service"
- name: "minimal-template-node"
- name: "solochain-template-node"
- name: "staging-node-cli"
- name: "polkadot-service"
- name: "parachain-template-node"
substrate/bin/node/cli/benches/block_production.rs
+7 -2
View File
@@ -104,8 +104,13 @@ fn new_node(tokio_handle: Handle) -> node_cli::service::NewFullBase {
wasm_runtime_overrides: None,
};
node_cli::service::new_full_base(config, None, false, |_, _| ())
.expect("creating a full node doesn't fail")
node_cli::service::new_full_base::<sc_network::NetworkWorker<_, _>>(
config,
None,
false,
|_, _| (),
)
.expect("creating a full node doesn't fail")
}
fn extrinsic_set_time(now: u64) -> OpaqueExtrinsic {
substrate/bin/node/cli/benches/transaction_pool.rs
+7 -1
View File
@@ -101,7 +101,13 @@ fn new_node(tokio_handle: Handle) -> node_cli::service::NewFullBase {
};
tokio_handle.block_on(async move {
node_cli::service::new_full_base(config, None, false, |_, _| ()).expect("Creates node")
node_cli::service::new_full_base::<sc_network::NetworkWorker<_, _>>(
config,
None,
false,
|_, _| (),
)
.expect("Creates node")
})
}
substrate/bin/node/cli/src/chain_spec.rs
+1 -1
View File
@@ -513,7 +513,7 @@ pub(crate) mod tests {
sc_service_test::connectivity(integration_test_config_with_two_authorities(), |config| {
let NewFullBase { task_manager, client, network, sync, transaction_pool, .. } =
new_full_base(config, None, false, |_, _| ())?;
new_full_base::<sc_network::NetworkWorker<_, _>>(config, None, false, |_, _| ())?;
Ok(sc_service_test::TestNetComponents::new(
task_manager,
client,
substrate/bin/node/cli/src/service.rs
+67 -17
View File
@@ -29,7 +29,9 @@ use kitchensink_runtime::RuntimeApi;
use node_primitives::Block;
use sc_client_api::{Backend, BlockBackend};
use sc_consensus_babe::{self, SlotProportion};
use sc_network::{event::Event, NetworkEventStream, NetworkService};
use sc_network::{
event::Event, service::traits::NetworkService, NetworkBackend, NetworkEventStream,
};
use sc_network_sync::{strategy::warp::WarpSyncParams, SyncingService};
use sc_service::{config::Configuration, error::Error as ServiceError, RpcHandlers, TaskManager};
use sc_statement_store::Store as StatementStore;
@@ -368,7 +370,7 @@ pub struct NewFullBase {
/// The client instance of the node.
pub client: Arc<FullClient>,
/// The networking service of the node.
pub network: Arc<NetworkService<Block, <Block as BlockT>::Hash>>,
pub network: Arc<dyn NetworkService>,
/// The syncing service of the node.
pub sync: Arc<SyncingService<Block>>,
/// The transaction pool of the node.
@@ -378,7 +380,7 @@ pub struct NewFullBase {
}
/// Creates a full service from the configuration.
pub fn new_full_base(
pub fn new_full_base<N: NetworkBackend<Block, <Block as BlockT>::Hash>>(
config: Configuration,
mixnet_config: Option<sc_mixnet::Config>,
disable_hardware_benchmarks: bool,
@@ -420,15 +422,26 @@ pub fn new_full_base(
(rpc_builder, import_setup, rpc_setup, mut telemetry, statement_store, mixnet_api_backend),
} = new_partial(&config, mixnet_config.as_ref())?;
let metrics = N::register_notification_metrics(
config.prometheus_config.as_ref().map(|cfg| &cfg.registry),
);
let shared_voter_state = rpc_setup;
let auth_disc_publish_non_global_ips = config.network.allow_non_globals_in_dht;
let auth_disc_public_addresses = config.network.public_addresses.clone();
let mut net_config = sc_network::config::FullNetworkConfiguration::new(&config.network);
let mut net_config =
sc_network::config::FullNetworkConfiguration::<_, _, N>::new(&config.network);
let genesis_hash = client.block_hash(0).ok().flatten().expect("Genesis block exists; qed");
let peer_store_handle = net_config.peer_store_handle();
let grandpa_protocol_name = grandpa::protocol_standard_name(&genesis_hash, &config.chain_spec);
let (grandpa_protocol_config, grandpa_notification_service) =
grandpa::grandpa_peers_set_config(grandpa_protocol_name.clone());
grandpa::grandpa_peers_set_config::<_, N>(
grandpa_protocol_name.clone(),
metrics.clone(),
Arc::clone(&peer_store_handle),
);
net_config.add_notification_protocol(grandpa_protocol_config);
let beefy_gossip_proto_name =
@@ -436,7 +449,7 @@ pub fn new_full_base(
// `beefy_on_demand_justifications_handler` is given to `beefy-gadget` task to be run,
// while `beefy_req_resp_cfg` is added to `config.network.request_response_protocols`.
let (beefy_on_demand_justifications_handler, beefy_req_resp_cfg) =
beefy::communication::request_response::BeefyJustifsRequestHandler::new(
beefy::communication::request_response::BeefyJustifsRequestHandler::new::<_, N>(
&genesis_hash,
config.chain_spec.fork_id(),
client.clone(),
@@ -444,23 +457,33 @@ pub fn new_full_base(
);
let (beefy_notification_config, beefy_notification_service) =
beefy::communication::beefy_peers_set_config(beefy_gossip_proto_name.clone());
beefy::communication::beefy_peers_set_config::<_, N>(
beefy_gossip_proto_name.clone(),
metrics.clone(),
Arc::clone(&peer_store_handle),
);
net_config.add_notification_protocol(beefy_notification_config);
net_config.add_request_response_protocol(beefy_req_resp_cfg);
let (statement_handler_proto, statement_config) =
sc_network_statement::StatementHandlerPrototype::new(
sc_network_statement::StatementHandlerPrototype::new::<_, _, N>(
genesis_hash,
config.chain_spec.fork_id(),
metrics.clone(),
Arc::clone(&peer_store_handle),
);
net_config.add_notification_protocol(statement_config);
let mixnet_protocol_name =
sc_mixnet::protocol_name(genesis_hash.as_ref(), config.chain_spec.fork_id());
let mixnet_notification_service = mixnet_config.as_ref().map(|mixnet_config| {
let (config, notification_service) =
sc_mixnet::peers_set_config(mixnet_protocol_name.clone(), mixnet_config);
let (config, notification_service) = sc_mixnet::peers_set_config::<_, N>(
mixnet_protocol_name.clone(),
mixnet_config,
metrics.clone(),
Arc::clone(&peer_store_handle),
);
net_config.add_notification_protocol(config);
notification_service
});
@@ -482,6 +505,7 @@ pub fn new_full_base(
block_announce_validator_builder: None,
warp_sync_params: Some(WarpSyncParams::WithProvider(warp_sync)),
block_relay: None,
metrics,
})?;
if let Some(mixnet_config) = mixnet_config {
@@ -615,7 +639,7 @@ pub fn new_full_base(
..Default::default()
},
client.clone(),
network.clone(),
Arc::new(network.clone()),
Box::pin(dht_event_stream),
authority_discovery_role,
prometheus_registry.clone(),
@@ -634,7 +658,7 @@ pub fn new_full_base(
// beefy is enabled if its notification service exists
let network_params = beefy::BeefyNetworkParams {
network: network.clone(),
network: Arc::new(network.clone()),
sync: sync_service.clone(),
gossip_protocol_name: beefy_gossip_proto_name,
justifications_protocol_name: beefy_on_demand_justifications_handler.protocol_name(),
@@ -746,7 +770,7 @@ pub fn new_full_base(
transaction_pool: Some(OffchainTransactionPoolFactory::new(
transaction_pool.clone(),
)),
network_provider: network.clone(),
network_provider: Arc::new(network.clone()),
is_validator: role.is_authority(),
enable_http_requests: true,
custom_extensions: move |_| {
@@ -773,8 +797,29 @@ pub fn new_full_base(
pub fn new_full(config: Configuration, cli: Cli) -> Result<TaskManager, ServiceError> {
let mixnet_config = cli.mixnet_params.config(config.role.is_authority());
let database_path = config.database.path().map(Path::to_path_buf);
let task_manager = new_full_base(config, mixnet_config, cli.no_hardware_benchmarks, |_, _| ())
.map(|NewFullBase { task_manager, .. }| task_manager)?;
let task_manager = match config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p => {
let task_manager = new_full_base::<sc_network::NetworkWorker<_, _>>(
config,
mixnet_config,
cli.no_hardware_benchmarks,
|_, _| (),
)
.map(|NewFullBase { task_manager, .. }| task_manager)?;
task_manager
},
sc_network::config::NetworkBackendType::Litep2p => {
let task_manager = new_full_base::<sc_network::Litep2pNetworkBackend>(
config,
mixnet_config,
cli.no_hardware_benchmarks,
|_, _| (),
)
.map(|NewFullBase { task_manager, .. }| task_manager)?;
task_manager
},
};
if let Some(database_path) = database_path {
sc_storage_monitor::StorageMonitorService::try_spawn(
@@ -851,7 +896,7 @@ mod tests {
|config| {
let mut setup_handles = None;
let NewFullBase { task_manager, client, network, sync, transaction_pool, .. } =
new_full_base(
new_full_base::<sc_network::NetworkWorker<_, _>>(
config,
None,
false,
@@ -1029,7 +1074,12 @@ mod tests {
crate::chain_spec::tests::integration_test_config_with_two_authorities(),
|config| {
let NewFullBase { task_manager, client, network, sync, transaction_pool, .. } =
new_full_base(config, None, false, |_, _| ())?;
new_full_base::<sc_network::NetworkWorker<_, _>>(
config,
None,
false,
|_, _| (),
)?;
Ok(sc_service_test::TestNetComponents::new(
task_manager,
client,
substrate/client/authority-discovery/Cargo.toml
+2 -4
View File
@@ -25,10 +25,7 @@ futures = "0.3.30"
futures-timer = "3.0.1"
ip_network = "0.4.1"
libp2p = { version = "0.51.4", features = ["ed25519", "kad"] }
multihash = { version = "0.18.1", default-features = false, features = [
"sha2",
"std",
] }
multihash = { version = "0.17.0", default-features = false, features = ["sha2", "std"] }
linked_hash_set = "0.1.4"
log = { workspace = true, default-features = true }
prost = "0.12"
@@ -37,6 +34,7 @@ thiserror = { workspace = true }
prometheus-endpoint = { package = "substrate-prometheus-endpoint", path = "../../utils/prometheus" }
sc-client-api = { path = "../api" }
sc-network = { path = "../network" }
sc-network-types = { path = "../network/types" }
sp-api = { path = "../../primitives/api" }
sp-authority-discovery = { path = "../../primitives/authority-discovery" }
sp-blockchain = { path = "../../primitives/blockchain" }
substrate/client/authority-discovery/src/error.rs
+4 -4
View File
@@ -35,7 +35,7 @@ pub enum Error {
VerifyingDhtPayload,
#[error("Failed to hash the authority id to be used as a dht key.")]
HashingAuthorityId(#[from] libp2p::core::multiaddr::multihash::Error),
HashingAuthorityId(#[from] sc_network::multiaddr::multihash::Error),
#[error("Failed calling into the Substrate runtime: {0}")]
CallingRuntime(#[from] sp_blockchain::Error),
@@ -53,10 +53,10 @@ pub enum Error {
EncodingDecodingScale(#[from] codec::Error),
#[error("Failed to parse a libp2p multi address.")]
ParsingMultiaddress(#[from] libp2p::core::multiaddr::Error),
ParsingMultiaddress(#[from] sc_network::multiaddr::Error),
#[error("Failed to parse a libp2p key.")]
ParsingLibp2pIdentity(#[from] libp2p::identity::DecodingError),
#[error("Failed to parse a libp2p key: {0}")]
ParsingLibp2pIdentity(String),
#[error("Failed to sign: {0}.")]
CannotSign(String),
substrate/client/authority-discovery/src/lib.rs
+8 -10
View File
@@ -40,8 +40,8 @@ use futures::{
Stream,
};
use libp2p::{Multiaddr, PeerId};
use sc_network::event::DhtEvent;
use sc_network::{event::DhtEvent, Multiaddr};
use sc_network_types::PeerId;
use sp_authority_discovery::AuthorityId;
use sp_blockchain::HeaderBackend;
use sp_runtime::traits::Block as BlockT;
@@ -117,16 +117,15 @@ impl Default for WorkerConfig {
/// Create a new authority discovery [`Worker`] and [`Service`].
///
/// See the struct documentation of each for more details.
pub fn new_worker_and_service<Client, Network, Block, DhtEventStream>(
pub fn new_worker_and_service<Client, Block, DhtEventStream>(
client: Arc<Client>,
network: Arc<Network>,
network: Arc<dyn NetworkProvider>,
dht_event_rx: DhtEventStream,
role: Role,
prometheus_registry: Option<prometheus_endpoint::Registry>,
) -> (Worker<Client, Network, Block, DhtEventStream>, Service)
) -> (Worker<Client, Block, DhtEventStream>, Service)
where
Block: BlockT + Unpin + 'static,
Network: NetworkProvider,
Client: AuthorityDiscovery<Block> + Send + Sync + 'static + HeaderBackend<Block>,
DhtEventStream: Stream<Item = DhtEvent> + Unpin,
{
@@ -143,17 +142,16 @@ where
/// Same as [`new_worker_and_service`] but with support for providing the `config`.
///
/// When in doubt use [`new_worker_and_service`] as it will use the default configuration.
pub fn new_worker_and_service_with_config<Client, Network, Block, DhtEventStream>(
pub fn new_worker_and_service_with_config<Client, Block, DhtEventStream>(
config: WorkerConfig,
client: Arc<Client>,
network: Arc<Network>,
network: Arc<dyn NetworkProvider>,
dht_event_rx: DhtEventStream,
role: Role,
prometheus_registry: Option<prometheus_endpoint::Registry>,
) -> (Worker<Client, Network, Block, DhtEventStream>, Service)
) -> (Worker<Client, Block, DhtEventStream>, Service)
where
Block: BlockT + Unpin + 'static,
Network: NetworkProvider,
Client: AuthorityDiscovery<Block> + 'static,
DhtEventStream: Stream<Item = DhtEvent> + Unpin,
{
substrate/client/authority-discovery/src/service.rs
+2 -1
View File
@@ -25,7 +25,8 @@ use futures::{
SinkExt,
};
use libp2p::{Multiaddr, PeerId};
use sc_network::Multiaddr;
use sc_network_types::PeerId;
use sp_authority_discovery::AuthorityId;
/// Service to interact with the [`crate::Worker`].
substrate/client/authority-discovery/src/tests.rs
+3 -6
View File
@@ -25,13 +25,10 @@ use crate::{
};
use futures::{channel::mpsc::channel, executor::LocalPool, task::LocalSpawn};
use libp2p::{
core::multiaddr::{Multiaddr, Protocol},
identity::ed25519,
PeerId,
};
use libp2p::identity::ed25519;
use std::{collections::HashSet, sync::Arc};
use sc_network::{multiaddr::Protocol, Multiaddr, PeerId};
use sp_authority_discovery::AuthorityId;
use sp_core::crypto::key_types;
use sp_keystore::{testing::MemoryKeystore, Keystore};
@@ -78,7 +75,7 @@ fn get_addresses_and_authority_id() {
);
assert_eq!(
Some(HashSet::from([remote_authority_id])),
service.get_authority_ids_by_peer_id(remote_peer_id).await,
service.get_authority_ids_by_peer_id(remote_peer_id.into()).await,
);
});
}
substrate/client/authority-discovery/src/worker.rs
+36 -23
View File
@@ -34,9 +34,8 @@ use futures::{channel::mpsc, future, stream::Fuse, FutureExt, Stream, StreamExt}
use addr_cache::AddrCache;
use codec::{Decode, Encode};
use ip_network::IpNetwork;
use libp2p::{core::multiaddr, identity::PublicKey, multihash::Multihash, Multiaddr, PeerId};
use linked_hash_set::LinkedHashSet;
use multihash_codetable::{Code, MultihashDigest};
use multihash::{Code, Multihash, MultihashDigest};
use log::{debug, error, log_enabled};
use prometheus_endpoint::{register, Counter, CounterVec, Gauge, Opts, U64};
@@ -44,8 +43,10 @@ use prost::Message;
use rand::{seq::SliceRandom, thread_rng};
use sc_network::{
event::DhtEvent, KademliaKey, NetworkDHTProvider, NetworkSigner, NetworkStateInfo, Signature,
event::DhtEvent, multiaddr, KademliaKey, Multiaddr, NetworkDHTProvider, NetworkSigner,
NetworkStateInfo,
};
use sc_network_types::PeerId;
use sp_api::{ApiError, ProvideRuntimeApi};
use sp_authority_discovery::{
AuthorityDiscoveryApi, AuthorityId, AuthorityPair, AuthoritySignature,
@@ -108,13 +109,13 @@ pub enum Role {
/// network peerset.
///
/// 5. Allow querying of the collected addresses via the [`crate::Service`].
pub struct Worker<Client, Network, Block, DhtEventStream> {
pub struct Worker<Client, Block, DhtEventStream> {
/// Channel receiver for messages send by a [`crate::Service`].
from_service: Fuse<mpsc::Receiver<ServicetoWorkerMsg>>,
client: Arc<Client>,
network: Arc<Network>,
network: Arc<dyn NetworkProvider>,
/// Channel we receive Dht events on.
dht_event_rx: DhtEventStream,
@@ -192,10 +193,9 @@ where
}
}
impl<Client, Network, Block, DhtEventStream> Worker<Client, Network, Block, DhtEventStream>
impl<Client, Block, DhtEventStream> Worker<Client, Block, DhtEventStream>
where
Block: BlockT + Unpin + 'static,
Network: NetworkProvider,
Client: AuthorityDiscovery<Block> + 'static,
DhtEventStream: Stream<Item = DhtEvent> + Unpin,
{
@@ -203,7 +203,7 @@ where
pub(crate) fn new(
from_service: mpsc::Receiver<ServicetoWorkerMsg>,
client: Arc<Client>,
network: Arc<Network>,
network: Arc<dyn NetworkProvider>,
dht_event_rx: DhtEventStream,
role: Role,
prometheus_registry: Option<prometheus_endpoint::Registry>,
@@ -406,10 +406,14 @@ where
Role::Discover => return Ok(()),
};
let keys = Worker::<Client, Network, Block, DhtEventStream>::get_own_public_keys_within_authority_set(
key_store.clone(),
self.client.as_ref(),
).await?.into_iter().collect::<HashSet<_>>();
let keys =
Worker::<Client, Block, DhtEventStream>::get_own_public_keys_within_authority_set(
key_store.clone(),
self.client.as_ref(),
)
.await?
.into_iter()
.collect::<HashSet<_>>();
if only_if_changed {
// If the authority keys did not change and the `publish_if_changed_interval` was
@@ -434,7 +438,7 @@ where
}
let serialized_record = serialize_authority_record(addresses)?;
let peer_signature = sign_record_with_peer_id(&serialized_record, self.network.as_ref())?;
let peer_signature = sign_record_with_peer_id(&serialized_record, &self.network)?;
let keys_vec = keys.iter().cloned().collect::<Vec<_>>();
@@ -634,12 +638,15 @@ where
// properly signed by the owner of the PeerId
if let Some(peer_signature) = peer_signature {
let public_key = PublicKey::try_decode_protobuf(&peer_signature.public_key)
.map_err(Error::ParsingLibp2pIdentity)?;
let signature = Signature { public_key, bytes: peer_signature.signature };
if !signature.verify(record, &remote_peer_id) {
return Err(Error::VerifyingDhtPayload)
match self.network.verify(
remote_peer_id.into(),
&peer_signature.public_key,
&peer_signature.signature,
&record,
) {
Ok(true) => {},
Ok(false) => return Err(Error::VerifyingDhtPayload),
Err(error) => return Err(Error::ParsingLibp2pIdentity(error)),
}
} else if self.strict_record_validation {
return Err(Error::MissingPeerIdSignature)
@@ -701,9 +708,15 @@ where
/// NetworkProvider provides [`Worker`] with all necessary hooks into the
/// underlying Substrate networking. Using this trait abstraction instead of
/// `sc_network::NetworkService` directly is necessary to unit test [`Worker`].
pub trait NetworkProvider: NetworkDHTProvider + NetworkStateInfo + NetworkSigner {}
pub trait NetworkProvider:
NetworkDHTProvider + NetworkStateInfo + NetworkSigner + Send + Sync
{
}
impl<T> NetworkProvider for T where T: NetworkDHTProvider + NetworkStateInfo + NetworkSigner {}
impl<T> NetworkProvider for T where
T: NetworkDHTProvider + NetworkStateInfo + NetworkSigner + Send + Sync
{
}
fn hash_authority_id(id: &[u8]) -> KademliaKey {
KademliaKey::new(&Code::Sha2_256.digest(id).digest())
@@ -741,7 +754,7 @@ fn sign_record_with_peer_id(
network: &impl NetworkSigner,
) -> Result<schema::PeerSignature> {
let signature = network
.sign_with_local_identity(serialized_record)
.sign_with_local_identity(serialized_record.to_vec())
.map_err(|e| Error::CannotSign(format!("{} (network packet)", e)))?;
let public_key = signature.public_key.encode_protobuf();
let signature = signature.bytes;
@@ -855,7 +868,7 @@ impl Metrics {
// Helper functions for unit testing.
#[cfg(test)]
impl<Block, Client, Network, DhtEventStream> Worker<Client, Network, Block, DhtEventStream> {
impl<Block, Client, DhtEventStream> Worker<Client, Block, DhtEventStream> {
pub(crate) fn inject_addresses(&mut self, authority: AuthorityId, addresses: Vec<Multiaddr>) {
self.addr_cache.insert(authority, addresses);
}
substrate/client/authority-discovery/src/worker/addr_cache.rs
+3 -5
View File
@@ -16,10 +16,8 @@
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use libp2p::{
core::multiaddr::{Multiaddr, Protocol},
PeerId,
};
use sc_network::{multiaddr::Protocol, Multiaddr};
use sc_network_types::PeerId;
use sp_authority_discovery::AuthorityId;
use std::collections::{hash_map::Entry, HashMap, HashSet};
@@ -178,7 +176,7 @@ fn addresses_to_peer_ids(addresses: &HashSet<Multiaddr>) -> HashSet<PeerId> {
mod tests {
use super::*;
use libp2p::multihash::{self, Multihash};
use multihash::{self, Multihash};
use quickcheck::{Arbitrary, Gen, QuickCheck, TestResult};
use sp_authority_discovery::{AuthorityId, AuthorityPair};
substrate/client/authority-discovery/src/worker/schema/tests.rs
+2 -1
View File
@@ -21,8 +21,9 @@ mod schema_v1 {
}
use super::*;
use libp2p::{identity::Keypair, multiaddr::Multiaddr, PeerId};
use libp2p::identity::Keypair;
use prost::Message;
use sc_network::{Multiaddr, PeerId};
#[test]
fn v2_decodes_v1() {
substrate/client/authority-discovery/src/worker/tests.rs
+32 -13
View File
@@ -29,16 +29,11 @@ use futures::{
sink::SinkExt,
task::LocalSpawn,
};
use libp2p::{
core::multiaddr,
identity::{Keypair, SigningError},
kad::record::Key as KademliaKey,
PeerId,
};
use libp2p::{core::multiaddr, identity::SigningError, kad::record::Key as KademliaKey, PeerId};
use prometheus_endpoint::prometheus::default_registry;
use sc_client_api::HeaderBackend;
use sc_network::Signature;
use sc_network::{service::signature::Keypair, Signature};
use sp_api::{ApiRef, ProvideRuntimeApi};
use sp_keystore::{testing::MemoryKeystore, Keystore};
use sp_runtime::traits::{Block as BlockT, NumberFor, Zero};
@@ -122,7 +117,7 @@ pub enum TestNetworkEvent {
}
pub struct TestNetwork {
peer_id: PeerId,
peer_id: sc_network_types::PeerId,
identity: Keypair,
external_addresses: Vec<Multiaddr>,
// Whenever functions on `TestNetwork` are called, the function arguments are added to the
@@ -158,10 +153,25 @@ impl Default for TestNetwork {
impl NetworkSigner for TestNetwork {
fn sign_with_local_identity(
&self,
msg: impl AsRef<[u8]>,
msg: Vec<u8>,
) -> std::result::Result<Signature, SigningError> {
Signature::sign_message(msg, &self.identity)
}
fn verify(
&self,
peer_id: sc_network_types::PeerId,
public_key: &Vec<u8>,
signature: &Vec<u8>,
message: &Vec<u8>,
) -> std::result::Result<bool, String> {
let public_key = libp2p::identity::PublicKey::try_decode_protobuf(&public_key)
.map_err(|error| error.to_string())?;
let peer_id: PeerId = peer_id.into();
let remote: libp2p::PeerId = public_key.to_peer_id();
Ok(peer_id == remote && public_key.verify(message, signature))
}
}
impl NetworkDHTProvider for TestNetwork {
@@ -182,8 +192,8 @@ impl NetworkDHTProvider for TestNetwork {
}
impl NetworkStateInfo for TestNetwork {
fn local_peer_id(&self) -> PeerId {
self.peer_id
fn local_peer_id(&self) -> sc_network_types::PeerId {
self.peer_id.into()
}
fn external_addresses(&self) -> Vec<Multiaddr> {
@@ -202,10 +212,20 @@ struct TestSigner<'a> {
impl<'a> NetworkSigner for TestSigner<'a> {
fn sign_with_local_identity(
&self,
msg: impl AsRef<[u8]>,
msg: Vec<u8>,
) -> std::result::Result<Signature, SigningError> {
Signature::sign_message(msg, self.keypair)
}
fn verify(
&self,
_: sc_network_types::PeerId,
_: &Vec<u8>,
_: &Vec<u8>,
_: &Vec<u8>,
) -> std::result::Result<bool, String> {
unimplemented!();
}
}
fn build_dht_event<Signer: NetworkSigner>(
@@ -500,7 +520,6 @@ struct DhtValueFoundTester {
pub local_worker: Option<
Worker<
TestApi,
TestNetwork,
sp_runtime::generic::Block<
sp_runtime::generic::Header<u64, sp_runtime::traits::BlakeTwo256>,
substrate_test_runtime_client::runtime::Extrinsic,
substrate/client/cli/src/arg_enums.rs
+20
View File
@@ -296,3 +296,23 @@ impl Into<sc_network::config::SyncMode> for SyncMode {
}
}
}
/// Network backend type.
#[derive(Debug, Clone, Copy, ValueEnum, PartialEq)]
#[value(rename_all = "lower")]
pub enum NetworkBackendType {
/// Use libp2p for P2P networking.
Libp2p,
/// Use litep2p for P2P networking.
Litep2p,
}
impl Into<sc_network::config::NetworkBackendType> for NetworkBackendType {
fn into(self) -> sc_network::config::NetworkBackendType {
match self {
Self::Libp2p => sc_network::config::NetworkBackendType::Libp2p,
Self::Litep2p => sc_network::config::NetworkBackendType::Litep2p,
}
}
}
substrate/client/cli/src/commands/build_spec_cmd.rs
+1 -1
View File
@@ -67,7 +67,7 @@ impl BuildSpecCmd {
let peer_id = keys.public().to_peer_id();
let addr = MultiaddrWithPeerId {
multiaddr: build_multiaddr![Ip4([127, 0, 0, 1]), Tcp(30333u16)],
peer_id,
peer_id: peer_id.into(),
};
spec.add_boot_node(addr)
}
substrate/client/cli/src/params/network_params.rs
+19 -1
View File
@@ -16,7 +16,10 @@
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use crate::{arg_enums::SyncMode, params::node_key_params::NodeKeyParams};
use crate::{
arg_enums::{NetworkBackendType, SyncMode},
params::node_key_params::NodeKeyParams,
};
use clap::Args;
use sc_network::{
config::{
@@ -166,6 +169,20 @@ pub struct NetworkParams {
/// and observe block requests timing out.
#[arg(long, value_name = "COUNT", default_value_t = 64)]
pub max_blocks_per_request: u32,
/// Network backend used for P2P networking.
///
/// litep2p network backend is considered experimental and isn't as stable as the libp2p
/// network backend.
#[arg(
long,
value_enum,
value_name = "NETWORK_BACKEND",
default_value_t = NetworkBackendType::Libp2p,
ignore_case = true,
verbatim_doc_comment
)]
pub network_backend: NetworkBackendType,
}
impl NetworkParams {
@@ -261,6 +278,7 @@ impl NetworkParams {
yamux_window_size: None,
ipfs_server: self.ipfs_server,
sync_mode: self.sync.into(),
network_backend: self.network_backend.into(),
}
}
}
substrate/client/consensus/beefy/Cargo.toml
+1
View File
@@ -28,6 +28,7 @@ sc-consensus = { path = "../common" }
sc-network = { path = "../../network" }
sc-network-gossip = { path = "../../network-gossip" }
sc-network-sync = { path = "../../network/sync" }
sc-network-types = { path = "../../network/types" }
sc-utils = { path = "../../utils" }
sp-api = { path = "../../../primitives/api" }
sp-application-crypto = { path = "../../../primitives/application-crypto" }
substrate/client/consensus/beefy/src/communication/gossip.rs
+11 -5
View File
@@ -18,8 +18,9 @@
use std::{collections::BTreeSet, sync::Arc, time::Duration};
use sc_network::{NetworkPeers, PeerId, ReputationChange};
use sc_network::{NetworkPeers, ReputationChange};
use sc_network_gossip::{MessageIntent, ValidationResult, Validator, ValidatorContext};
use sc_network_types::PeerId;
use sp_runtime::traits::{Block, Hash, Header, NumberFor};
use codec::{Decode, DecodeAll, Encode};
@@ -506,6 +507,7 @@ pub(crate) mod tests {
}
}
#[async_trait::async_trait]
impl NetworkPeers for TestNetwork {
fn set_authorized_peers(&self, _: std::collections::HashSet<PeerId>) {
unimplemented!()
@@ -581,6 +583,10 @@ pub(crate) mod tests {
fn peer_role(&self, _: PeerId, _: Vec<u8>) -> Option<sc_network::ObservedRole> {
unimplemented!()
}
async fn reserved_peers(&self) -> Result<Vec<PeerId>, ()> {
unimplemented!();
}
}
struct TestContext;
@@ -591,11 +597,11 @@ pub(crate) mod tests {
fn broadcast_message(&mut self, _topic: B::Hash, _message: Vec<u8>, _force: bool) {}
fn send_message(&mut self, _who: &sc_network::PeerId, _message: Vec<u8>) {
fn send_message(&mut self, _who: &sc_network_types::PeerId, _message: Vec<u8>) {
unimplemented!()
}
fn send_topic(&mut self, _who: &sc_network::PeerId, _topic: B::Hash, _force: bool) {
fn send_topic(&mut self, _who: &sc_network_types::PeerId, _topic: B::Hash, _force: bool) {
unimplemented!()
}
}
@@ -772,7 +778,7 @@ pub(crate) mod tests {
Arc::new(TestNetwork::new().0),
);
gv.update_filter(GossipFilterCfg { start: 0, end: 10, validator_set: &validator_set });
let sender = sc_network::PeerId::random();
let sender = sc_network_types::PeerId::random();
let topic = Default::default();
let intent = MessageIntent::Broadcast;
@@ -852,7 +858,7 @@ pub(crate) mod tests {
Arc::new(TestNetwork::new().0),
);
gv.update_filter(GossipFilterCfg { start: 0, end: 10, validator_set: &validator_set });
let sender = sc_network::PeerId::random();
let sender = sc_network_types::PeerId::random();
let topic = Default::default();
let vote = dummy_vote(1);
substrate/client/consensus/beefy/src/communication/mod.rs
+16 -5
View File
@@ -65,17 +65,28 @@ pub(crate) mod beefy_protocol_name {
/// Returns the configuration value to put in
/// [`sc_network::config::FullNetworkConfiguration`].
/// For standard protocol name see [`beefy_protocol_name::gossip_protocol_name`].
pub fn beefy_peers_set_config(
pub fn beefy_peers_set_config<
B: sp_runtime::traits::Block,
N: sc_network::NetworkBackend<B, <B as sp_runtime::traits::Block>::Hash>,
>(
gossip_protocol_name: sc_network::ProtocolName,
) -> (sc_network::config::NonDefaultSetConfig, Box<dyn sc_network::NotificationService>) {
let (mut cfg, notification_service) = sc_network::config::NonDefaultSetConfig::new(
metrics: sc_network::service::NotificationMetrics,
peer_store_handle: std::sync::Arc<dyn sc_network::peer_store::PeerStoreProvider>,
) -> (N::NotificationProtocolConfig, Box<dyn sc_network::NotificationService>) {
let (cfg, notification_service) = N::notification_config(
gossip_protocol_name,
Vec::new(),
1024 * 1024,
None,
Default::default(),
sc_network::config::SetConfig {
in_peers: 25,
out_peers: 25,
reserved_nodes: Vec::new(),
non_reserved_mode: sc_network::config::NonReservedPeerMode::Accept,
},
metrics,
peer_store_handle,
);
cfg.allow_non_reserved(25, 25);
(cfg, notification_service)
}
substrate/client/consensus/beefy/src/communication/peers.rs
+2 -1
View File
@@ -18,7 +18,8 @@
//! Logic for keeping track of BEEFY peers.
use sc_network::{PeerId, ReputationChange};
use sc_network::ReputationChange;
use sc_network_types::PeerId;
use sp_runtime::traits::{Block, NumberFor, Zero};
use std::collections::{HashMap, VecDeque};
substrate/client/consensus/beefy/src/communication/request_response/incoming_requests_handler.rs
+8 -7
View File
@@ -21,9 +21,10 @@ use futures::{channel::oneshot, StreamExt};
use log::{debug, trace};
use sc_client_api::BlockBackend;
use sc_network::{
config as netconfig, config::RequestResponseConfig, types::ProtocolName, PeerId,
ReputationChange,
config as netconfig, service::traits::RequestResponseConfig, types::ProtocolName,
NetworkBackend, ReputationChange,
};
use sc_network_types::PeerId;
use sp_consensus_beefy::BEEFY_ENGINE_ID;
use sp_runtime::traits::Block;
use std::{marker::PhantomData, sync::Arc};
@@ -139,15 +140,15 @@ where
Client: BlockBackend<B> + Send + Sync,
{
/// Create a new [`BeefyJustifsRequestHandler`].
pub fn new<Hash: AsRef<[u8]>>(
pub fn new<Hash: AsRef<[u8]>, Network: NetworkBackend<B, <B as Block>::Hash>>(
genesis_hash: Hash,
fork_id: Option<&str>,
client: Arc<Client>,
prometheus_registry: Option<prometheus::Registry>,
) -> (Self, RequestResponseConfig) {
let (request_receiver, config) =
on_demand_justifications_protocol_config(genesis_hash, fork_id);
let justif_protocol_name = config.name.clone();
) -> (Self, Network::RequestResponseProtocolConfig) {
let (request_receiver, config): (_, Network::RequestResponseProtocolConfig) =
on_demand_justifications_protocol_config::<_, _, Network>(genesis_hash, fork_id);
let justif_protocol_name = config.protocol_name().clone();
let metrics = register_metrics(prometheus_registry);
(
Self { request_receiver, justif_protocol_name, client, metrics, _block: PhantomData },
substrate/client/consensus/beefy/src/communication/request_response/mod.rs
+14 -9
View File
@@ -26,7 +26,8 @@ pub use incoming_requests_handler::BeefyJustifsRequestHandler;
use std::time::Duration;
use codec::{Decode, Encode, Error as CodecError};
use sc_network::{config::RequestResponseConfig, PeerId};
use sc_network::NetworkBackend;
use sc_network_types::PeerId;
use sp_runtime::traits::{Block, NumberFor};
use crate::communication::{beefy_protocol_name::justifications_protocol_name, peers::PeerReport};
@@ -47,23 +48,27 @@ const BEEFY_SYNC_LOG_TARGET: &str = "beefy::sync";
/// `ProtocolConfig`.
///
/// Consider using [`BeefyJustifsRequestHandler`] instead of this low-level function.
pub(crate) fn on_demand_justifications_protocol_config<Hash: AsRef<[u8]>>(
pub(crate) fn on_demand_justifications_protocol_config<
Hash: AsRef<[u8]>,
B: Block,
Network: NetworkBackend<B, <B as Block>::Hash>,
>(
genesis_hash: Hash,
fork_id: Option<&str>,
) -> (IncomingRequestReceiver, RequestResponseConfig) {
) -> (IncomingRequestReceiver, Network::RequestResponseProtocolConfig) {
let name = justifications_protocol_name(genesis_hash, fork_id);
let fallback_names = vec![];
let (tx, rx) = async_channel::bounded(JUSTIF_CHANNEL_SIZE);
let rx = IncomingRequestReceiver::new(rx);
let cfg = RequestResponseConfig {
let cfg = Network::request_response_config(
name,
fallback_names,
max_request_size: 32,
max_response_size: MAX_RESPONSE_SIZE,
32,
MAX_RESPONSE_SIZE,
// We are connected to all validators:
request_timeout: JUSTIF_REQUEST_TIMEOUT,
inbound_queue: Some(tx),
};
JUSTIF_REQUEST_TIMEOUT,
Some(tx),
);
(rx, cfg)
}
substrate/client/consensus/beefy/src/communication/request_response/outgoing_requests_engine.rs
+2 -1
View File
@@ -24,8 +24,9 @@ use log::{debug, warn};
use parking_lot::Mutex;
use sc_network::{
request_responses::{IfDisconnected, RequestFailure},
NetworkRequest, PeerId, ProtocolName,
NetworkRequest, ProtocolName,
};
use sc_network_types::PeerId;
use sp_consensus_beefy::{ecdsa_crypto::AuthorityId, ValidatorSet};
use sp_runtime::traits::{Block, NumberFor};
use std::{collections::VecDeque, result::Result, sync::Arc};
substrate/client/consensus/beefy/src/tests.rs
+5 -1
View File
@@ -125,7 +125,11 @@ impl BeefyTestNet {
let mut net = BeefyTestNet { peers: Vec::with_capacity(n_authority), beefy_genesis };
for i in 0..n_authority {
let (rx, cfg) = on_demand_justifications_protocol_config(GENESIS_HASH, None);
let (rx, cfg) = on_demand_justifications_protocol_config::<
_,
Block,
sc_network::NetworkWorker<_, _>,
>(GENESIS_HASH, None);
let justif_protocol_name = cfg.name.clone();
net.add_authority_peer(vec![cfg]);
substrate/client/consensus/common/Cargo.toml
+1 -1
View File
@@ -19,7 +19,6 @@ targets = ["x86_64-unknown-linux-gnu"]
async-trait = "0.1.79"
futures = { version = "0.3.30", features = ["thread-pool"] }
futures-timer = "3.0.1"
libp2p-identity = { version = "0.1.3", features = ["ed25519", "peerid"] }
log = { workspace = true, default-features = true }
mockall = "0.11.3"
parking_lot = "0.12.1"
@@ -27,6 +26,7 @@ serde = { features = ["derive"], workspace = true, default-features = true }
thiserror = { workspace = true }
prometheus-endpoint = { package = "substrate-prometheus-endpoint", path = "../../../utils/prometheus" }
sc-client-api = { path = "../../api" }
sc-network-types = { path = "../../network/types" }
sc-utils = { path = "../../utils" }
sp-api = { path = "../../../primitives/api" }
sp-blockchain = { path = "../../../primitives/blockchain" }
substrate/client/consensus/common/src/import_queue.rs
+1 -1
View File
@@ -64,7 +64,7 @@ pub type BoxJustificationImport<B> =
Box<dyn JustificationImport<B, Error = ConsensusError> + Send + Sync>;
/// Maps to the RuntimeOrigin used by the network.
pub type RuntimeOrigin = libp2p_identity::PeerId;
pub type RuntimeOrigin = sc_network_types::PeerId;
/// Block data used by the queue.
#[derive(Debug, PartialEq, Eq, Clone)]
substrate/client/consensus/common/src/import_queue/basic_queue.rs
+1 -1
View File
@@ -632,7 +632,7 @@ mod tests {
let hash = Hash::random();
finality_sender
.unbounded_send(worker_messages::ImportJustification(
libp2p_identity::PeerId::random(),
sc_network_types::PeerId::random(),
hash,
1,
(*b"TEST", Vec::new()),
substrate/client/consensus/grandpa/Cargo.toml
+1
View File
@@ -41,6 +41,7 @@ sc-network = { path = "../../network" }
sc-network-gossip = { path = "../../network-gossip" }
sc-network-common = { path = "../../network/common" }
sc-network-sync = { path = "../../network/sync" }
sc-network-types = { path = "../../network/types" }
sc-telemetry = { path = "../../telemetry" }
sc-utils = { path = "../../utils" }
sp-api = { path = "../../../primitives/api" }
substrate/client/consensus/grandpa/src/communication/gossip.rs
+2 -1
View File
@@ -90,9 +90,10 @@ use log::{debug, trace};
use parity_scale_codec::{Decode, DecodeAll, Encode};
use prometheus_endpoint::{register, CounterVec, Opts, PrometheusError, Registry, U64};
use rand::seq::SliceRandom;
use sc_network::{PeerId, ReputationChange};
use sc_network::ReputationChange;
use sc_network_common::role::ObservedRole;
use sc_network_gossip::{MessageIntent, ValidatorContext};
use sc_network_types::PeerId;
use sc_telemetry::{telemetry, TelemetryHandle, CONSENSUS_DEBUG};
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver, TracingUnboundedSender};
use sp_consensus_grandpa::AuthorityId;
substrate/client/consensus/grandpa/src/communication/mod.rs
+1 -1
View File
@@ -488,7 +488,7 @@ impl<B: BlockT, N: Network<B>, S: Syncing<B>> NetworkBridge<B, N, S> {
/// connected to (NOTE: this assumption will change in the future #3629).
pub(crate) fn set_sync_fork_request(
&self,
peers: Vec<sc_network::PeerId>,
peers: Vec<sc_network_types::PeerId>,
hash: B::Hash,
number: NumberFor<B>,
) {
substrate/client/consensus/grandpa/src/communication/periodic.rs
+2 -2
View File
@@ -27,7 +27,7 @@ use std::{
time::Duration,
};
use sc_network::PeerId;
use sc_network_types::PeerId;
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver, TracingUnboundedSender};
use sp_runtime::traits::{Block as BlockT, NumberFor};
@@ -44,7 +44,7 @@ impl<B: BlockT> NeighborPacketSender<B> {
/// Send a neighbor packet for the background worker to gossip to peers.
pub fn send(
&self,
who: Vec<sc_network::PeerId>,
who: Vec<sc_network_types::PeerId>,
neighbor_packet: NeighborPacket<NumberFor<B>>,
) {
if let Err(err) = self.0.unbounded_send((who, neighbor_packet)) {
substrate/client/consensus/grandpa/src/communication/tests.rs
+11 -29
View File
@@ -30,14 +30,14 @@ use sc_network::{
event::Event as NetworkEvent,
service::traits::{Direction, MessageSink, NotificationEvent, NotificationService},
types::ProtocolName,
Multiaddr, NetworkBlock, NetworkEventStream, NetworkNotification, NetworkPeers,
NetworkSyncForkRequest, NotificationSenderError, NotificationSenderT as NotificationSender,
PeerId, ReputationChange,
Multiaddr, NetworkBlock, NetworkEventStream, NetworkPeers, NetworkSyncForkRequest,
ReputationChange,
};
use sc_network_common::role::{ObservedRole, Roles};
use sc_network_gossip::Validator;
use sc_network_sync::{SyncEvent as SyncStreamEvent, SyncEventStream};
use sc_network_test::{Block, Hash};
use sc_network_types::PeerId;
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver, TracingUnboundedSender};
use sp_consensus_grandpa::AuthorityList;
use sp_keyring::Ed25519Keyring;
@@ -62,6 +62,7 @@ pub(crate) struct TestNetwork {
sender: TracingUnboundedSender<Event>,
}
#[async_trait::async_trait]
impl NetworkPeers for TestNetwork {
fn set_authorized_peers(&self, _peers: HashSet<PeerId>) {
unimplemented!();
@@ -134,6 +135,10 @@ impl NetworkPeers for TestNetwork {
.ok()
.and_then(|role| Some(ObservedRole::from(role)))
}
async fn reserved_peers(&self) -> Result<Vec<PeerId>, ()> {
unimplemented!();
}
}
impl NetworkEventStream for TestNetwork {
@@ -147,24 +152,6 @@ impl NetworkEventStream for TestNetwork {
}
}
impl NetworkNotification for TestNetwork {
fn write_notification(&self, target: PeerId, _protocol: ProtocolName, message: Vec<u8>) {
let _ = self.sender.unbounded_send(Event::WriteNotification(target, message));
}
fn notification_sender(
&self,
_target: PeerId,
_protocol: ProtocolName,
) -> Result<Box<dyn NotificationSender>, NotificationSenderError> {
unimplemented!();
}
fn set_notification_handshake(&self, _protocol: ProtocolName, _handshake: Vec<u8>) {
unimplemented!();
}
}
impl NetworkBlock<Hash, NumberFor<Block>> for TestNetwork {
fn announce_block(&self, hash: Hash, _data: Option<Vec<u8>>) {
let _ = self.sender.unbounded_send(Event::Announce(hash));
@@ -185,12 +172,7 @@ impl sc_network_gossip::ValidatorContext<Block> for TestNetwork {
fn broadcast_message(&mut self, _: Hash, _: Vec<u8>, _: bool) {}
fn send_message(&mut self, who: &PeerId, data: Vec<u8>) {
<Self as NetworkNotification>::write_notification(
self,
*who,
grandpa_protocol_name::NAME.into(),
data,
);
let _ = self.sender.unbounded_send(Event::WriteNotification(*who, data));
}
fn send_topic(&mut self, _: &PeerId, _: Hash, _: bool) {}
@@ -241,13 +223,13 @@ impl NotificationService for TestNotificationService {
}
/// Send synchronous `notification` to `peer`.
fn send_sync_notification(&self, peer: &PeerId, notification: Vec<u8>) {
fn send_sync_notification(&mut self, peer: &PeerId, notification: Vec<u8>) {
let _ = self.sender.unbounded_send(Event::WriteNotification(*peer, notification));
}
/// Send asynchronous `notification` to `peer`, allowing sender to exercise backpressure.
async fn send_async_notification(
&self,
&mut self,
_peer: &PeerId,
_notification: Vec<u8>,
) -> Result<(), sc_network::error::Error> {
substrate/client/consensus/grandpa/src/lib.rs
+10 -6
View File
@@ -67,7 +67,7 @@ use sc_client_api::{
BlockchainEvents, CallExecutor, ExecutorProvider, Finalizer, LockImportRun, StorageProvider,
};
use sc_consensus::BlockImport;
use sc_network::{types::ProtocolName, NotificationService};
use sc_network::{types::ProtocolName, NetworkBackend, NotificationService};
use sc_telemetry::{telemetry, TelemetryHandle, CONSENSUS_DEBUG, CONSENSUS_INFO};
use sc_transaction_pool_api::OffchainTransactionPoolFactory;
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver};
@@ -343,7 +343,7 @@ pub(crate) trait BlockSyncRequester<Block: BlockT> {
/// connected to (NOTE: this assumption will change in the future #3629).
fn set_sync_fork_request(
&self,
peers: Vec<sc_network::PeerId>,
peers: Vec<sc_network_types::PeerId>,
hash: Block::Hash,
number: NumberFor<Block>,
);
@@ -357,7 +357,7 @@ where
{
fn set_sync_fork_request(
&self,
peers: Vec<sc_network::PeerId>,
peers: Vec<sc_network_types::PeerId>,
hash: Block::Hash,
number: NumberFor<Block>,
) {
@@ -707,11 +707,13 @@ pub struct GrandpaParams<Block: BlockT, C, N, S, SC, VR> {
/// Returns the configuration value to put in
/// [`sc_network::config::FullNetworkConfiguration`].
/// For standard protocol name see [`crate::protocol_standard_name`].
pub fn grandpa_peers_set_config(
pub fn grandpa_peers_set_config<B: BlockT, N: NetworkBackend<B, <B as BlockT>::Hash>>(
protocol_name: ProtocolName,
) -> (sc_network::config::NonDefaultSetConfig, Box<dyn NotificationService>) {
metrics: sc_network::service::NotificationMetrics,
peer_store_handle: Arc<dyn sc_network::peer_store::PeerStoreProvider>,
) -> (N::NotificationProtocolConfig, Box<dyn NotificationService>) {
use communication::grandpa_protocol_name;
sc_network::config::NonDefaultSetConfig::new(
N::notification_config(
protocol_name,
grandpa_protocol_name::LEGACY_NAMES.iter().map(|&n| n.into()).collect(),
// Notifications reach ~256kiB in size at the time of writing on Kusama and Polkadot.
@@ -723,6 +725,8 @@ pub fn grandpa_peers_set_config(
reserved_nodes: Vec::new(),
non_reserved_mode: sc_network::config::NonReservedPeerMode::Deny,
},
metrics,
peer_store_handle,
)
}
substrate/client/consensus/grandpa/src/observer.rs
+1 -1
View File
@@ -410,7 +410,7 @@ mod tests {
communication::tests::{make_test_network, Event},
};
use assert_matches::assert_matches;
use sc_network::PeerId;
use sc_network_types::PeerId;
use sc_utils::mpsc::tracing_unbounded;
use sp_blockchain::HeaderBackend as _;
use substrate_test_runtime_client::{TestClientBuilder, TestClientBuilderExt};
substrate/client/consensus/grandpa/src/until_imported.rs
+1 -1
View File
@@ -632,7 +632,7 @@ mod tests {
impl BlockSyncRequesterT<Block> for TestBlockSyncRequester {
fn set_sync_fork_request(
&self,
_peers: Vec<sc_network::PeerId>,
_peers: Vec<sc_network_types::PeerId>,
hash: Hash,
number: NumberFor<Block>,
) {
substrate/client/mixnet/Cargo.toml
+1 -1
View File
@@ -23,13 +23,13 @@ bytes = "1"
codec = { package = "parity-scale-codec", version = "3.6.1", default-features = false, features = ["derive"] }
futures = "0.3.30"
futures-timer = "3.0.2"
libp2p-identity = { version = "0.1.3", features = ["peerid"] }
log = { workspace = true, default-features = true }
mixnet = "0.7.0"
multiaddr = "0.17.1"
parking_lot = "0.12.1"
sc-client-api = { path = "../api" }
sc-network = { path = "../network" }
sc-network-types = { path = "../network/types" }
sc-transaction-pool-api = { path = "../transaction-pool/api" }
sp-api = { path = "../../primitives/api" }
sp-consensus = { path = "../../primitives/consensus/common" }
substrate/client/mixnet/src/packet_dispatcher.rs
+1 -1
View File
@@ -20,11 +20,11 @@
use super::peer_id::{from_core_peer_id, to_core_peer_id};
use arrayvec::ArrayVec;
use libp2p_identity::PeerId;
use log::{debug, warn};
use mixnet::core::{AddressedPacket, NetworkStatus, Packet, PeerId as CorePeerId};
use parking_lot::Mutex;
use sc_network::NotificationService;
use sc_network_types::PeerId;
use std::{collections::HashMap, future::Future, sync::Arc};
const LOG_TARGET: &str = "mixnet";
substrate/client/mixnet/src/peer_id.rs
+3 -11
View File
@@ -16,21 +16,15 @@
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use libp2p_identity::PeerId;
use mixnet::core::PeerId as CorePeerId;
use sc_network_types::PeerId;
/// Convert a libp2p [`PeerId`] into a mixnet core [`PeerId`](CorePeerId).
///
/// This will succeed only if `peer_id` is an Ed25519 public key ("hashed" using the identity
/// hasher). Returns `None` on failure.
pub fn to_core_peer_id(peer_id: &PeerId) -> Option<CorePeerId> {
let hash = peer_id.as_ref();
if hash.code() != 0 {
// Hash is not identity
return None
}
let public = libp2p_identity::PublicKey::try_decode_protobuf(hash.digest()).ok()?;
public.try_into_ed25519().ok().map(|public| public.to_bytes())
peer_id.into_ed25519()
}
/// Convert a mixnet core [`PeerId`](CorePeerId) into a libp2p [`PeerId`].
@@ -38,7 +32,5 @@ pub fn to_core_peer_id(peer_id: &PeerId) -> Option<CorePeerId> {
/// This will succeed only if `peer_id` represents a point on the Ed25519 curve. Returns `None` on
/// failure.
pub fn from_core_peer_id(core_peer_id: &CorePeerId) -> Option<PeerId> {
let public = libp2p_identity::ed25519::PublicKey::try_from_bytes(core_peer_id).ok()?;
let public: libp2p_identity::PublicKey = public.into();
Some(public.into())
PeerId::from_ed25519(core_peer_id)
}
substrate/client/mixnet/src/protocol.rs
+31 -17
View File
@@ -19,9 +19,12 @@
use super::config::Config;
use mixnet::core::PACKET_SIZE;
use sc_network::{
config::{NonDefaultSetConfig, NonReservedPeerMode, SetConfig},
NotificationService, ProtocolName,
config::{NonReservedPeerMode, SetConfig},
peer_store::PeerStoreProvider,
service::NotificationMetrics,
NetworkBackend, NotificationService, ProtocolName,
};
use sp_runtime::traits::Block as BlockT;
/// Returns the protocol name to use for the mixnet controlled by the given chain.
pub fn protocol_name(genesis_hash: &[u8], fork_id: Option<&str>) -> ProtocolName {
@@ -34,26 +37,37 @@ pub fn protocol_name(genesis_hash: &[u8], fork_id: Option<&str>) -> ProtocolName
}
/// Returns the peers set configuration for the mixnet protocol.
pub fn peers_set_config(
pub fn peers_set_config<Block: BlockT, Network: NetworkBackend<Block, <Block as BlockT>::Hash>>(
name: ProtocolName,
config: &Config,
) -> (NonDefaultSetConfig, Box<dyn NotificationService>) {
let (mut set_config, service) = NonDefaultSetConfig::new(
name,
Vec::new(),
PACKET_SIZE as u64,
None,
metrics: NotificationMetrics,
peerstore_handle: std::sync::Arc<dyn PeerStoreProvider>,
) -> (Network::NotificationProtocolConfig, Box<dyn NotificationService>) {
let set_config = if config.substrate.num_gateway_slots != 0 {
// out_peers is always 0; we are only interested in connecting to mixnodes, which we do by
// setting them as reserved nodes
SetConfig {
in_peers: config.substrate.num_gateway_slots,
out_peers: 0,
reserved_nodes: Vec::new(),
non_reserved_mode: NonReservedPeerMode::Accept,
}
} else {
SetConfig {
in_peers: 0,
out_peers: 0,
reserved_nodes: Vec::new(),
non_reserved_mode: NonReservedPeerMode::Deny,
},
);
if config.substrate.num_gateway_slots != 0 {
// out_peers is always 0; we are only interested in connecting to mixnodes, which we do by
// setting them as reserved nodes
set_config.allow_non_reserved(config.substrate.num_gateway_slots, 0);
}
(set_config, service)
}
};
Network::notification_config(
name,
Vec::new(),
PACKET_SIZE as u64,
None,
set_config,
metrics,
peerstore_handle,
)
}
substrate/client/mixnet/src/run.rs
+4 -5
View File
@@ -44,8 +44,8 @@ use mixnet::{
};
use sc_client_api::{BlockchainEvents, HeaderBackend};
use sc_network::{
service::traits::{NotificationEvent, ValidationResult},
NetworkNotification, NetworkPeers, NetworkStateInfo, NotificationService, ProtocolName,
service::traits::{NetworkService, NotificationEvent, ValidationResult},
NetworkPeers, NetworkStateInfo, NotificationService, ProtocolName,
};
use sc_transaction_pool_api::{
LocalTransactionPool, OffchainTransactionPoolFactory, TransactionPool,
@@ -146,12 +146,12 @@ fn time_until(instant: Instant) -> Duration {
/// Run the mixnet service. If `keystore` is `None`, the service will not attempt to register the
/// local node as a mixnode, even if `config.register` is `true`.
pub async fn run<B, C, S, N, P>(
pub async fn run<B, C, S, P>(
config: Config,
mut api_backend: ApiBackend,
client: Arc<C>,
sync: Arc<S>,
network: Arc<N>,
network: Arc<dyn NetworkService>,
protocol_name: ProtocolName,
transaction_pool: Arc<P>,
keystore: Option<KeystorePtr>,
@@ -161,7 +161,6 @@ pub async fn run<B, C, S, N, P>(
C: BlockchainEvents<B> + ProvideRuntimeApi<B> + HeaderBackend<B>,
C::Api: MixnetApi<B>,
S: SyncOracle,
N: NetworkStateInfo + NetworkNotification + NetworkPeers,
P: TransactionPool<Block = B> + LocalTransactionPool<Block = B> + 'static,
{
let local_peer_id = network.local_peer_id();
substrate/client/mixnet/src/sync_with_runtime.rs
+1 -1
View File
@@ -20,13 +20,13 @@
//! runtime to the core mixnet state. It is called every time a block is finalised.
use super::peer_id::from_core_peer_id;
use libp2p_identity::PeerId;
use log::{debug, info};
use mixnet::core::{
Mixnet, Mixnode as CoreMixnode, MixnodesErr as CoreMixnodesErr, RelSessionIndex,
SessionPhase as CoreSessionPhase, SessionStatus as CoreSessionStatus,
};
use multiaddr::{multiaddr, Multiaddr, Protocol};
use sc_network_types::PeerId;
use sp_api::{ApiError, ApiRef};
use sp_mixnet::{
runtime_api::MixnetApi,
substrate/client/network-gossip/Cargo.toml
+1
View File
@@ -28,6 +28,7 @@ prometheus-endpoint = { package = "substrate-prometheus-endpoint", path = "../..
sc-network = { path = "../network" }
sc-network-common = { path = "../network/common" }
sc-network-sync = { path = "../network/sync" }
sc-network-types = { path = "../network/types" }
sp-runtime = { path = "../../primitives/runtime" }
[dev-dependencies]
substrate/client/network-gossip/src/bridge.rs
+9 -24
View File
@@ -32,9 +32,9 @@ use futures::{
channel::mpsc::{channel, Receiver, Sender},
prelude::*,
};
use libp2p::PeerId;
use log::trace;
use prometheus_endpoint::Registry;
use sc_network_types::PeerId;
use sp_runtime::traits::Block as BlockT;
use std::{
collections::{HashMap, VecDeque},
@@ -359,9 +359,7 @@ mod tests {
use sc_network::{
config::MultiaddrWithPeerId,
service::traits::{Direction, MessageSink, NotificationEvent},
Event, NetworkBlock, NetworkEventStream, NetworkNotification, NetworkPeers,
NotificationSenderError, NotificationSenderT as NotificationSender, NotificationService,
Roles,
Event, NetworkBlock, NetworkEventStream, NetworkPeers, NotificationService, Roles,
};
use sc_network_common::role::ObservedRole;
use sc_network_sync::SyncEventStream;
@@ -381,6 +379,7 @@ mod tests {
#[derive(Clone, Default)]
struct TestNetworkInner {}
#[async_trait::async_trait]
impl NetworkPeers for TestNetwork {
fn set_authorized_peers(&self, _peers: HashSet<PeerId>) {
unimplemented!();
@@ -453,6 +452,10 @@ mod tests {
.ok()
.and_then(|role| Some(ObservedRole::from(role)))
}
async fn reserved_peers(&self) -> Result<Vec<PeerId>, ()> {
unimplemented!();
}
}
impl NetworkEventStream for TestNetwork {
@@ -461,24 +464,6 @@ mod tests {
}
}
impl NetworkNotification for TestNetwork {
fn write_notification(&self, _target: PeerId, _protocol: ProtocolName, _message: Vec<u8>) {
unimplemented!();
}
fn notification_sender(
&self,
_target: PeerId,
_protocol: ProtocolName,
) -> Result<Box<dyn NotificationSender>, NotificationSenderError> {
unimplemented!();
}
fn set_notification_handshake(&self, _protocol: ProtocolName, _handshake: Vec<u8>) {
unimplemented!();
}
}
impl NetworkBlock<<Block as BlockT>::Hash, NumberFor<Block>> for TestNetwork {
fn announce_block(&self, _hash: <Block as BlockT>::Hash, _data: Option<Vec<u8>>) {
unimplemented!();
@@ -544,12 +529,12 @@ mod tests {
unimplemented!();
}
fn send_sync_notification(&self, _peer: &PeerId, _notification: Vec<u8>) {
fn send_sync_notification(&mut self, _peer: &PeerId, _notification: Vec<u8>) {
unimplemented!();
}
async fn send_async_notification(
&self,
&mut self,
_peer: &PeerId,
_notification: Vec<u8>,
) -> Result<(), sc_network::error::Error> {
substrate/client/network-gossip/src/lib.rs
+4 -6
View File
@@ -67,11 +67,9 @@ pub use self::{
validator::{DiscardAll, MessageIntent, ValidationResult, Validator, ValidatorContext},
};
use libp2p::{multiaddr, PeerId};
use sc_network::{
types::ProtocolName, NetworkBlock, NetworkEventStream, NetworkNotification, NetworkPeers,
};
use sc_network::{multiaddr, types::ProtocolName, NetworkBlock, NetworkEventStream, NetworkPeers};
use sc_network_sync::SyncEventStream;
use sc_network_types::PeerId;
use sp_runtime::traits::{Block as BlockT, NumberFor};
use std::iter;
@@ -80,7 +78,7 @@ mod state_machine;
mod validator;
/// Abstraction over a network.
pub trait Network<B: BlockT>: NetworkPeers + NetworkEventStream + NetworkNotification {
pub trait Network<B: BlockT>: NetworkPeers + NetworkEventStream {
fn add_set_reserved(&self, who: PeerId, protocol: ProtocolName) {
let addr =
iter::once(multiaddr::Protocol::P2p(who.into())).collect::<multiaddr::Multiaddr>();
@@ -97,7 +95,7 @@ pub trait Network<B: BlockT>: NetworkPeers + NetworkEventStream + NetworkNotific
}
}
impl<T, B: BlockT> Network<B> for T where T: NetworkPeers + NetworkEventStream + NetworkNotification {}
impl<T, B: BlockT> Network<B> for T where T: NetworkPeers + NetworkEventStream {}
/// Abstraction over the syncing subsystem.
pub trait Syncing<B: BlockT>: SyncEventStream + NetworkBlock<B::Hash, NumberFor<B>> {}
substrate/client/network-gossip/src/state_machine.rs
+9 -23
View File
@@ -19,7 +19,7 @@
use crate::{MessageIntent, Network, ValidationResult, Validator, ValidatorContext};
use ahash::AHashSet;
use libp2p::PeerId;
use sc_network_types::PeerId;
use schnellru::{ByLength, LruMap};
use prometheus_endpoint::{register, Counter, PrometheusError, Registry, U64};
@@ -546,8 +546,7 @@ mod tests {
use futures::prelude::*;
use sc_network::{
config::MultiaddrWithPeerId, event::Event, service::traits::NotificationEvent, MessageSink,
NetworkBlock, NetworkEventStream, NetworkNotification, NetworkPeers,
NotificationSenderError, NotificationSenderT as NotificationSender, ReputationChange,
NetworkBlock, NetworkEventStream, NetworkPeers, ReputationChange,
};
use sp_runtime::{
testing::{Block as RawBlock, ExtrinsicWrapper, H256},
@@ -608,6 +607,7 @@ mod tests {
peer_reports: Vec<(PeerId, ReputationChange)>,
}
#[async_trait::async_trait]
impl NetworkPeers for NoOpNetwork {
fn set_authorized_peers(&self, _peers: HashSet<PeerId>) {
unimplemented!();
@@ -680,6 +680,10 @@ mod tests {
fn peer_role(&self, _peer_id: PeerId, _handshake: Vec<u8>) -> Option<ObservedRole> {
None
}
async fn reserved_peers(&self) -> Result<Vec<PeerId>, ()> {
unimplemented!();
}
}
impl NetworkEventStream for NoOpNetwork {
@@ -688,24 +692,6 @@ mod tests {
}
}
impl NetworkNotification for NoOpNetwork {
fn write_notification(&self, _target: PeerId, _protocol: ProtocolName, _message: Vec<u8>) {
unimplemented!();
}
fn notification_sender(
&self,
_target: PeerId,
_protocol: ProtocolName,
) -> Result<Box<dyn NotificationSender>, NotificationSenderError> {
unimplemented!();
}
fn set_notification_handshake(&self, _protocol: ProtocolName, _handshake: Vec<u8>) {
unimplemented!();
}
}
impl NetworkBlock<<Block as BlockT>::Hash, NumberFor<Block>> for NoOpNetwork {
fn announce_block(&self, _hash: <Block as BlockT>::Hash, _data: Option<Vec<u8>>) {
unimplemented!();
@@ -736,13 +722,13 @@ mod tests {
}
/// Send synchronous `notification` to `peer`.
fn send_sync_notification(&self, _peer: &PeerId, _notification: Vec<u8>) {
fn send_sync_notification(&mut self, _peer: &PeerId, _notification: Vec<u8>) {
unimplemented!();
}
/// Send asynchronous `notification` to `peer`, allowing sender to exercise backpressure.
async fn send_async_notification(
&self,
&mut self,
_peer: &PeerId,
_notification: Vec<u8>,
) -> Result<(), sc_network::error::Error> {
substrate/client/network-gossip/src/validator.rs
+1 -1
View File
@@ -16,8 +16,8 @@
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use libp2p::PeerId;
use sc_network_common::role::ObservedRole;
use sc_network_types::PeerId;
use sp_runtime::traits::Block as BlockT;
/// Validates consensus messages.
substrate/client/network/Cargo.toml
+14 -1
View File
@@ -16,12 +16,16 @@ workspace = true
[package.metadata.docs.rs]
targets = ["x86_64-unknown-linux-gnu"]
[build-dependencies]
prost-build = "0.11"
[dependencies]
array-bytes = "6.1"
async-channel = "1.8.0"
async-trait = "0.1.79"
asynchronous-codec = "0.6"
bytes = "1"
cid = "0.9.0"
codec = { package = "parity-scale-codec", version = "3.6.1", features = ["derive"] }
either = "1.5.3"
fnv = "1.0.6"
@@ -42,17 +46,23 @@ smallvec = "1.11.0"
thiserror = { workspace = true }
tokio = { version = "1.22.0", features = ["macros", "sync"] }
tokio-stream = "0.1.7"
unsigned-varint = { version = "0.7.1", features = ["asynchronous_codec", "futures"] }
unsigned-varint = { version = "0.7.2", features = ["asynchronous_codec", "futures"] }
zeroize = "1.4.3"
prometheus-endpoint = { package = "substrate-prometheus-endpoint", path = "../../utils/prometheus" }
prost = "0.11"
sc-client-api = { path = "../api" }
sc-network-common = { path = "common" }
sc-network-types = { path = "types" }
sc-utils = { path = "../utils" }
sp-arithmetic = { path = "../../primitives/arithmetic" }
sp-blockchain = { path = "../../primitives/blockchain" }
sp-core = { path = "../../primitives/core" }
sp-runtime = { path = "../../primitives/runtime" }
wasm-timer = "0.2"
litep2p = { git = "https://github.com/paritytech/litep2p", branch = "master" }
once_cell = "1.18.0"
void = "1.0.2"
schnellru = "0.2.1"
[dev-dependencies]
assert_matches = "1.3"
@@ -63,8 +73,11 @@ tempfile = "3.1.0"
tokio = { version = "1.22.0", features = ["macros"] }
tokio-util = { version = "0.7.4", features = ["compat"] }
tokio-test = "0.4.2"
sc-block-builder = { path = "../block-builder" }
sc-network-light = { path = "light" }
sc-network-sync = { path = "sync" }
sp-crypto-hashing = { path = "../../primitives/crypto/hashing" }
sp-consensus = { path = "../../primitives/consensus/common" }
sp-test-primitives = { path = "../../primitives/test-primitives" }
sp-tracing = { path = "../../primitives/tracing" }
substrate-test-runtime = { path = "../../test-utils/runtime" }
substrate/client/network/bitswap/Cargo.toml
-42
View File
@@ -1,42 +0,0 @@
[package]
description = "Substrate bitswap protocol"
name = "sc-network-bitswap"
version = "0.33.0"
license = "GPL-3.0-or-later WITH Classpath-exception-2.0"
authors.workspace = true
edition.workspace = true
homepage = "https://substrate.io"
repository.workspace = true
documentation = "https://docs.rs/sc-network-bitswap"
[lints]
workspace = true
[package.metadata.docs.rs]
targets = ["x86_64-unknown-linux-gnu"]
[build-dependencies]
prost-build = "0.11"
[dependencies]
async-channel = "1.8.0"
cid = "0.9.0"
futures = "0.3.30"
libp2p-identity = { version = "0.1.3", features = ["peerid"] }
log = { workspace = true, default-features = true }
prost = "0.12"
thiserror = { workspace = true }
unsigned-varint = { version = "0.7.1", features = ["asynchronous_codec", "futures"] }
sc-client-api = { path = "../../api" }
sc-network = { path = ".." }
sp-blockchain = { path = "../../../primitives/blockchain" }
sp-runtime = { path = "../../../primitives/runtime" }
[dev-dependencies]
tokio = { version = "1.22.0", features = ["full"] }
sc-block-builder = { path = "../../block-builder" }
sc-consensus = { path = "../../consensus/common" }
sp-crypto-hashing = { path = "../../../primitives/crypto/hashing" }
sp-consensus = { path = "../../../primitives/consensus/common" }
substrate-test-runtime = { path = "../../../test-utils/runtime" }
substrate-test-runtime-client = { path = "../../../test-utils/runtime/client" }
substrate/client/network/bitswap/build.rs → substrate/client/network/build.rs
View File
substrate/client/network/common/Cargo.toml
+1
View File
@@ -27,6 +27,7 @@ codec = { package = "parity-scale-codec", version = "3.6.1", features = [
futures = "0.3.30"
libp2p-identity = { version = "0.1.3", features = ["peerid"] }
sc-consensus = { path = "../../consensus/common" }
sc-network-types = { path = "../types" }
sp-consensus = { path = "../../../primitives/consensus/common" }
sp-consensus-grandpa = { path = "../../../primitives/consensus/grandpa" }
sp-runtime = { path = "../../../primitives/runtime" }
substrate/client/network/light/Cargo.toml
+1 -1
View File
@@ -25,11 +25,11 @@ codec = { package = "parity-scale-codec", version = "3.6.1", features = [
"derive",
] }
futures = "0.3.30"
libp2p-identity = { version = "0.1.3", features = ["peerid"] }
log = { workspace = true, default-features = true }
prost = "0.12"
sp-blockchain = { path = "../../../primitives/blockchain" }
sc-client-api = { path = "../../api" }
sc-network-types = { path = "../types" }
sc-network = { path = ".." }
sp-core = { path = "../../../primitives/core" }
sp-runtime = { path = "../../../primitives/runtime" }
substrate/client/network/light/src/light_client_requests.rs
+19 -14
View File
@@ -18,7 +18,8 @@
//! Helpers for outgoing and incoming light client requests.
use sc_network::{config::ProtocolId, request_responses::ProtocolConfig};
use sc_network::{config::ProtocolId, request_responses::IncomingRequest, NetworkBackend};
use sp_runtime::traits::Block;
use std::time::Duration;
@@ -40,20 +41,24 @@ fn generate_legacy_protocol_name(protocol_id: &ProtocolId) -> String {
format!("/{}/light/2", protocol_id.as_ref())
}
/// Generates a [`ProtocolConfig`] for the light client request protocol, refusing incoming
/// requests.
pub fn generate_protocol_config<Hash: AsRef<[u8]>>(
/// Generates a `RequestResponseProtocolConfig` for the light client request protocol, refusing
/// incoming requests.
pub fn generate_protocol_config<
Hash: AsRef<[u8]>,
B: Block,
N: NetworkBackend<B, <B as Block>::Hash>,
>(
protocol_id: &ProtocolId,
genesis_hash: Hash,
fork_id: Option<&str>,
) -> ProtocolConfig {
ProtocolConfig {
name: generate_protocol_name(genesis_hash, fork_id).into(),
fallback_names: std::iter::once(generate_legacy_protocol_name(protocol_id).into())
.collect(),
max_request_size: 1 * 1024 * 1024,
max_response_size: 16 * 1024 * 1024,
request_timeout: Duration::from_secs(15),
inbound_queue: None,
}
inbound_queue: async_channel::Sender<IncomingRequest>,
) -> N::RequestResponseProtocolConfig {
N::request_response_config(
generate_protocol_name(genesis_hash, fork_id).into(),
std::iter::once(generate_legacy_protocol_name(protocol_id).into()).collect(),
1 * 1024 * 1024,
16 * 1024 * 1024,
Duration::from_secs(15),
Some(inbound_queue),
)
}
substrate/client/network/light/src/light_client_requests/handler.rs
+7 -7
View File
@@ -25,15 +25,15 @@
use crate::schema;
use codec::{self, Decode, Encode};
use futures::prelude::*;
use libp2p_identity::PeerId;
use log::{debug, trace};
use prost::Message;
use sc_client_api::{BlockBackend, ProofProvider};
use sc_network::{
config::ProtocolId,
request_responses::{IncomingRequest, OutgoingResponse, ProtocolConfig},
ReputationChange,
request_responses::{IncomingRequest, OutgoingResponse},
NetworkBackend, ReputationChange,
};
use sc_network_types::PeerId;
use sp_core::{
hexdisplay::HexDisplay,
storage::{ChildInfo, ChildType, PrefixedStorageKey},
@@ -61,14 +61,14 @@ where
Client: BlockBackend<B> + ProofProvider<B> + Send + Sync + 'static,
{
/// Create a new [`LightClientRequestHandler`].
pub fn new(
pub fn new<N: NetworkBackend<B, <B as Block>::Hash>>(
protocol_id: &ProtocolId,
fork_id: Option<&str>,
client: Arc<Client>,
) -> (Self, ProtocolConfig) {
) -> (Self, N::RequestResponseProtocolConfig) {
let (tx, request_receiver) = async_channel::bounded(MAX_LIGHT_REQUEST_QUEUE);
let mut protocol_config = super::generate_protocol_config(
let protocol_config = super::generate_protocol_config::<_, B, N>(
protocol_id,
client
.block_hash(0u32.into())
@@ -76,8 +76,8 @@ where
.flatten()
.expect("Genesis block exists; qed"),
fork_id,
tx,
);
protocol_config.inbound_queue = Some(tx);
(Self { client, request_receiver, _block: PhantomData::default() }, protocol_config)
}
substrate/client/network/src/behaviour.rs
+3 -3
View File
@@ -20,7 +20,7 @@ use crate::{
discovery::{DiscoveryBehaviour, DiscoveryConfig, DiscoveryOut},
event::DhtEvent,
peer_info,
peer_store::PeerStoreHandle,
peer_store::PeerStoreProvider,
protocol::{CustomMessageOutcome, NotificationsSink, Protocol},
protocol_controller::SetId,
request_responses::{self, IfDisconnected, ProtocolConfig, RequestFailure},
@@ -173,7 +173,7 @@ impl<B: BlockT> Behaviour<B> {
local_public_key: PublicKey,
disco_config: DiscoveryConfig,
request_response_protocols: Vec<ProtocolConfig>,
peer_store_handle: PeerStoreHandle,
peer_store_handle: Arc<dyn PeerStoreProvider>,
external_addresses: Arc<Mutex<HashSet<Multiaddr>>>,
) -> Result<Self, request_responses::RegisterError> {
Ok(Self {
@@ -186,7 +186,7 @@ impl<B: BlockT> Behaviour<B> {
discovery: disco_config.finish(),
request_responses: request_responses::RequestResponsesBehaviour::new(
request_response_protocols.into_iter(),
Box::new(peer_store_handle),
peer_store_handle,
)?,
})
}
substrate/client/network/bitswap/src/lib.rs → substrate/client/network/src/bitswap/mod.rs
+8 -7
View File
@@ -20,16 +20,17 @@
//! Only supports bitswap 1.2.0.
//! CID is expected to reference 256-bit Blake2b transaction hash.
use cid::{self, Version};
use futures::StreamExt;
use libp2p_identity::PeerId;
use log::{debug, error, trace};
use prost::Message;
use sc_client_api::BlockBackend;
use sc_network::{
use crate::{
request_responses::{IncomingRequest, OutgoingResponse, ProtocolConfig},
types::ProtocolName,
};
use cid::{self, Version};
use futures::StreamExt;
use log::{debug, error, trace};
use prost::Message;
use sc_client_api::BlockBackend;
use sc_network_types::PeerId;
use schema::bitswap::{
message::{wantlist::WantType, Block as MessageBlock, BlockPresence, BlockPresenceType},
Message as BitswapMessage,
substrate/client/network/bitswap/src/schema.rs → substrate/client/network/src/bitswap/schema.rs
View File
substrate/client/network/src/config.rs
+166 -18
View File
@@ -23,21 +23,26 @@
pub use crate::{
discovery::DEFAULT_KADEMLIA_REPLICATION_FACTOR,
peer_store::PeerStoreProvider,
protocol::{notification_service, NotificationsSink, ProtocolHandlePair},
request_responses::{
IncomingRequest, OutgoingResponse, ProtocolConfig as RequestResponseConfig,
},
service::traits::NotificationService,
service::{
metrics::NotificationMetrics,
traits::{NotificationConfig, NotificationService, PeerStore},
},
types::ProtocolName,
};
pub use libp2p::{
build_multiaddr,
identity::{self, ed25519, Keypair},
multiaddr, Multiaddr, PeerId,
multiaddr, Multiaddr,
};
use sc_network_types::PeerId;
use crate::peer_store::PeerStoreHandle;
use crate::service::{ensure_addresses_consistent_with_transport, traits::NetworkBackend};
use codec::Encode;
use prometheus_endpoint::Registry;
use zeroize::Zeroize;
@@ -61,6 +66,7 @@ use std::{
path::{Path, PathBuf},
pin::Pin,
str::{self, FromStr},
sync::Arc,
};
/// Protocol name prefix, transmitted on the wire for legacy protocol names.
@@ -99,7 +105,7 @@ impl fmt::Debug for ProtocolId {
/// let (peer_id, addr) = parse_str_addr(
/// "/ip4/198.51.100.19/tcp/30333/p2p/QmSk5HQbn6LhUwDiNMseVUjuRYhEtYj4aUZ6WfWoGURpdV"
/// ).unwrap();
/// assert_eq!(peer_id, "QmSk5HQbn6LhUwDiNMseVUjuRYhEtYj4aUZ6WfWoGURpdV".parse::<PeerId>().unwrap());
/// assert_eq!(peer_id, "QmSk5HQbn6LhUwDiNMseVUjuRYhEtYj4aUZ6WfWoGURpdV".parse::<PeerId>().unwrap().into());
/// assert_eq!(addr, "/ip4/198.51.100.19/tcp/30333".parse::<Multiaddr>().unwrap());
/// ```
pub fn parse_str_addr(addr_str: &str) -> Result<(PeerId, Multiaddr), ParseErr> {
@@ -569,6 +575,17 @@ impl NonDefaultSetConfig {
}
}
impl NotificationConfig for NonDefaultSetConfig {
fn set_config(&self) -> &SetConfig {
&self.set_config
}
/// Get reference to protocol name.
fn protocol_name(&self) -> &ProtocolName {
&self.protocol_name
}
}
/// Network service configuration.
#[derive(Clone, Debug)]
pub struct NetworkConfiguration {
@@ -655,6 +672,9 @@ pub struct NetworkConfiguration {
/// a modification of the way the implementation works. Different nodes with different
/// configured values remain compatible with each other.
pub yamux_window_size: Option<u32>,
/// Networking backend used for P2P communication.
pub network_backend: NetworkBackendType,
}
impl NetworkConfiguration {
@@ -687,6 +707,7 @@ impl NetworkConfiguration {
.expect("value is a constant; constant is non-zero; qed."),
yamux_window_size: None,
ipfs_server: false,
network_backend: NetworkBackendType::Libp2p,
}
}
@@ -722,18 +743,15 @@ impl NetworkConfiguration {
}
/// Network initialization parameters.
pub struct Params<Block: BlockT> {
pub struct Params<Block: BlockT, H: ExHashT, N: NetworkBackend<Block, H>> {
/// Assigned role for our node (full, light, ...).
pub role: Role,
/// How to spawn background tasks.
pub executor: Box<dyn Fn(Pin<Box<dyn Future<Output = ()> + Send>>) + Send>,
pub executor: Box<dyn Fn(Pin<Box<dyn Future<Output = ()> + Send>>) + Send + Sync>,
/// Network layer configuration.
pub network_config: FullNetworkConfiguration,
/// Peer store with known nodes, peer reputations, etc.
pub peer_store: PeerStoreHandle,
pub network_config: FullNetworkConfiguration<Block, H, N>,
/// Legacy name of the protocol to use on the wire. Should be different for each chain.
pub protocol_id: ProtocolId,
@@ -749,25 +767,43 @@ pub struct Params<Block: BlockT> {
pub metrics_registry: Option<Registry>,
/// Block announce protocol configuration
pub block_announce_config: NonDefaultSetConfig,
pub block_announce_config: N::NotificationProtocolConfig,
/// Bitswap configuration, if the server has been enabled.
pub bitswap_config: Option<N::BitswapConfig>,
/// Notification metrics.
pub notification_metrics: NotificationMetrics,
}
/// Full network configuration.
pub struct FullNetworkConfiguration {
pub struct FullNetworkConfiguration<B: BlockT + 'static, H: ExHashT, N: NetworkBackend<B, H>> {
/// Installed notification protocols.
pub(crate) notification_protocols: Vec<NonDefaultSetConfig>,
pub(crate) notification_protocols: Vec<N::NotificationProtocolConfig>,
/// List of request-response protocols that the node supports.
pub(crate) request_response_protocols: Vec<RequestResponseConfig>,
pub(crate) request_response_protocols: Vec<N::RequestResponseProtocolConfig>,
/// Network configuration.
pub network_config: NetworkConfiguration,
/// [`PeerStore`](crate::peer_store::PeerStore),
peer_store: Option<N::PeerStore>,
/// Handle to [`PeerStore`](crate::peer_store::PeerStore).
peer_store_handle: Arc<dyn PeerStoreProvider>,
}
impl FullNetworkConfiguration {
impl<B: BlockT + 'static, H: ExHashT, N: NetworkBackend<B, H>> FullNetworkConfiguration<B, H, N> {
/// Create new [`FullNetworkConfiguration`].
pub fn new(network_config: &NetworkConfiguration) -> Self {
let bootnodes = network_config.boot_nodes.iter().map(|bootnode| bootnode.peer_id).collect();
let peer_store = N::peer_store(bootnodes);
let peer_store_handle = peer_store.handle();
Self {
peer_store: Some(peer_store),
peer_store_handle,
notification_protocols: Vec::new(),
request_response_protocols: Vec::new(),
network_config: network_config.clone(),
@@ -775,19 +811,131 @@ impl FullNetworkConfiguration {
}
/// Add a notification protocol.
pub fn add_notification_protocol(&mut self, config: NonDefaultSetConfig) {
pub fn add_notification_protocol(&mut self, config: N::NotificationProtocolConfig) {
self.notification_protocols.push(config);
}
/// Get reference to installed notification protocols.
pub fn notification_protocols(&self) -> &Vec<NonDefaultSetConfig> {
pub fn notification_protocols(&self) -> &Vec<N::NotificationProtocolConfig> {
&self.notification_protocols
}
/// Add a request-response protocol.
pub fn add_request_response_protocol(&mut self, config: RequestResponseConfig) {
pub fn add_request_response_protocol(&mut self, config: N::RequestResponseProtocolConfig) {
self.request_response_protocols.push(config);
}
/// Get handle to [`PeerStore`].
pub fn peer_store_handle(&self) -> Arc<dyn PeerStoreProvider> {
Arc::clone(&self.peer_store_handle)
}
/// Take [`PeerStore`].
///
/// `PeerStore` is created when `FullNetworkConfig` is initialized so that `PeerStoreHandle`s
/// can be passed onto notification protocols. `PeerStore` itself should be started only once
/// and since technically it's not a libp2p task, it should be started with `SpawnHandle` in
/// `builder.rs` instead of using the libp2p/litep2p executor in the networking backend. This
/// function consumes `PeerStore` and starts its event loop in the appropriate place.
pub fn take_peer_store(&mut self) -> N::PeerStore {
self.peer_store
.take()
.expect("`PeerStore` can only be taken once when it's started; qed")
}
/// Verify addresses are consistent with enabled transports.
pub fn sanity_check_addresses(&self) -> Result<(), crate::error::Error> {
ensure_addresses_consistent_with_transport(
self.network_config.listen_addresses.iter(),
&self.network_config.transport,
)?;
ensure_addresses_consistent_with_transport(
self.network_config.boot_nodes.iter().map(|x| &x.multiaddr),
&self.network_config.transport,
)?;
ensure_addresses_consistent_with_transport(
self.network_config
.default_peers_set
.reserved_nodes
.iter()
.map(|x| &x.multiaddr),
&self.network_config.transport,
)?;
for notification_protocol in &self.notification_protocols {
ensure_addresses_consistent_with_transport(
notification_protocol.set_config().reserved_nodes.iter().map(|x| &x.multiaddr),
&self.network_config.transport,
)?;
}
ensure_addresses_consistent_with_transport(
self.network_config.public_addresses.iter(),
&self.network_config.transport,
)?;
Ok(())
}
/// Check for duplicate bootnodes.
pub fn sanity_check_bootnodes(&self) -> Result<(), crate::error::Error> {
self.network_config.boot_nodes.iter().try_for_each(|bootnode| {
if let Some(other) = self
.network_config
.boot_nodes
.iter()
.filter(|o| o.multiaddr == bootnode.multiaddr)
.find(|o| o.peer_id != bootnode.peer_id)
{
Err(crate::error::Error::DuplicateBootnode {
address: bootnode.multiaddr.clone(),
first_id: bootnode.peer_id.into(),
second_id: other.peer_id.into(),
})
} else {
Ok(())
}
})
}
/// Collect all reserved nodes and bootnodes addresses.
pub fn known_addresses(&self) -> Vec<(PeerId, Multiaddr)> {
let mut addresses: Vec<_> = self
.network_config
.default_peers_set
.reserved_nodes
.iter()
.map(|reserved| (reserved.peer_id, reserved.multiaddr.clone()))
.chain(self.notification_protocols.iter().flat_map(|protocol| {
protocol
.set_config()
.reserved_nodes
.iter()
.map(|reserved| (reserved.peer_id, reserved.multiaddr.clone()))
}))
.chain(
self.network_config
.boot_nodes
.iter()
.map(|bootnode| (bootnode.peer_id, bootnode.multiaddr.clone())),
)
.collect();
// Remove possible duplicates.
addresses.sort();
addresses.dedup();
addresses
}
}
/// Network backend type.
#[derive(Debug, Clone)]
pub enum NetworkBackendType {
/// Use libp2p for P2P networking.
Libp2p,
/// Use litep2p for P2P networking.
Litep2p,
}
#[cfg(test)]
substrate/client/network/src/error.rs
+3
View File
@@ -77,6 +77,9 @@ pub enum Error {
/// Connection closed.
#[error("Connection closed")]
ConnectionClosed,
/// Litep2p error.
#[error("Litep2p error: `{0}`")]
Litep2p(litep2p::Error),
}
// Make `Debug` use the `Display` implementation.
substrate/client/network/src/lib.rs
+8 -5
View File
@@ -243,6 +243,8 @@
//! More precise usage details are still being worked on and will likely change in the future.
mod behaviour;
mod bitswap;
mod litep2p;
mod protocol;
#[cfg(test)]
@@ -262,27 +264,28 @@ pub mod transport;
pub mod types;
pub mod utils;
pub use crate::litep2p::Litep2pNetworkBackend;
pub use event::{DhtEvent, Event};
#[doc(inline)]
pub use libp2p::{multiaddr, Multiaddr, PeerId};
pub use request_responses::{Config, IfDisconnected, RequestFailure};
pub use sc_network_common::{
role::{ObservedRole, Roles},
types::ReputationChange,
};
pub use service::{
metrics::NotificationMetrics,
signature::Signature,
traits::{
KademliaKey, MessageSink, NetworkBlock, NetworkDHTProvider, NetworkEventStream,
NetworkNotification, NetworkPeers, NetworkRequest, NetworkSigner, NetworkStateInfo,
NetworkStatus, NetworkStatusProvider, NetworkSyncForkRequest,
KademliaKey, MessageSink, NetworkBackend, NetworkBlock, NetworkDHTProvider,
NetworkEventStream, NetworkPeers, NetworkRequest, NetworkSigner, NetworkStateInfo,
NetworkStatus, NetworkStatusProvider, NetworkSyncForkRequest, NotificationConfig,
NotificationSender as NotificationSenderT, NotificationSenderError,
NotificationSenderReady, NotificationService,
},
DecodingError, Keypair, NetworkService, NetworkWorker, NotificationSender, OutboundFailure,
PublicKey,
};
pub use types::ProtocolName;
pub use types::{multiaddr, Multiaddr, PeerId, ProtocolName};
/// The maximum allowed number of established connections per peer.
///

Some files were not shown because too many files have changed in this diff Show More