mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-04-25 19:57:56 +00:00

Files

T

Aaro Altonen 80616f6d03 Integrate litep2p into Polkadot SDK (#2944 )

[litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P
networking library. It supports all of the features of `rust-libp2p`
that are currently being utilized by Polkadot SDK.

Compared to `rust-libp2p`, `litep2p` has a quite different architecture
which is why the new `litep2p` network backend is only able to use a
little of the existing code in `sc-network`. The design has been mainly
influenced by how we'd wish to structure our networking-related code in
Polkadot SDK: independent higher-levels protocols directly communicating
with the network over links that support bidirectional backpressure. A
good example would be `NotificationHandle`/`RequestResponseHandle`
abstractions which allow, e.g., `SyncingEngine` to directly communicate
with peers to announce/request blocks.

I've tried running `polkadot --network-backend litep2p` with a few
different peer configurations and there is a noticeable reduction in
networking CPU usage. For high load (`--out-peers 200`), networking CPU
usage goes down from ~110% to ~30% (80 pp) and for normal load
(`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp).

These should not be taken as final numbers because:

a) there are still some low-hanging optimization fruits, such as
enabling [receive window
auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating
`Peerset` more closely with `litep2p` or improving memory usage of the
WebSocket transport
b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less
work will increase the networking CPU usage
c) verification in a more diverse set of tests/conditions is needed

Nevertheless, these numbers should give an early estimate for CPU usage
of the new networking backend.

This PR consists of three separate changes:
* introduce a generic `PeerId` (wrapper around `Multihash`) so that we
don't have use `NetworkService::PeerId` in every part of the code that
uses a `PeerId`
* introduce `NetworkBackend` trait, implement it for the libp2p network
stack and make Polkadot SDK generic over `NetworkBackend`
  * implement `NetworkBackend` for litep2p

The new library should be considered experimental which is why
`rust-libp2p` will remain as the default option for the time being. This
PR currently depends on the master branch of `litep2p` but I'll cut a
new release for the library once all review comments have been
addresses.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>

2024-04-08 16:44:13 +00:00

src

Integrate litep2p into Polkadot SDK (#2944 )

2024-04-08 16:44:13 +00:00

Cargo.toml

Integrate litep2p into Polkadot SDK (#2944 )

2024-04-08 16:44:13 +00:00

README.md

Markdown linter (#1309 )

2023-09-04 12:02:32 +03:00

README.md

Polite gossiping.

This crate provides gossiping capabilities on top of a network.

Gossip messages are separated by two categories: "topics" and consensus engine ID. The consensus engine ID is sent over the wire with the message, while the topic is not, with the expectation that the topic can be derived implicitly from the content of the message, assuming it is valid.

Topics are a single 32-byte tag associated with a message, used to group those messages in an opaque way. Consensus code can invoke broadcast_topic to attempt to send all messages under a single topic to all peers who don't have them yet, and send_topic to send all messages under a single topic to a specific peer.

Usage

Implement the Network trait, representing the low-level networking primitives. It is already implemented on sc_network::NetworkService.
Implement the Validator trait. See the section below.
Decide on a ConsensusEngineId. Each gossiping protocol should have a different one.
Build a GossipEngine using these three elements.
Use the methods of the GossipEngine in order to send out messages and receive incoming messages.

What is a validator?

The primary role of a Validator is to process incoming messages from peers, and decide whether to discard them or process them. It also decides whether to re-broadcast the message.

The secondary role of the Validator is to check if a message is allowed to be sent to a given peer. All messages, before being sent, will be checked against this filter. This enables the validator to use information it's aware of about connected peers to decide whether to send messages to them at any given moment in time - In particular, to wait until peers can accept and process the message before sending it.

Lastly, the fact that gossip validators can decide not to rebroadcast messages opens the door for neighbor status packets to be baked into the gossip protocol. These status packets will typically contain light pieces of information used to inform peers of a current view of protocol state.

License: GPL-3.0-or-later WITH Classpath-exception-2.0