Aaro Altonen
80616f6d03
[litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P networking library. It supports all of the features of `rust-libp2p` that are currently being utilized by Polkadot SDK. Compared to `rust-libp2p`, `litep2p` has a quite different architecture which is why the new `litep2p` network backend is only able to use a little of the existing code in `sc-network`. The design has been mainly influenced by how we'd wish to structure our networking-related code in Polkadot SDK: independent higher-levels protocols directly communicating with the network over links that support bidirectional backpressure. A good example would be `NotificationHandle`/`RequestResponseHandle` abstractions which allow, e.g., `SyncingEngine` to directly communicate with peers to announce/request blocks. I've tried running `polkadot --network-backend litep2p` with a few different peer configurations and there is a noticeable reduction in networking CPU usage. For high load (`--out-peers 200`), networking CPU usage goes down from ~110% to ~30% (80 pp) and for normal load (`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp). These should not be taken as final numbers because: a) there are still some low-hanging optimization fruits, such as enabling [receive window auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating `Peerset` more closely with `litep2p` or improving memory usage of the WebSocket transport b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less work will increase the networking CPU usage c) verification in a more diverse set of tests/conditions is needed Nevertheless, these numbers should give an early estimate for CPU usage of the new networking backend. This PR consists of three separate changes: * introduce a generic `PeerId` (wrapper around `Multihash`) so that we don't have use `NetworkService::PeerId` in every part of the code that uses a `PeerId` * introduce `NetworkBackend` trait, implement it for the libp2p network stack and make Polkadot SDK generic over `NetworkBackend` * implement `NetworkBackend` for litep2p The new library should be considered experimental which is why `rust-libp2p` will remain as the default option for the time being. This PR currently depends on the master branch of `litep2p` but I'll cut a new release for the library once all review comments have been addresses. --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
NOTE: We have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into this repository. Read more about the changes here.
Polkadot SDK
The Polkadot SDK repository provides all the resources needed to start building on the Polkadot network, a multi-chain blockchain platform that enables different blockchains to interoperate and share information in a secure and scalable way. The Polkadot SDK comprises three main pieces of software:
Polkadot
Implementation of a node for the https://polkadot.network in Rust, using the Substrate framework. This directory
currently contains runtimes for the Westend and Rococo test networks. Polkadot, Kusama and their system chain runtimes
are located in the runtimes repository maintained by
the Polkadot Technical Fellowship.
Substrate
Substrate is the primary blockchain SDK used by developers to create the parachains that make up the Polkadot network. Additionally, it allows for the development of self-sovereign blockchains that operate completely independently of Polkadot.
Cumulus
Cumulus is a set of tools for writing Substrate-based Polkadot parachains.
Releases
Note
Our release process is still Work-In-Progress and may not yet reflect the aspired outline here.
The Polkadot-SDK has two release channels: stable and nightly. Production software is advised to only use stable.
nightly is meant for tinkerers to try out the latest features. The detailed release process is described in
RELEASE.md.
Stable
stable releases have a support duration of three months. In this period, the release will not have any breaking
changes. It will receive bug fixes, security fixes, performance fixes and new non-breaking features on a two week
cadence.
Nightly
nightly releases are released every night from the master branch, potentially with breaking changes. They have
pre-release version numbers in the format major.0.0-nightlyYYMMDD.
Upstream Dependencies
Below are the primary upstream dependencies utilized in this project:
Security
The security policy and procedures can be found in docs/contributor/SECURITY.md.
Contributing & Code of Conduct
Ensure you follow our contribution guidelines. In every interaction and contribution, this project adheres to the Contributor Covenant Code of Conduct.
Additional Resources
- For monitoring upcoming changes and current proposals related to the technical implementation of the Polkadot network,
visit the
Requests for Comment (RFC)repository. While it's maintained by the Polkadot Fellowship, the RFC process welcomes contributions from everyone.