Aaro Altonen
80616f6d03
[litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P networking library. It supports all of the features of `rust-libp2p` that are currently being utilized by Polkadot SDK. Compared to `rust-libp2p`, `litep2p` has a quite different architecture which is why the new `litep2p` network backend is only able to use a little of the existing code in `sc-network`. The design has been mainly influenced by how we'd wish to structure our networking-related code in Polkadot SDK: independent higher-levels protocols directly communicating with the network over links that support bidirectional backpressure. A good example would be `NotificationHandle`/`RequestResponseHandle` abstractions which allow, e.g., `SyncingEngine` to directly communicate with peers to announce/request blocks. I've tried running `polkadot --network-backend litep2p` with a few different peer configurations and there is a noticeable reduction in networking CPU usage. For high load (`--out-peers 200`), networking CPU usage goes down from ~110% to ~30% (80 pp) and for normal load (`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp). These should not be taken as final numbers because: a) there are still some low-hanging optimization fruits, such as enabling [receive window auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating `Peerset` more closely with `litep2p` or improving memory usage of the WebSocket transport b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less work will increase the networking CPU usage c) verification in a more diverse set of tests/conditions is needed Nevertheless, these numbers should give an early estimate for CPU usage of the new networking backend. This PR consists of three separate changes: * introduce a generic `PeerId` (wrapper around `Multihash`) so that we don't have use `NetworkService::PeerId` in every part of the code that uses a `PeerId` * introduce `NetworkBackend` trait, implement it for the libp2p network stack and make Polkadot SDK generic over `NetworkBackend` * implement `NetworkBackend` for litep2p The new library should be considered experimental which is why `rust-libp2p` will remain as the default option for the time being. This PR currently depends on the master branch of `litep2p` but I'll cut a new release for the library once all review comments have been addresses. --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
Integration of the GRANDPA finality gadget into Substrate.
This crate is unstable and the API and usage may change.
This crate provides a long-running future that produces finality notifications.
Usage
First, create a block-import wrapper with the block_import function. The
GRANDPA worker needs to be linked together with this block import object, so
a LinkHalf is returned as well. All blocks imported (from network or
consensus or otherwise) must pass through this wrapper, otherwise consensus
is likely to break in unexpected ways.
Next, use the LinkHalf and a local configuration to run_grandpa_voter.
This requires a Network implementation. The returned future should be
driven to completion and will finalize blocks in the background.
Changing authority sets
The rough idea behind changing authority sets in GRANDPA is that at some point, we obtain agreement for some maximum block height that the current set can finalize, and once a block with that height is finalized the next set will pick up finalization from there.
Technically speaking, this would be implemented as a voting rule which says, "if there is a signal for a change in N blocks in block B, only vote on chains with length NUM(B) + N if they contain B". This conditional-inclusion logic is complex to compute because it requires looking arbitrarily far back in the chain.
Instead, we keep track of a list of all signals we've seen so far (across all forks), sorted ascending by the block number they would be applied at. We never vote on chains with number higher than the earliest handoff block number (this is num(signal) + N). When finalizing a block, we either apply or prune any signaled changes based on whether the signaling block is included in the newly-finalized chain.
License: GPL-3.0-or-later WITH Classpath-exception-2.0