Introduce a middleware called `NetworkServiceProvider` which the
`ChainSync` can use to communicate with `NetworkService`. `ChainSync` is
given a `NetworkServiceHandle` which it uses to call `NetworkServiceProvider`
which then dispatches the calls to `NetworkService` on behalf of `ChainSync`.
This change will allow `ChainSync` to disconnect and report peers and
in the future it'll be possible to send requests and notifications
through the `NetworkServiceProvider`.
`NetworkServiceProvider` is needed only until the `ChainSync` object
has been removed from `Protocol`. After that, a normal `NetworkService`
handle can be passed onto `ChainSync` and these changes can be
deprecated.
Co-authored-by: parity-processbot <>
* Refactor service tests in `sc-network`
Create a separate directory for the tests and move common network
instantion related code to `mod.rs` from where it can be used by both
service and chainsync tests.
Use the builder pattern when creating the `TestNetwork` object to reduce
code duplication between the test files.
* Update client/network/src/service/tests/mod.rs
Co-authored-by: Dmitrii Markin <dmitry@markin.tech>
Co-authored-by: Dmitrii Markin <dmitry@markin.tech>
Co-authored-by: parity-processbot <>
* cargo upgrade libp2p
* Get rid of `NetworkBehaviourEventProcess` in handling of `CustomMessageOutcome`
* Get rid of `NetworkBehaviourEventProcess` in handling of `request_responses::Event`
* Get rid of `NetworkBehaviourEventProcess` in handling of `peer_info::PeerInfoEvent`
* Get rid of `NetworkBehaviourEventProcess` in handling of `DiscoveryOut`
* Get rid of `poll()` method in `Bahaviour`
* minor: comments
* Upgrade libp2p to 0.49.0 (unreleased)
* Support multiple Kad protocol names
* Make borrow checker happy
* minor: wording
* Make substrate build with libp2p-0.49.0
* rustfmt
* Get rid of MdnsWrapper
* Resolve deprecation warnings
* Fix documentation
* Apply suggestions from code review: fix typos
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
* Apply suggestion: simplify kad protocol name matching
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
* Introduce `ChainSyncInterface`
`ChainSyncInterface` provides an asynchronous interface for other
subsystems to submit calls to `ChainSync`. This allows `NetworkService`
to delegate calls to `ChainSync` while still providing the same API
for other subsystems (for now). This makes it possible to move the
syncing code in piecemeal fashion out of `protocol.rs` as the calls
are just forwarded to `ChainSync`.
* Apply review comments
* Fix tests
* Introduce mockable `ChainSync` object for testing
`mockall` allows to mock `ChainSync` and to verify that the calls made
to `ChaiSync` are firstly executed at all, that they're executed in
correct order and with correct parameters.
This allows to verify, e.g., that delegating calls directly to
`ChainSync` from `NetworkService` still calls the correct functions with
correct arguments even if `Protocol` middleman is removed.
* Add Cargo.lock
* Fix tests
* Update client/network/Cargo.toml
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update Cargo.lock
* Fix clippy and documentation
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: parity-processbot <>
Sometimes `NotificationStreamOpenened` would be received for the
other protocol before `SyncConnected` was received so when the
connection was closed, an incorrect event was read from the event
stream.
* Use `array-bytes` for All Array/Bytes/Hex Operations
Signed-off-by: Xavier Lau <xavier@inv.cafe>
* Reorder
* Self Review
* Format
* Fix Tests
* Bump `array-bytes`
* Optimize large test res
Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
* BREAKING: Rename Origin
* more renaming
* a bit more renaming
* fix
* more fixing
* fix in frame_support
* even more fixes
* fix
* small fix
* ...
* update .stderr
* docs
* update docs
* update docs
* docs
* Receive and import target block body
* Request target block
* minor: wording
* Check for block body in the test
* Import target block justifications
* Fix: do not fail block validation if no justifications received
* Fix: import target blocks without justifications
Co-authored-by: arkpar <arkady.paronyan@gmail.com>
* Introduce `sc-network-bitswap`
Move the bitswap protocol out of `sc-network` to its own crate.
* Improve test coverage
* Remove mention of bitswap from `sc-network`
* Fix documentation
* Fix clippy and remove the void dependency
* Remove unneeded trait bound and bump prost to 0.11
* Add ProtocolName custom type
* Use new ProtocolName in sc_network_common
* Use new ProtocolName in sc_network
* Use new ProtocolName for BEEFY and GRANDPA
* Use new ProtocolName for notifications
* Use new ProtocolName in sc_network (part 2)
* Use new ProtocolName in sc_network_gossip
* Use new ProtocolName in sc_offchain
* Remove unused imports
* Some more fixes
* Add tests
* Fix minor import issues
* Re-export ProtocolName in sc_network
* Revert "Re-export ProtocolName in sc_network"
This reverts commit 8d8ff71927e7750757f29c9bbd88dc0ba181d214.
* Re-export ProtocolName in sc_network
* Remove dependency on sc-network-common from beefy-gadget
The logging before is confusing as it says Propagating but it's not in fact when gossip_enabled is
false. Now it's also consistent with `propagate_transactions` below.
* Move `MultiaddrWithPeerId` and related parsing functions into `sc-network-common`, remove dependency on `sc-network` from `sc-chain-spec`
* Remove dependency on `sc-network` from `sc-offchain`
* Remove dependency on `sc-network` from `sc-network-gossip`
* Extract `NetworkKVProvider` trait in `sc-authority-discovery` and remove unnecessary dependency
* Extract `NetworkSyncForkRequest` trait in `sc-finality-grandpa`
* Relax requirements on `SyncOracle` trait, remove extra native methods from `NetworkService` that are already provided by trait impls
* Move `NetworkSigner` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
* Move `NetworkKVProvider` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
* Minimize `sc-authority-discovery` dependency on `sc-network`
* Move `NetworkSyncForkRequest` trait from `sc-finality-grandpa` to `sc-network-common` and de-duplicate methods in `NetworkService`
* Extract `NetworkStatusProvider` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkPeers` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkEventStream` trait and de-duplicate methods on `NetworkService`
* Move more methods from `NetworkService` into `NetworkPeers` trait
* Move `NetworkStateInfo` trait into `sc-network-common`
* Extract `NetworkNotification` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkRequest` trait and de-duplicate methods on `NetworkService`
* Remove `NetworkService::local_peer_id()`, it is already provided by `NetworkStateInfo` impl
* Extract `NetworkTransaction` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkBlock` trait and de-duplicate methods on `NetworkService`
* Remove dependencies on `NetworkService` from most of the methods of `sc-service`
* Address simple review comments
* Rename transactions protocol to include genesis hash
* Add protocol name generation to sc_network::utils
* Use utils functions for transactions protocol name generation
* Extract protocol name generation into public module
* Use sc_network::protocol_name::standard_protocol_name() for BEEFY and GRANDPA
* minor: add missing newline at EOF
* Change block-announces protocol name to include genesis_hash & fork_id
* Change protocol names to include genesis hash and fork id
Protocols changed:
- sync
- state
- light
- sync/warp
* Revert "Use sc_network::protocol_name::standard_protocol_name() for BEEFY and GRANDPA"
This reverts commit cd60a95a3face397e1b67f4bc95dd0f2b581bfae.
* Get rid of `protocol_name` module
* Remove direct dependency of `sc-network` on `sc-network-light`
* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`
* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`
* Create `sync` module in `sc-network-common`, create `ChainSync` trait there (not used yet), move a bunch of associated data structures from `sc-network-sync`
* Switch from concrete implementation to `ChainSync` trait from `sc-network-common`
* Introduce `OpaqueStateRequest`/`OpaqueStateResponse` to remove generics from `StateSync` trait
* Introduce `OpaqueBlockRequest`/`OpaqueBlockResponse`, make `scheme` module of `sc-network-sync` private
* Surface `sc-network-sync` into `sc-service` and make `sc-network` not depend on it anymore
* Remove now unnecessary dependency from `sc-network`
* Replace crate links with just text since dependencies are gone now
* Remove `warp_sync` re-export from `sc-network-common`
* Update copyright in network-related files
* Address review comments about documentation
* Apply review suggestion
* Rename `extra_requests` module to `metrics`
Co-authored-by: Bastian Köcher <info@kchr.de>
* Upgrade to libp2p 0.45.1
* Limit max_negotiating_inbound_streams to 512
* Upgrade prost-build to 0.10
* Set max_negotiating_inbound_streams to 2048
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Fix authority discovery protobuf
* Fix comments in authority-discovery schema
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Add a comment about transport initialization
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
When passing reserved nodes only with a peer id it was failing with the `DuplicateBootnode` error.
Besides that there are some clean ups. We for example added the bootnodes twice to the `known_addresses`.
* Move `api.v1.proto` schema into new crate `sc-network-sync`
* Move `sc_network::protocol::sync::state` module into `sc_network_sync::state`
* Move `sc_network::protocol::sync::blocks` module into `sc_network_sync::blocks` and some data structures from `sc_network::protocol::message` module into `sc_network_sync::message`
* Move some data structures from `sc_network::config` and `sc_network::request_responses` into new `sc-network-common` crate
* Move `sc_network::protocol::sync::warm` and `sc_network::warp_request_handler` modules into `sc_network_sync`
* Move `client/network/sync/src/lib.rs` to `client/network/sync/src/lib_old.rs` to preserve history of changes of the file in the next commit
* Move `client/network/src/protocol/sync.rs` on top of `client/network/sync/src/lib.rs` to preserve history of changes
* Move `sc_network::protocol::sync` to `sc_network_sync` with submodules, move message data structures around accordingly
* Move `sc_network::block_request_handler` to `sc_network_sync::block_request_handler`
* Move `sc_network::state_request_handler` to `sc_network_sync::state_request_handler`
* Add re-exports for compatibility reasons
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Remove unnecessary imports, move one internal re-export into where it is actually used, make one import explicit
* Move a few data structures down into modules
* Use generic parameters in `sc-network` instead of `chain::Client` trait
* Remove unnecessary bound
* Prepare for rust stable 1.59
Besides preparing the UI tests this also adds a new script update-rust-stable.sh script for
simplifying the update of a rust stable version. This script will run all UI tests for the new
rust stable version and updating the expected output.
* Ensure we run the UI tests in CI
* use staging ci image
* More test updates
* Unignore test (#11097)
* empty commit for pipeline rerun
* empty commit for pipeline rerun
* Try to make clippy happy
* More clippy fixes
* FMT
* ci image production
Co-authored-by: alvicsam <alvicsam@gmail.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
* sc-cli: Fix bugs after switching to clap3
Before switching to clap3 we support cli options like `--reserved-nodes A B` and after you needed to
pass `--reserved-nodes` cli option multiple times `--reserved-nodes A --reserved-nodes B`. This is
fixed by setting `multiple_occurrences(true)` option. This also done for all the other `Vec` cli
options in `sc-cli`. Besides that `--sync` wasn't supporting case insensitive parsing of the value.
This is now also supported. For both regressions a test is added. Besides that the pr removes all
the `rename_all = PascalCase` attributes, because they are not needed. All other `ArgEnum`s were
checked and all are already using `ignore_case(true)`.
* Bring back `PascalCase`, because otherwise it falls back to `kebab-case`...
* Upgraded dependencies
* Adapting code to scale v3
* Empty commit to trigger CI
* Triggering CI
* Fixing UI test
* Remove superfluous dev-dep added by #9228
* Cryout for CI
* Simplify `num_connected_peers`
* Track requested peer counts
* Revert "Track requested peer counts"
This reverts commit 9f1c8704353df6afc17ed7e9f4ab8d8e29466ae4.
* Remove `substrate_sub_libp2p_peerset_num_requested` metric
* Remove two unused functions that I forgot to get rid of in previous commit
Dmitrii Markin