* Move service tests to `client/network/tests`
These tests depend on `sc-network` and `sc-network-sync` so they should
live outside the crate.
* Move some configs from `sc-network-common` to `sc-network`
* Move `NetworkService` traits to `sc-network`
* Move request-responses to `sc-network`
* Remove more stuff
* Remove rest of configs from `sc-network-common` to `sc-network`
* Remove more stuff
* Fix warnings
* Update client/network/src/request_responses.rs
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
* Fix cargo doc
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
* Move import queue out of `sc-network`
Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.
This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.
* Move stuff to SyncingEngine
* Move `ChainSync` instanation to `SyncingEngine`
Some of the tests have to be rewritten
* Move peer hashmap to `SyncingEngine`
* Let `SyncingEngine` to implement `ChainSyncInterface`
* Introduce `SyncStatusProvider`
* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`
* Implement `SyncEventStream`
Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.
Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.
* Introduce `ChainSyncInterface`
This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.
* Move event stream polling to `SyncingEngine`
Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.
The code needs refactoring.
* Make `SyncingEngine` into an asynchronous runner
This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.
Code needs a lot of refactoring.
* Fix warnings
* Code refactoring
* Use `SyncingService` for BEEFY
* Use `SyncingService` for GRANDPA
* Remove call delegation from `NetworkService`
* Remove `ChainSyncService`
* Remove `ChainSync` service tests
They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.
* Refactor code
* Refactor code
* Update client/finality-grandpa/src/communication/tests.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Fix warnings
* Apply review comments
* Fix docs
* Fix test
* cargo-fmt
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Add missing docs
* Refactor code
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* Rename `*-private-ipv4` to `*-private-ip` CLI args
Renames the `*-private-ipv4` to `*-private-ip` in the CLI interface. The old names are staying as
alias, thus it will not break for anyone. Besides that it also fixes the naming in the rest of the code.
* FMT
* Move import queue out of `sc-network`
Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.
This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.
* Fix tests
* Apply review comments
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update client/network/sync/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Bastian Köcher <git@kchr.de>
* Replace deprecated libp2p feature specs with correct ones
* Bump tokio to 1.21.2
* Replace async-std libp2p primitives with tokio ones
* minor: rustfmt
* Fix TestNet to run initialization in the tokio context
* Convert telemetry test from async-std to tokio
* Convert notifications tests from async-std to tokio
* Convert chain sync tests from async-std to tokio
* Ditch async-std completely
* Make executor mandatory
* Bump tokio to 1.22.0
* minor: rustfmt
* Explicitly use tokio runtime in tests
* Move more tests to explicit tokio runtime
* Explicitly set multithreaded runtime in tokio test
* minor: rustfmt
* minor: fix comment
* Replace async-std with tokio in MMR tests
* cargo upgrade libp2p
* Get rid of `NetworkBehaviourEventProcess` in handling of `CustomMessageOutcome`
* Get rid of `NetworkBehaviourEventProcess` in handling of `request_responses::Event`
* Get rid of `NetworkBehaviourEventProcess` in handling of `peer_info::PeerInfoEvent`
* Get rid of `NetworkBehaviourEventProcess` in handling of `DiscoveryOut`
* Get rid of `poll()` method in `Bahaviour`
* minor: comments
* Upgrade libp2p to 0.49.0 (unreleased)
* Support multiple Kad protocol names
* Make borrow checker happy
* minor: wording
* Make substrate build with libp2p-0.49.0
* rustfmt
* Get rid of MdnsWrapper
* Resolve deprecation warnings
* Fix documentation
* Apply suggestions from code review: fix typos
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
* Apply suggestion: simplify kad protocol name matching
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
* Introduce `ChainSyncInterface`
`ChainSyncInterface` provides an asynchronous interface for other
subsystems to submit calls to `ChainSync`. This allows `NetworkService`
to delegate calls to `ChainSync` while still providing the same API
for other subsystems (for now). This makes it possible to move the
syncing code in piecemeal fashion out of `protocol.rs` as the calls
are just forwarded to `ChainSync`.
* Apply review comments
* Fix tests
* Use `array-bytes` for All Array/Bytes/Hex Operations
Signed-off-by: Xavier Lau <xavier@inv.cafe>
* Reorder
* Self Review
* Format
* Fix Tests
* Bump `array-bytes`
* Optimize large test res
Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
* Introduce `sc-network-bitswap`
Move the bitswap protocol out of `sc-network` to its own crate.
* Improve test coverage
* Remove mention of bitswap from `sc-network`
* Fix documentation
* Fix clippy and remove the void dependency
* Remove unneeded trait bound and bump prost to 0.11
* Add ProtocolName custom type
* Use new ProtocolName in sc_network_common
* Use new ProtocolName in sc_network
* Use new ProtocolName for BEEFY and GRANDPA
* Use new ProtocolName for notifications
* Use new ProtocolName in sc_network (part 2)
* Use new ProtocolName in sc_network_gossip
* Use new ProtocolName in sc_offchain
* Remove unused imports
* Some more fixes
* Add tests
* Fix minor import issues
* Re-export ProtocolName in sc_network
* Revert "Re-export ProtocolName in sc_network"
This reverts commit 8d8ff71927e7750757f29c9bbd88dc0ba181d214.
* Re-export ProtocolName in sc_network
* Remove dependency on sc-network-common from beefy-gadget
* Move `MultiaddrWithPeerId` and related parsing functions into `sc-network-common`, remove dependency on `sc-network` from `sc-chain-spec`
* Remove dependency on `sc-network` from `sc-offchain`
* Remove dependency on `sc-network` from `sc-network-gossip`
* Rename transactions protocol to include genesis hash
* Add protocol name generation to sc_network::utils
* Use utils functions for transactions protocol name generation
* Extract protocol name generation into public module
* Use sc_network::protocol_name::standard_protocol_name() for BEEFY and GRANDPA
* minor: add missing newline at EOF
* Change block-announces protocol name to include genesis_hash & fork_id
* Change protocol names to include genesis hash and fork id
Protocols changed:
- sync
- state
- light
- sync/warp
* Revert "Use sc_network::protocol_name::standard_protocol_name() for BEEFY and GRANDPA"
This reverts commit cd60a95a3face397e1b67f4bc95dd0f2b581bfae.
* Get rid of `protocol_name` module
* Remove direct dependency of `sc-network` on `sc-network-light`
* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`
* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`
* Create `sync` module in `sc-network-common`, create `ChainSync` trait there (not used yet), move a bunch of associated data structures from `sc-network-sync`
* Switch from concrete implementation to `ChainSync` trait from `sc-network-common`
* Introduce `OpaqueStateRequest`/`OpaqueStateResponse` to remove generics from `StateSync` trait
* Introduce `OpaqueBlockRequest`/`OpaqueBlockResponse`, make `scheme` module of `sc-network-sync` private
* Surface `sc-network-sync` into `sc-service` and make `sc-network` not depend on it anymore
* Remove now unnecessary dependency from `sc-network`
* Replace crate links with just text since dependencies are gone now
* Remove `warp_sync` re-export from `sc-network-common`
* Update copyright in network-related files
* Address review comments about documentation
* Apply review suggestion
* Rename `extra_requests` module to `metrics`
Co-authored-by: Bastian Köcher <info@kchr.de>
* Move `api.v1.proto` schema into new crate `sc-network-sync`
* Move `sc_network::protocol::sync::state` module into `sc_network_sync::state`
* Move `sc_network::protocol::sync::blocks` module into `sc_network_sync::blocks` and some data structures from `sc_network::protocol::message` module into `sc_network_sync::message`
* Move some data structures from `sc_network::config` and `sc_network::request_responses` into new `sc-network-common` crate
* Move `sc_network::protocol::sync::warm` and `sc_network::warp_request_handler` modules into `sc_network_sync`
* Move `client/network/sync/src/lib.rs` to `client/network/sync/src/lib_old.rs` to preserve history of changes of the file in the next commit
* Move `client/network/src/protocol/sync.rs` on top of `client/network/sync/src/lib.rs` to preserve history of changes
* Move `sc_network::protocol::sync` to `sc_network_sync` with submodules, move message data structures around accordingly
* Move `sc_network::block_request_handler` to `sc_network_sync::block_request_handler`
* Move `sc_network::state_request_handler` to `sc_network_sync::state_request_handler`
* Add re-exports for compatibility reasons
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Remove unnecessary imports, move one internal re-export into where it is actually used, make one import explicit
* Move a few data structures down into modules
* Use generic parameters in `sc-network` instead of `chain::Client` trait
* Remove unnecessary bound
* sc-cli: Fix bugs after switching to clap3
Before switching to clap3 we support cli options like `--reserved-nodes A B` and after you needed to
pass `--reserved-nodes` cli option multiple times `--reserved-nodes A --reserved-nodes B`. This is
fixed by setting `multiple_occurrences(true)` option. This also done for all the other `Vec` cli
options in `sc-cli`. Besides that `--sync` wasn't supporting case insensitive parsing of the value.
This is now also supported. For both regressions a test is added. Besides that the pr removes all
the `rename_all = PascalCase` attributes, because they are not needed. All other `ArgEnum`s were
checked and all are already using `ignore_case(true)`.
* Bring back `PascalCase`, because otherwise it falls back to `kebab-case`...
* grandpa: update notif protocol name
* grandpa: add chain id prefix to protocol name
* grandpa: beautify protocol name handling
* grandpa: prepend genesis hash to protocol name
* chain-spec: add optional 'fork_id'
'fork_id' is used to uniquely identify forks of the same chain/network
'ChainSpec' trait provides default 'None' implementation, meaning this
chain hasn't been forked.
* grandpa: protocol_name mod instead of struct
* beefy: add genesis hash prefix to protocol name
* chainspec: add fork_id
* grandpa: simplify protocol name
* grandpa: contain protocol name building logic
* beefy: contain protocol name building logic
* grandpa: fix tests
* fix merge damage
* fix docs reference visibility
Signed-off-by: acatangiu <adrian@parity.io>
* Update client/finality-grandpa/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update client/finality-grandpa/src/communication/mod.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update client/beefy/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update client/beefy/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* avoid using hash default, even for protocol names
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Clean up sc-network
- Avoid using clone() for the Copy type `PeerId`.
- Use `find_map` for `filter_map` and `next`.
- Use `Self`.
* More on Copy types
* Cargo +nightly fmt --all
* More ..
* fmt
* Revert vec![default_notif_handshake_message]
* Comment out browser stuff
* Remove browser stuff
* Remove more wasm transport code
* Remove ExtTransport and rework how telemetry initialises.
* Change (most) wasm-timer using code to use std::time
* Rename CI-job
* Aura does not compile for wasm
* Remove testing in the browser on CI
* Update README
* Leave `StreamSink` be
* fmt
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* network: allow gossiping to light clients
* grandpa: gossip global messages to light clients
* grandpa: don't send neighbor packets to light clients
* grandpa: fix tests
* grandpa: export run_grandpa_observer
* node: run grandpa observer on light client
* node: start network at end
* Use wasm_timer in finality-grandpa
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Allow fallback names for protocols
* Apply suggestions from code review
Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
* Fix some issues
* Fix compilation after merging master
Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
Sebastian Kunert