* Move import queue out of `sc-network`
Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.
This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.
* Move stuff to SyncingEngine
* Move `ChainSync` instanation to `SyncingEngine`
Some of the tests have to be rewritten
* Move peer hashmap to `SyncingEngine`
* Let `SyncingEngine` to implement `ChainSyncInterface`
* Introduce `SyncStatusProvider`
* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`
* Implement `SyncEventStream`
Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.
Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.
* Introduce `ChainSyncInterface`
This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.
* Move event stream polling to `SyncingEngine`
Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.
The code needs refactoring.
* Make `SyncingEngine` into an asynchronous runner
This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.
Code needs a lot of refactoring.
* Fix warnings
* Code refactoring
* Use `SyncingService` for BEEFY
* Use `SyncingService` for GRANDPA
* Remove call delegation from `NetworkService`
* Remove `ChainSyncService`
* Remove `ChainSync` service tests
They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.
* Refactor code
* Refactor code
* Update client/finality-grandpa/src/communication/tests.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Fix warnings
* Apply review comments
* Fix docs
* Fix test
* cargo-fmt
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Add missing docs
* Refactor code
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* Add ProtocolName custom type
* Use new ProtocolName in sc_network_common
* Use new ProtocolName in sc_network
* Use new ProtocolName for BEEFY and GRANDPA
* Use new ProtocolName for notifications
* Use new ProtocolName in sc_network (part 2)
* Use new ProtocolName in sc_network_gossip
* Use new ProtocolName in sc_offchain
* Remove unused imports
* Some more fixes
* Add tests
* Fix minor import issues
* Re-export ProtocolName in sc_network
* Revert "Re-export ProtocolName in sc_network"
This reverts commit 8d8ff71927e7750757f29c9bbd88dc0ba181d214.
* Re-export ProtocolName in sc_network
* Remove dependency on sc-network-common from beefy-gadget
* Move `MultiaddrWithPeerId` and related parsing functions into `sc-network-common`, remove dependency on `sc-network` from `sc-chain-spec`
* Remove dependency on `sc-network` from `sc-offchain`
* Remove dependency on `sc-network` from `sc-network-gossip`
* Extract `NetworkKVProvider` trait in `sc-authority-discovery` and remove unnecessary dependency
* Extract `NetworkSyncForkRequest` trait in `sc-finality-grandpa`
* Relax requirements on `SyncOracle` trait, remove extra native methods from `NetworkService` that are already provided by trait impls
* Move `NetworkSigner` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
* Move `NetworkKVProvider` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
* Minimize `sc-authority-discovery` dependency on `sc-network`
* Move `NetworkSyncForkRequest` trait from `sc-finality-grandpa` to `sc-network-common` and de-duplicate methods in `NetworkService`
* Extract `NetworkStatusProvider` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkPeers` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkEventStream` trait and de-duplicate methods on `NetworkService`
* Move more methods from `NetworkService` into `NetworkPeers` trait
* Move `NetworkStateInfo` trait into `sc-network-common`
* Extract `NetworkNotification` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkRequest` trait and de-duplicate methods on `NetworkService`
* Remove `NetworkService::local_peer_id()`, it is already provided by `NetworkStateInfo` impl
* Extract `NetworkTransaction` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkBlock` trait and de-duplicate methods on `NetworkService`
* Remove dependencies on `NetworkService` from most of the methods of `sc-service`
* Address simple review comments
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Sync: Propagate block announcement data
This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.
* Review comment
* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked
* Also use out_peers
* Rework priority groups
* Broken tests fix
* Fix warning causing CI to fail
* [Hack] Try restore backwards-compatibility
* Fix peerset bug
* Doc fixes and clean up
* Error on state mismatch
* Try debug CI
* CI debugging
* [CI debug] Can I please see this line
* Revert "[CI debug] Can I please see this line"
This reverts commit 4b7cf7c1511f579cd818b21d46bd11642dfac5cb.
* Revert "CI debugging"
This reverts commit 9011f1f564b860386dc7dd6ffa9fc34ea7107623.
* Fix error! which isn't actually an error
* Fix Ok() returned when actually Err()
* Tweaks and fixes
* Fix build
* Peerset bugfix
* [Debug] Try outbound GrandPa slots
* Another bugfix
* Revert "[Debug] Try outbound GrandPa slots"
This reverts commit d175b9208c088faad77d9f0ce36ff6f48bd92dd3.
* [Debug] Try outbound GrandPa slots
* Apply suggestions from code review
Co-authored-by: Max Inden <mail@max-inden.de>
* Use consts for hardcoded peersets
* Revert "Try debug CI"
This reverts commit 62c4ad5e79c03d561c714a008022ecac463a597e.
* Renames
* Line widths
* Add doc
Co-authored-by: Max Inden <mail@max-inden.de>
* Remove sc_network::NetworkService::register_notifications_protocol
* Missing calls to .into()
* Wrong crate name
* [WIP] Fix Grandpa tests
* One more passing
* One more. Two to go.
* This one was actually already passing 🎉
* Last one compiles
* Progress
* grandpa: fix voter_persists_its_votes test
* Restore other tests
* Try spawn future later
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Remove necessity to pass ConsensusEngineId when registering notifications protocol
* Line width
* Fix tests protocol name
* Other renames
* Doc update
* Change issue in TODO
* client/*: Treat protocol name as str and not [u8]
Notification protocol names are in practice always valid utf8 strings.
Instead of treating them as such in the type system, thus far they were
casted to a [u8] at creation time.
With this commit protocol names are instead treated as valid utf8
strings throughout the codebase and passed as `Cow<'static, str>`
instead of `Cow<'static, [u8]>`. Among other things this eliminates the
need for string casting when logging.
* client/network: Don't allocate when protocol name is borrowed
* Open one substream for each notifications protocol
* Fix WASM build
* Apply suggestions from code review
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Address concerns
* Use unsigned-varint to read the varint
* Use unsigned-varint
* Forgot Cargo.lock
Co-authored-by: Toralf Wittner <tw@dtex.org>
* Remove usage of sc_network::Context trait
* Remove Context::send_consensus
* Pass &mut dyn Network instead of &dyn Network
* Move Validator traits and related to separate module
* Entirely update substrate-telemetry to futures 0.3
* Add a Closed error
* Update to libp2p 0.14
* More work
* More work
* More work
* More work
* Fix warnings
* Remove unwrap()
* Work on tests fixing
* Fix network tests
* Fix external network tests
* Update libp2p and restore Yamux in discovery test
* Ignore DNS if initializatio nfails
* Restore variables ordering
* Forgot browser-utils
* Fix downfall after merge
* Fix tests
The finality-grandpa module needs two sets of functionalities from the
network:
1. Everything gossip related, e.g. event_stream, write_notification, ...
2. The ability to set a fork sync request for a specific block hash.
Instead of embedding (2) inside of (1) this patch extracts (2) from (1)
having finality-grandpa depend on a `Network` that fulfills the
`network_gossip::Network` trait and that can set block sync requests.
On the one hand this improves the overall structure splitting things
that don't logically belong together. On the other hand it does
reintroduce a lot of trait bounds within finality-grandpa.
Aaro Altonen