* client/beefy: create communication module and move gossip there
* client/beefy: move beefy_protocol_name module to communication
* client/beefy: move notification module under communication
* client/beefy: add incoming request_response protocol handler
* client/beefy: keep track of connected peers and their progress
* client/beefy: add logic for generating Justif requests
* client/beefy: cancel outdated on-demand justification requests
* try Andre's suggestion for JustificationEngine
* justif engine add justifs validation
* client/beefy: impl OnDemandJustificationsEngine async next()
* move beefy proto name test
* client/beefy: initialize OnDemandJustificationsEngine
* client/tests: allow for custom req-resp protocols
* client/beefy: on-demand-justif: implement simple peer selection strategy
* client/beefy: fix voter initialization
Fix corner case where voter gets a single burst of finality
notifications just when it starts.
The notification stream was consumed by "wait_for_pallet" logic,
then main loop would subscribe to finality notifications, but by that
time some notifications might've been lost.
Fix this by subscribing the main loop to notifications before waiting
for pallet to become available. Share the same stream with the main loop
so that notifications for blocks before pallet available are ignored,
while _all_ notifications after pallet available are processed.
Add regression test for this.
Signed-off-by: acatangiu <adrian@parity.io>
* client/beefy: make sure justif requests are always out for mandatory blocks
* client/beefy: add test for on-demand justifications sync
* client/beefy: tweak main loop event processing order
* client/beefy: run on-demand-justif-handler under same async task as voter
* client/beefy: add test for known-peers
* client/beefy: reorg request-response module
* client/beefy: add issue references for future work todos
* client/beefy: consolidate on-demand-justifications engine state machine
Signed-off-by: acatangiu <adrian@parity.io>
* client/beefy: fix for polkadot companion
* client/beefy: implement review suggestions
* cargo fmt and clippy
* fix merge damage
* fix rust-doc
* fix merge damage
* fix merge damage
* client/beefy: add test for justif proto name
Signed-off-by: acatangiu <adrian@parity.io>
* Receive and import target block body
* Request target block
* minor: wording
* Check for block body in the test
* Import target block justifications
* Fix: do not fail block validation if no justifications received
* Fix: import target blocks without justifications
Co-authored-by: arkpar <arkady.paronyan@gmail.com>
* Introduce `sc-network-bitswap`
Move the bitswap protocol out of `sc-network` to its own crate.
* Improve test coverage
* Remove mention of bitswap from `sc-network`
* Fix documentation
* Fix clippy and remove the void dependency
* Remove unneeded trait bound and bump prost to 0.11
* Add ProtocolName custom type
* Use new ProtocolName in sc_network_common
* Use new ProtocolName in sc_network
* Use new ProtocolName for BEEFY and GRANDPA
* Use new ProtocolName for notifications
* Use new ProtocolName in sc_network (part 2)
* Use new ProtocolName in sc_network_gossip
* Use new ProtocolName in sc_offchain
* Remove unused imports
* Some more fixes
* Add tests
* Fix minor import issues
* Re-export ProtocolName in sc_network
* Revert "Re-export ProtocolName in sc_network"
This reverts commit 8d8ff71927e7750757f29c9bbd88dc0ba181d214.
* Re-export ProtocolName in sc_network
* Remove dependency on sc-network-common from beefy-gadget
* Move `MultiaddrWithPeerId` and related parsing functions into `sc-network-common`, remove dependency on `sc-network` from `sc-chain-spec`
* Remove dependency on `sc-network` from `sc-offchain`
* Remove dependency on `sc-network` from `sc-network-gossip`
* Extract `NetworkKVProvider` trait in `sc-authority-discovery` and remove unnecessary dependency
* Extract `NetworkSyncForkRequest` trait in `sc-finality-grandpa`
* Relax requirements on `SyncOracle` trait, remove extra native methods from `NetworkService` that are already provided by trait impls
* Move `NetworkSigner` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
* Move `NetworkKVProvider` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
* Minimize `sc-authority-discovery` dependency on `sc-network`
* Move `NetworkSyncForkRequest` trait from `sc-finality-grandpa` to `sc-network-common` and de-duplicate methods in `NetworkService`
* Extract `NetworkStatusProvider` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkPeers` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkEventStream` trait and de-duplicate methods on `NetworkService`
* Move more methods from `NetworkService` into `NetworkPeers` trait
* Move `NetworkStateInfo` trait into `sc-network-common`
* Extract `NetworkNotification` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkRequest` trait and de-duplicate methods on `NetworkService`
* Remove `NetworkService::local_peer_id()`, it is already provided by `NetworkStateInfo` impl
* Extract `NetworkTransaction` trait and de-duplicate methods on `NetworkService`
* Extract `NetworkBlock` trait and de-duplicate methods on `NetworkService`
* Remove dependencies on `NetworkService` from most of the methods of `sc-service`
* Address simple review comments
* Rename transactions protocol to include genesis hash
* Add protocol name generation to sc_network::utils
* Use utils functions for transactions protocol name generation
* Extract protocol name generation into public module
* Use sc_network::protocol_name::standard_protocol_name() for BEEFY and GRANDPA
* minor: add missing newline at EOF
* Change block-announces protocol name to include genesis_hash & fork_id
* Change protocol names to include genesis hash and fork id
Protocols changed:
- sync
- state
- light
- sync/warp
* Revert "Use sc_network::protocol_name::standard_protocol_name() for BEEFY and GRANDPA"
This reverts commit cd60a95a3face397e1b67f4bc95dd0f2b581bfae.
* Get rid of `protocol_name` module
* client/beefy: don't accept vote for older rounds
* client/beefy: clean up and reorg the worker struct
* client/beefy: first step towards Full BEEFY
The first step from Lean->Full BEEFY is to have the worker
enforce uninterrupted line of BEEFY finalized mandatory blocks.
There is one mandatory block per session (the first block in the
session). As such, votes processing and votes generation now
enforces that all mandatory blocks are finalized in strict
monotonically increasing sequence and no block 'N' will be worked
on if there is any GRANDPA finalized but BEEFY non-final mandatory
block 'M', where 'M < N'.
Implementation details:
- Introduced 'VoterOracle' to separate the voting decisions logic,
and track new/pending sessions.
- New sessions get queued up with the worker operating either:
1. up-to-date - all mandatory blocks leading up to current GRANDPA
finalized: queue has ONE element, the 'current session' where
`mandatory_done == true`,
2. lagging behind GRANDPA: queue has [1, N] elements, where all
`mandatory_done == false`.
In this state, everytime a session gets its mandatory block
BEEFY finalized, the session is popped off the queue,
eventually getting to operating mode `1. up-to-date`.
- Votes get triaged and those that fall withing the `VoterOracle`
allowed window get processed, the others get dropped if stale,
or buffered for later processing (when they reach the window).
- Worker general code was also updated to fall in one of two roles:
1. react to external events and change internal 'state',
2. generate events/votes based on internal 'state'.
Signed-off-by: acatangiu <adrian@parity.io>
* client/beefy: sketch idea for block import and sync
Signed-off-by: acatangiu <adrian@parity.io>
* client/beefy: add BEEFY block import
* client/beefy: process justifications from block import
* client/beefy: add TODOs for sync protocol
* client/beefy: add more docs and comments
* client/beefy-rpc: fix RPC error
* client/beefy: verify justification validity on block import
* client/beefy: more tests
* client/beefy: small fixes
- first handle and note the self vote before gossiping it,
- don't shortcircuit on err when processing pending votes.
* client/beefy: remove invalid justifications at block import
* todo: beefy block import tests
* RFC: ideas for multiple justifications per block
* Revert "RFC: ideas for multiple justifications per block"
This reverts commit 8256fb07d3124db69daf252720b3c0208202624d.
* client/beefy: append justif to backend on block import
* client/beefy: groundwork for block import test
* client/beefy: groundwork2 for block import test
* client/beefy: groundwork3 for block import test
* client/beefy: add block import test
* client/beefy: add required trait bounds to block import builder
* remove client from beefy block import, backend gets the job done
Signed-off-by: acatangiu <adrian@parity.io>
* Remove direct dependency of `sc-network` on `sc-network-light`
* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`
* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`
* Create `sync` module in `sc-network-common`, create `ChainSync` trait there (not used yet), move a bunch of associated data structures from `sc-network-sync`
* Switch from concrete implementation to `ChainSync` trait from `sc-network-common`
* Introduce `OpaqueStateRequest`/`OpaqueStateResponse` to remove generics from `StateSync` trait
* Introduce `OpaqueBlockRequest`/`OpaqueBlockResponse`, make `scheme` module of `sc-network-sync` private
* Surface `sc-network-sync` into `sc-service` and make `sc-network` not depend on it anymore
* Remove now unnecessary dependency from `sc-network`
* Replace crate links with just text since dependencies are gone now
* Remove `warp_sync` re-export from `sc-network-common`
* Update copyright in network-related files
* Address review comments about documentation
* Apply review suggestion
* Rename `extra_requests` module to `metrics`
Co-authored-by: Bastian Köcher <info@kchr.de>
* Upgrade to libp2p 0.45.1
* Limit max_negotiating_inbound_streams to 512
* Upgrade prost-build to 0.10
* Set max_negotiating_inbound_streams to 2048
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Fix authority discovery protobuf
* Fix comments in authority-discovery schema
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Add a comment about transport initialization
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Move `api.v1.proto` schema into new crate `sc-network-sync`
* Move `sc_network::protocol::sync::state` module into `sc_network_sync::state`
* Move `sc_network::protocol::sync::blocks` module into `sc_network_sync::blocks` and some data structures from `sc_network::protocol::message` module into `sc_network_sync::message`
* Move some data structures from `sc_network::config` and `sc_network::request_responses` into new `sc-network-common` crate
* Move `sc_network::protocol::sync::warm` and `sc_network::warp_request_handler` modules into `sc_network_sync`
* Move `client/network/sync/src/lib.rs` to `client/network/sync/src/lib_old.rs` to preserve history of changes of the file in the next commit
* Move `client/network/src/protocol/sync.rs` on top of `client/network/sync/src/lib.rs` to preserve history of changes
* Move `sc_network::protocol::sync` to `sc_network_sync` with submodules, move message data structures around accordingly
* Move `sc_network::block_request_handler` to `sc_network_sync::block_request_handler`
* Move `sc_network::state_request_handler` to `sc_network_sync::state_request_handler`
* Add re-exports for compatibility reasons
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Remove unnecessary imports, move one internal re-export into where it is actually used, make one import explicit
* Move a few data structures down into modules
* Use generic parameters in `sc-network` instead of `chain::Client` trait
* Remove unnecessary bound
* Upgraded dependencies
* Adapting code to scale v3
* Empty commit to trigger CI
* Triggering CI
* Fixing UI test
* Remove superfluous dev-dep added by #9228
* Cryout for CI
* Add stale branches heads to finality notifications
Warning. Previous implementation was sending a notification for
each block between the previous (explicitly) finalized block and
the new finalized one (with an hardcoded limit of 256).
Now finality notification is sent only for the new finalized head and it
contains the hash of the new finalized head, new finalized head header,
a list of all the implicitly finalized blocks and a list of stale
branches heads (i.e. the branches heads that are not part of the
canonical chain anymore).
* Add implicitly finalized blocks list to `ChainEvent::Finalized` message
The list contains all the blocks between the previously finalized block
up to the parent of the currently finalized one, sorted by block number.
`Finalized` messages handler, part of the `MaintainedTransactionPool`
implementation for `BasicPool`, still propagate full set of finalized
blocks to the txpool by iterating over implicitly finalized blocks list.
* Rust fmt
* Greedy evaluation of `stale_heads` during finalization
* Fix outdated assumption in a comment
* Removed a test optimization that is no more relevant
The loop was there to prevent sending to
`peer.network.on_block_finalized` the full list of finalized blocks.
Now only the finalized heads are received.
* Last finalized block lookup not required anymore
* Tests for block finality notifications payloads
* Document a bit tricky condition to avoid duplicate finalization notifications
* More idiomatic way to skip an iterator entry
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Cargo fmt iteration
* Typo fix
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix potential failure when a finalized orphan block is imported
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Introduce `SecretUri`
* `inspect-key`: Adds support for `expect-public`
`expect-public` can be used to check that a given secret uri corresponds to the given public key.
This is mainly useful when the secret uri is protected by a password and a new derived account
should be generated. With `--expect-public` the user can pass the public key/account-id of the
"base" secret uri aka the one without any derivation to ensure the correct password was inserted.
* Fixes
* 🤦
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Review feedback
* FMT
* Bump the versions
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* state machine proofs.
* initial implementation
* Remove todo.
* Extend test and fix import.
* fix no proof, with proof ko.
* fix start at logic.
* Restore response size.
* Rework comments.
* Add explicit ref
* Use compact proof.
* ref change
* elaborato on empty change set condition.
* KeyValueState renaming.
* Do not add two time child trie with same root to sync reply.
* rust format
* Fix merge.
* fix warnings and fmt
* fmt
* update protocol id to V2
Aaro Altonen