"sc-consensus-beefy: restart voter on pallet reset #14821" introduced
a mechanism to reinitialize the BEEFY worker on certain errors; but
re-creating the GossipEngine doesn't play well with
"Rework the event system of sc-network #14197".
So this PR slightly changes the re-initialization logic to reuse the original
GossipEngine and not recreate it.
Signed-off-by: Adrian Catangiu <adrian@parity.io>
When detecting pallet-beefy consensus reset, just reinitialize the worker
and continue without bringing down the task (and possibly the node).
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* Merged BEEFY primitives with generic signature and keyset commitment support from old pull to current code
* - Add bls-experimental feature to application-crypto and beefy primitives
- Fix remaining crypto -> ecdsa_crypto
- code build but not tests
* Make beefy primitive tests compile
* move bls related beefy primitives code and test behind bls-experimental flag
* Make BEEFY clients complies with BEEFY API depending on AuthorityId
* - Rename `BeefyAuthoritySet.root` → `BeefyAuthoritySet.keyset_commitment`.
- Remove apk proof keyset_commitment from `BeefyAuthoritySet`.
- Fix failing signed commitment and signature to witness test.
- Make client compatible with BeefyAPI generic on AuthorityId.
- `crypto` → `ecdsa_crypto` in BEEFY client and frame.
* Commit Cargo lock remove ark-serialize from BEEFY primitives
* Use Codec instead of Encode + Decode in primitives/consensus/beefy/src/lib.rs
Co-authored-by: Davide Galassi <davxy@datawok.net>
* - Make `BeefyApi` generic over Signature type.
- Make new `BeeyApi` functinos also generic over AuthorityId and Signature
* Unmake BeefyAPI generic over Signature. Recover Signature type from AuthId.
* - dont use hex or hex-literal use array-bytes instead in beefy primitives and bls crypto.
- CamelCase ECDSA and BLS everywhere.
* Move the definition of BEEFY key type from `primitives/beefy` to `crypto.rs` according to new convention.
* - Add bls377_generate_new to `sp-io` and `application_crypto::bls`.
- Add `bls-experimental` to `sp-io`
Does not compile because PassByCodec can not derive PassBy using customly implemented PassByIner.
* Implement PassBy for `bls::Public` manually
* fix Beefy `KEY_TYPE` in `frame/beefy` tests to come from `sp-core::key_types` enum
* specify both generic for `hex2array_unchecked` in `sp-core/bls.rs`
* Rename `crypto`→`ecdsa_crypto` in `primitives/consensus/beefy/src/test_utils.rs` docs
* remove commented-out code in `primitives/consensus/beefy/src/commitment.rs`
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Fix inconsistency in panic message in `primitives/io/src/lib.rs`
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Remove redundant feature activation in `primitives/io/Cargo.toml`
Co-authored-by: Davide Galassi <davxy@datawok.net>
* - make `w3f-bls` a dev-dependancy only for beefy primitives.
- clean up comments.
Co-authored-by: Davide Galassi <davxy@datawok.net>
* export BEEFY KEY_TYPE from primitives/consensus/beefy
make `frame/consensus/beefy` in dependent of sp_crypto_app
use consistent naming in the beefy primitive tests.
* - implement `BeefyAuthorityId` for `bls_crypto::AuthorityId`.
- implement `bls_verify_works` test for BEEFY `bls_crypto`.
* Remove BEEFY `ecdsa_n_bls_crypto` for now for later re-introduction
* Make commitment and witness BEEFY tests not use Keystore.
* put `bls_beefy_verify_works` test under `bls-experimental` flag.
* bump up Runtime `BeefyAPI` to version 3 due to introducing generic AuthorityId.
* reuse code and encapsulate w3f-bls backend in sp-core as most as possible
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Make comments in primities BEEFY `commitment.rs` and `witness.rs``tests convention conforming
* Use master dep versions
* Trivial change. Mostly to trigger CI
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix Cargo.toml
* Trigger CI with cumulus companion
* Trigger CI after polkadot companion change
---------
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* frame-benchmarking-cli: Remove native dispatch requirement
No need for this, we can just use the `WasmExecutor` directly.
* Fixes
* Pass benchmarking host functions
* Ensure we can pass custom host functions
BEEFY consensus can be restarted by resetting "genesisBlock" in
pallet-beefy, but we don't want to also reset authority set IDs so
that they are uniquely identified across the entire chain history
regardless of how many times BEEFY consensus has been reset/restarted.
This is why the client now also accepts initial authority_set_id != 0.
BEEFY client now detects pallet-beefy reset/reinit and errors-out and
asks for a restart.
BEEFY client persisted state should be discarded on client restarts
following pallet-beefy reset/reinit.
End result is BEEFY client/voter can now completely reinitialize using
"new" on-chain info following pallet-beefy reset/reinit, discarding old state.
Fixes#14203Fixes#14204
Signed-off-by: acatangiu <adrian@parity.io>
* sc-consensus-beefy: improve beefy gossip validator
Old gossip validator was pretty dumb, being very permissive with
incoming votes - only condition it had was to be newer than best
finalized.
New filter conditions:
- voter rounds are initialized (discarding votes until voter is
actually active),
- only votes for current active set id are accepted,
- only votes for rounds in the current voting session are accepted,
- only votes for GRANDPA finalized blocks are accepted,
- when BEEFY voter reaches mandatory round, only votes for said
mandatory round are accepted.
New validator uses the VoterOracle to easily implement above conditions
and only allow through votes that are immediately useful to the voter.
After every GRANDPA or BEEFY finality, the gossip validator filter is
updated.
* sc-consensus-beefy: remove votes enqueueing
Since gossip validator will simply disallow votes for future rounds,
and only allow votes that the voter can immediately process, there
is no need for the voter to enqueue votes.
It will see these "future" votes later in rebroadcasts, when voter
will also be able to process them. Only at that point does gossip
accept and consume them.
* sc-consensus-beefy: refactor persistent state
Move best-beefy and best-grandpa into VoterOracle instead
of passing them around as params.
VoterOracle ultimately needs to know best-beefy and/or best-grandpa
for most of its functions.
* sc-consensus-beefy: further restrict gossip validator
Assuming mandatory done in current session:
Instead of allowing votes for any round in the current session, only
accept votes for rounds equal or better than best BEEFY finalized.
* sc-consensus-beefy: add a couple of comments
* sc-consensus-beefy: fix tests involving multiple tasks
Finalize blocks one a time in tests where we want gossip to happen
in a certain round. Otherwise, some tasks may be left behind in
terms of gossip round numbers because once "scheduled" a task will
greedily process as much as possible.
This change should be in line with the real-world scenario where
voters run "in parallel" across nodes, the only points of
synchronization being the finality notifications.
* sc-consensus-beefy: address review comments
---------
Signed-off-by: acatangiu <adrian@parity.io>
`futures_util::pending!()` macro only yields to the event loop once,
resulting in many calls to the `OnDemandJustificationsEngine::next()`
made by the tokio reactor even though the on-demand-engine is idle.
Replace it with the correct `futures::future::pending::<()>()` function
which forever yields to the event loop, which is what we want when
the engine is idle.
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* Move import queue out of `sc-network`
Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.
This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.
* Move stuff to SyncingEngine
* Move `ChainSync` instanation to `SyncingEngine`
Some of the tests have to be rewritten
* Move peer hashmap to `SyncingEngine`
* Let `SyncingEngine` to implement `ChainSyncInterface`
* Introduce `SyncStatusProvider`
* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`
* Implement `SyncEventStream`
Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.
Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.
* Introduce `ChainSyncInterface`
This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.
* Move event stream polling to `SyncingEngine`
Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.
The code needs refactoring.
* Make `SyncingEngine` into an asynchronous runner
This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.
Code needs a lot of refactoring.
* Fix warnings
* Code refactoring
* Use `SyncingService` for BEEFY
* Use `SyncingService` for GRANDPA
* Remove call delegation from `NetworkService`
* Remove `ChainSyncService`
* Remove `ChainSync` service tests
They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.
* Refactor code
* Refactor code
* Update client/finality-grandpa/src/communication/tests.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Fix warnings
* Apply review comments
* Fix docs
* Fix test
* cargo-fmt
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Add missing docs
* Refactor code
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Adrian Catangiu