pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-05-01 17:07:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Integrate BEEFY (#9833)

Browse Source 
* Initial project setup and skeleton (#4)

* initial project setup for beefy gadget client

* update editorconfig

* update gitignore

* add initial skeleton for beefy gadget worker

* add skeleton for gossip processing

* add app crypto

* move around some code

* add basic flow for voting

* add logic for picking blocks to sign

* add rustfmt config

* add example node with beefy gadget

* use u32::next_power_of_two

* make maximum periodicity configurable

* add copyright header

* rename max_periodicity to min_interval

* CI stuff (#5)

* CI stuff.

* Fix workspace.

* cargo fmt --all

* Add license for beefy-gadget

* One toolchain to rule them all.

* Clippy.

* Fix clippy.

* Clippy in the runtime.

* Fix clippy grumbles.

* cargo fmt --all

* Primitives & Light Client examples (#8)

* Primitives.

* Docs.

* Document primitives.

* Simple tests.

* Light client examples.

* Fix stuff.

* cargo fmt --all

* Add a bunch of tests for imports.

* Add more examples.

* cargo fmt --all

* Fix clippy.

* cargo fmt --all

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Add GRANDPA / FG clarifications.

* Fix min number of signatures.

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update to substrate master (#22)

* update to substrate master

* update dependencies

* fix clippy issues

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Add beefy pallet (#25)

* move beefy application crypto to primitives

* make primitives compile under no_std

* add beefy pallet that maintains authority set

* add beefy pallet to node example runtime

* tabify node-example cargo.toml files

* use double quotes in Cargo.toml files

* add missing hex-literal dependency

* add runtime api to fetch BEEFY authorities

* fix clippy warnings

* rename beefy-pallet to pallet-beefy

* sort dependencies in node-example/runtime/Cargo.toml

* Signed commitments rpc pubsub (#26)

* move beefy application crypto to primitives

* make primitives compile under no_std

* add beefy pallet that maintains authority set

* add beefy pallet to node example runtime

* tabify node-example cargo.toml files

* use double quotes in Cargo.toml files

* add missing hex-literal dependency

* add runtime api to fetch BEEFY authorities

* fix clippy warnings

* gadget: use commitment and signedcommitment

* gadget: send notifications for signed commitments

* gadget: add rpc pubsub for signed commitments

* node-example: enable beefy rpc

* gadget: fix clippy warnings

* rename beefy-pallet to pallet-beefy

* sort dependencies in node-example/runtime/Cargo.toml

* gadget: add documentation on SignedCommitment rpc wrapper type

* gadget: add todos about dummy beefy commitments

* gadget: remove redundant closure

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Integrate MMR and deposit root into the digest. (#24)

* Add basic MMR.

* Deposit digest item.

* cargo fmt --all

* Merge with primitives.

* cargo fmt --all

* Fix extra spaces.

* cargo fmt --all

* Switch branch.

* remove stray whitespace

* update to latest td-mmr commit

* fix clippy error

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* use new mmr root as commitment payload (#27)

* use new mmr root as commitment payload

* fix mmr root codec index

* warn on MMR root digest not found

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* add type alias for MMR root hash

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Bump serde_json from 1.0.59 to 1.0.60 (#28)

* Update to latest substrate. (#32)

* Update to latest substrate.

* Fix tests.

* cargo fmt --all

* Switch to master.

* Bump serde from 1.0.117 to 1.0.118 (#29)

* Bump serde from 1.0.117 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.117 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.117...v1.0.118)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump arc-swap.

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Remove transition flag (#35)

* Get rid of is_set_transition_flag

* Fix tests.

* cargo fmt --all

* Bump futures from 0.3.9 to 0.3.12 (#50)

* Bump log from 0.4.11 to 0.4.13 (#52)

* Bump Substrate and Deps (#57)

* Update README (#58)

* Update README

* Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* address review comments

* missed a typo

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Add validator set to the pallet. (#65)

* Bump Substrate and Deps (#71)

* Bump Substrate and Deps

* pin serde and syn

* bump Substrate again for '__Nonexhaustive' fix

* add cargo deny ignore

* Beefy pallet test (#74)

* setup mock

* test session change

* silence beefy

* clippy still

* no change - no log

* clippy again

* Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* code review changes, added additional test

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Beefy node cleanup (#75)

* bump serde

* bump substrate, scale-codec 2.0.0

* we need a proper beefy node

* rename primitives as well

* Sort members.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Migrate beefy-pallet to FRAMEv2 (#76)

* migrate beefy-pallet to FRAMEv2

* Code review

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Run BEEFY worker as non-validator (#77)

* run BEEFY worker as non-validator

* don't check for roloe.is_authority

* change enum type name

* Bump Substrate and Deps (#79)

* Add BEEFY gadget as extra peer set (#80)

* Add BEEFY gadget as extra peer set

* use BEEFY protocol

* Add ValidatorSetId to BEEFY digest (#85)

* add ValidatorSetId to BEEFY digest

* apply review changes

* Bump Substrate and Deps (#91)

* Bump Substrate and Deps

* Bump Substrate again in order to include a hot-fix

* redo again

* use CryptoStore issue

* cargo fmt

* Bump serde_json from 1.0.63 to 1.0.64 (#93)

* Track BEEFY validator set (#94)

* Track BEEFY validator set

* Add validator_set_id to BeefyWorker

* Make validattor_set_id optional

* Ad 92 (#97)

* sign_commitment()

* Error handling todo

* Add error type (#99)

* Add error type

* Address review

* Extract worker and round logic (#104)

* Bump serde from 1.0.123 to 1.0.124 (#106)

* Rework BeefyAPI (#110)

* Initialize BeefyWorker with current validator set (#111)

* Update toolchain (#115)

* Use nightly toolchain

* dongradde to latest clippy stable

* GH workflow trail and error

* next try

* use stable for clippy

* update wasm builder

* yet another try

* fun with CI

* no env var

* and one more

* allow from_over_into bco contruct_runtime

* back to start

* well ...

* full circle

* old version was still used

* Bump Substrate and Deps (#117)

* Bump Substrate and Deps

* cargo fmt should enforce uniform imports

* merge some imports

* Delayed BEEFY worker initialization (#121)

* lifecycle state

* add Client convenience trait

* rework trait identifiers

* WIP

* rework BeefyWorker::new() signature

* Delayed BEEFY gadget initialization

* address review

* Bump substrate. (#123)

* Bump substrate.

* Fix tests.

* Lower log-level for a missing validator set (#124)

* lower log-level for a missing validator set

* move best_finalized_block initialization

* Setup Prometheus metrics (#125)

* setup Prometheus metrics

* expose validator set id

* cargo fmt

* Update beefy-gadget/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* add vote messages gossiped metric

* track authorities change, before checking for MMR root digest

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Make Client convenience trait public (#126)

* Bump serde from 1.0.124 to 1.0.125 (#131)

* Reset rounds on new validator set. (#133)

* Re-set rounds on new validator set.

* Fix docs.

* Bump Substrate and Deps (#134)

* beefy: authority set changes fixes (#139)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type (#140)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* gadget: reduce gossip spam (#141)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type

* gadget: only gossip last 5 rounds

* gadget: note round to gossip validator before gossiping message

* gadget: fix clippy warnings

* gadget: update docs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* gadget: verify SignedCommitment message signature (#142)

* gadget: verify SignedCommitment message signature

* gadget: log messages with bad sigs

* gadget: move todo comment

* Bump futures from 0.3.13 to 0.3.14 (#145)

* Milestone 1 (#144)

* use best_finalized, prevent race

* make best_finalized_block an Option, should_vote_on bails on None

* Bump futures from 0.3.13 to 0.3.14

* Revert futures bump

* Revert "Revert futures bump"

This reverts commit a1b5e7e9bac526f2897ebfdfee7f02dd29a13ac5.

* Revert "Bump futures from 0.3.13 to 0.3.14"

This reverts commit a4e508b118ad2c4b52909d24143c284073961458.

* debug msg if the bail voting

* validator_set()

* local_id()

* get rid of worker state

* Apply review suggestions

* fix should_vote_on()

* Extract BeefyGossipValidator (#147)

* Extract BeefyGossipValidator

* Apply review suggestions

* Add block_delta parameter to start_beefy_gadget (#151)

* Add block_delta parameter

* rename to min_block_delta

* Add additional metrics (#152)

* Add additional metrics

* add skipped session metric

* add some comment for temp metric

* don't log under info for every concluded round (#156)

* don't log error on missing validator keys (#157)

* don't log error on missing validator keys

* remove unused import

* Fix validator set change handling (#158)

* reduce some logs from debug to trace

* fix validator set changes handling

* rename validator module to gossip

* run rustfmt

* Fix should_vote_on() (#160)

* Fix should_vote_on()

* by the textbook

* fix the algorithm

* Apply review suggestions

* don't use NumberFor in vote_target

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* Make KeyStore optional (#173)

* Use builder pattern for NonDefaultSetConfig (#178)

Co-authored-by: adoerr <0xad@gmx.net>

* Append SignedCommitment to block justifications (#177)

* Append SignedCommitment

* add BeefyParams

* add WorkerParams

* use warn

* versioned variant for SignedCommitment

* Bump serde from 1.0.125 to 1.0.126 (#184)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.125 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.125...v1.0.126)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump strum from 0.20.0 to 0.21.0 (#195)

* Bump strum from 0.20.0 to 0.21.0

Bumps [strum](https://github.com/Peternator7/strum) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use dervie feature for strum; clippy and deny housekeeping

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Make concluded round an info log (#200)

* Remove external crypto trait bounds (#207)

* BeefyKeystore newtype

* WIP

* remove mod ecdsa

* WIP

* fix tests

* some polishing

* Rename AuthorityId to BeefyId to avoid type conflict in UI (#211)

* Add trace points; Reduce MAX_LIVE_GOSSIP_ROUNDS (#210)

* Add trace points; Reduce MAX_LIVE_GOSSIP_ROUNDS

* log local authority id

* Additional initial authority id's (#217)

* Scratch concluded rounds

* adjust testnet doc

* fix authority key typo

* We don't want no scratches

* address review comments

* Fix note_round() (#219)

* rename BeefyGossipValidator

* Fix note_round()

* use const for assert

* put message trace points back in

* test case note_same_round_twice()

* address review comments

* remove redundant check

* Use LocalKeystore for tests (#224)

* private_keys()

* Use LocalKeystore for tests

* Use keystore helper

* Address review

* some reformatting

* Cache known votes in gossip (#227)

* Implement known messages cache.

* Add tests.

* Appease clippy.

* More clippy

Co-authored-by: adoerr <0xad@gmx.net>

* Some key store sanity checks (#232)

* verify vote message

* verify_validator_set()

* rework logging

* some rework

* Tone down warnings.

* Add signature verification.

* Tone down more.

* Fix clippy

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Use Binary Merkle Tree instead of a trie (#225)

* Binary tree merkle root.

* Add proofs and verification.

* Clean up debug.

* Use BEEFY addresses instead of pubkeys.

* Use new merkle tree.

* Optimize allocations.

* Add test for larger trees.

* Add tests for larger cases.

* Appease clippy

* Appease clippy2.

* Fix proof generation & verification.

* Add more test data.

* Fix CLI.

* Update README

* Bump version.

* Update docs.

* Rename beefy-merkle-root to beefy-merkle-tree

Co-authored-by: adoerr <0xad@gmx.net>

* Bump Substrate and Deps (#235)

* BEEFY+MMR pallet (#236)

* Add MMR leaf format to primitives.

* Fix tests

* Initial work on the BEEFY-MMR pallet.

* Add tests to MMR pallet.

* Use eth addresses.

* Use binary merkle tree.

* Bump libsecp256k1

* Fix compilation.

* Bump deps.

* Appease cargo deny.

* Re-format.

* Module-level docs.

* no-std fix.

* update README

Co-authored-by: adoerr <0xad@gmx.net>

* Fix noting rounds for non-authorities (#238)

* Bump env_logger from 0.8.4 to 0.9.0 (#242)

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.4 to 0.9.0.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.4...v0.9.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* gadget: add global timeout for rebroadcasting messages (#243)

* gadget: add global timeout for rebroadcasting messages

* update rustfmt.toml

* make message_allowed() a debug trace

Co-authored-by: adoerr <0xad@gmx.net>

* Bump Substrate and Deps (#245)

* Bump Substrate and Deps

* Bump Substrate again

* Bump futures from 0.3.15 to 0.3.16 (#247)

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump libsecp256k1 from 0.5.0 to 0.6.0 (#249)

* Bump libsecp256k1 from 0.5.0 to 0.6.0

Bumps [libsecp256k1](https://github.com/paritytech/libsecp256k1) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/paritytech/libsecp256k1/releases)
- [Changelog](https://github.com/paritytech/libsecp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/libsecp256k1/commits)

---
updated-dependencies:
- dependency-name: libsecp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use correct crate name

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Derive `scale_info::TypeInfo` for types used in polkadot (#218)

* Add scale-info TypeInfo derives

* Update scale-info

* Add crates.io patches

* Use substrate aj-metadata-vnext branch

* Revert master branch substrate deps

* Add scale-info to beefy-pallet

* scale-info v0.9.0

* Remove github dependencies and patches

* More TypeInfo derives

* Update scale-info to 0.10.0

* Add missing scale-info dependency

* Add missing TypeInfo derive

* Hide TypeInfo under a feature.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Bump serde from 1.0.126 to 1.0.127 (#260)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Substrate and Deps (#262)

* Update jsonrpc (#265)

* Update jsonrpc

* Update Substrate

* bump Substrate and Deps (#268)

* Bump serde from 1.0.127 to 1.0.128 (#272)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix spelling (#271)

* Bump serde from 1.0.128 to 1.0.130 (#276)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.128 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.130)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump scale-info from 0.10.0 to 0.12.0 (#275)

Bumps [scale-info](https://github.com/paritytech/scale-info) from 0.10.0 to 0.12.0.
- [Release notes](https://github.com/paritytech/scale-info/releases)
- [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/scale-info/commits)

---
updated-dependencies:
- dependency-name: scale-info
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Update to scale-info 1.0 (#278)

* bump substrate (#282)

* bump Substrate and Deps

* cargo fmt

Co-authored-by: Wenfeng Wang <kalot.wang@gmail.com>

* Update worker.rs (#287)

* Bump anyhow from 1.0.43 to 1.0.44 (#290)

* Bump anyhow from 1.0.43 to 1.0.44

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.43 to 1.0.44.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.43...1.0.44)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* derive Default

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Remove optional `scale-info` feature (#292)

* Make scale-info dependency non-optional

* Remove feature gated TypeInfo derives

* Import TypeInfo

* Update substrate

* Fix up runtime

* prune .git suffix (#294)

* remove unused deps (#295)

* remove unused deps

* update lock file

* Bump libsecp256k1 from 0.6.0 to 0.7.0 (#296)

* Bump libsecp256k1 from 0.6.0 to 0.7.0

Bumps [libsecp256k1](https://github.com/paritytech/libsecp256k1) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/paritytech/libsecp256k1/releases)
- [Changelog](https://github.com/paritytech/libsecp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/libsecp256k1/commits)

---
updated-dependencies:
- dependency-name: libsecp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update sec advisories

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* clean compile

* use path dependencies

* beefy-gadget license header

* pallet-beefy license header

* pallet-beefy-mmr license header

* beefy-primitves license header

* carg fmt

* more formatting

* shorten line

* downgrade parity-scale-codec to 2.2.0

* use path dependency for Prometheus endpoint

* remove clippy annotations

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: drewstone <drewstone329@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Wenfeng Wang <kalot.wang@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
This commit is contained in:
Andreas Doerr
2021-09-23 21:02:30 +02:00
committed by GitHub
parent 6845666c5c
commit 283c8daa81
31 changed files with 4992 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
substrate/client/beefy/src/error.rs
+31
View File
@@ -0,0 +1,31 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! BEEFY gadget specific errors
//!
//! Used for BEEFY gadget interal error handling only
use std::fmt::Debug;
#[derive(Debug, thiserror::Error, PartialEq)]
pub enum Error {
#[error("Keystore error: {0}")]
Keystore(String),
#[error("Signature error: {0}")]
Signature(String),
}
substrate/client/beefy/src/gossip.rs
+236
View File
@@ -0,0 +1,236 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{collections::BTreeMap, time::Duration};
use sc_network::PeerId;
use sc_network_gossip::{MessageIntent, ValidationResult, Validator, ValidatorContext};
use sp_core::hashing::twox_64;
use sp_runtime::traits::{Block, Hash, Header, NumberFor};
use codec::{Decode, Encode};
use log::{debug, trace};
use parking_lot::{Mutex, RwLock};
use wasm_timer::Instant;
use beefy_primitives::{
crypto::{Public, Signature},
MmrRootHash, VoteMessage,
};
use crate::keystore::BeefyKeystore;
#[cfg(test)]
#[path = "gossip_tests.rs"]
mod tests;
// Limit BEEFY gossip by keeping only a bound number of voting rounds alive.
const MAX_LIVE_GOSSIP_ROUNDS: usize = 3;
// Timeout for rebroadcasting messages.
const REBROADCAST_AFTER: Duration = Duration::from_secs(60 * 5);
/// Gossip engine messages topic
pub(crate) fn topic<B: Block>() -> B::Hash
where
B: Block,
{
<<B::Header as Header>::Hashing as Hash>::hash(b"beefy")
}
/// A type that represents hash of the message.
pub type MessageHash = [u8; 8];
type KnownVotes<B> = BTreeMap<NumberFor<B>, fnv::FnvHashSet<MessageHash>>;
/// BEEFY gossip validator
///
/// Validate BEEFY gossip messages and limit the number of live BEEFY voting rounds.
///
/// Allows messages from last [`MAX_LIVE_GOSSIP_ROUNDS`] to flow, everything else gets
/// rejected/expired.
///
///All messaging is handled in a single BEEFY global topic.
pub(crate) struct GossipValidator<B>
where
B: Block,
{
topic: B::Hash,
known_votes: RwLock<KnownVotes<B>>,
next_rebroadcast: Mutex<Instant>,
}
impl<B> GossipValidator<B>
where
B: Block,
{
pub fn new() -> GossipValidator<B> {
GossipValidator {
topic: topic::<B>(),
known_votes: RwLock::new(BTreeMap::new()),
next_rebroadcast: Mutex::new(Instant::now() + REBROADCAST_AFTER),
}
}
/// Note a voting round.
///
/// Noting `round` will keep `round` live.
///
/// We retain the [`MAX_LIVE_GOSSIP_ROUNDS`] most **recent** voting rounds as live.
/// As long as a voting round is live, it will be gossiped to peer nodes.
pub(crate) fn note_round(&self, round: NumberFor<B>) {
debug!(target: "beefy", "🥩 About to note round #{}", round);
let mut live = self.known_votes.write();
if !live.contains_key(&round) {
live.insert(round, Default::default());
}
if live.len() > MAX_LIVE_GOSSIP_ROUNDS {
let to_remove = live.iter().next().map(|x| x.0).copied();
if let Some(first) = to_remove {
live.remove(&first);
}
}
}
fn add_known(known_votes: &mut KnownVotes<B>, round: &NumberFor<B>, hash: MessageHash) {
known_votes.get_mut(round).map(|known| known.insert(hash));
}
// Note that we will always keep the most recent unseen round alive.
//
// This is a preliminary fix and the detailed description why we are
// doing this can be found as part of the issue below
//
// https://github.com/paritytech/grandpa-bridge-gadget/issues/237
//
fn is_live(known_votes: &KnownVotes<B>, round: &NumberFor<B>) -> bool {
let unseen_round = if let Some(max_known_round) = known_votes.keys().last() {
round > max_known_round
} else {
known_votes.is_empty()
};
known_votes.contains_key(round) || unseen_round
}
fn is_known(known_votes: &KnownVotes<B>, round: &NumberFor<B>, hash: &MessageHash) -> bool {
known_votes.get(round).map(|known| known.contains(hash)).unwrap_or(false)
}
}
impl<B> Validator<B> for GossipValidator<B>
where
B: Block,
{
fn validate(
&self,
_context: &mut dyn ValidatorContext<B>,
sender: &PeerId,
mut data: &[u8],
) -> ValidationResult<B::Hash> {
if let Ok(msg) =
VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(&mut data)
{
let msg_hash = twox_64(data);
let round = msg.commitment.block_number;
// Verify general usefulness of the message.
// We are going to discard old votes right away (without verification)
// Also we keep track of already received votes to avoid verifying duplicates.
{
let known_votes = self.known_votes.read();
if !GossipValidator::<B>::is_live(&known_votes, &round) {
return ValidationResult::Discard
}
if GossipValidator::<B>::is_known(&known_votes, &round, &msg_hash) {
return ValidationResult::ProcessAndKeep(self.topic)
}
}
if BeefyKeystore::verify(&msg.id, &msg.signature, &msg.commitment.encode()) {
GossipValidator::<B>::add_known(&mut *self.known_votes.write(), &round, msg_hash);
return ValidationResult::ProcessAndKeep(self.topic)
} else {
// TODO: report peer
debug!(target: "beefy", "🥩 Bad signature on message: {:?}, from: {:?}", msg, sender);
}
}
ValidationResult::Discard
}
fn message_expired<'a>(&'a self) -> Box<dyn FnMut(B::Hash, &[u8]) -> bool + 'a> {
let known_votes = self.known_votes.read();
Box::new(move |_topic, mut data| {
let msg = match VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(
&mut data,
) {
Ok(vote) => vote,
Err(_) => return true,
};
let round = msg.commitment.block_number;
let expired = !GossipValidator::<B>::is_live(&known_votes, &round);
trace!(target: "beefy", "🥩 Message for round #{} expired: {}", round, expired);
expired
})
}
fn message_allowed<'a>(
&'a self,
) -> Box<dyn FnMut(&PeerId, MessageIntent, &B::Hash, &[u8]) -> bool + 'a> {
let do_rebroadcast = {
let now = Instant::now();
let mut next_rebroadcast = self.next_rebroadcast.lock();
if now >= *next_rebroadcast {
*next_rebroadcast = now + REBROADCAST_AFTER;
true
} else {
false
}
};
let known_votes = self.known_votes.read();
Box::new(move |_who, intent, _topic, mut data| {
if let MessageIntent::PeriodicRebroadcast = intent {
return do_rebroadcast
}
let msg = match VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(
&mut data,
) {
Ok(vote) => vote,
Err(_) => return true,
};
let round = msg.commitment.block_number;
let allowed = GossipValidator::<B>::is_live(&known_votes, &round);
debug!(target: "beefy", "🥩 Message for round #{} allowed: {}", round, allowed);
allowed
})
}
}
substrate/client/beefy/src/gossip_tests.rs
+182
View File
@@ -0,0 +1,182 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use sc_keystore::LocalKeystore;
use sc_network_test::Block;
use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
use beefy_primitives::{crypto::Signature, Commitment, MmrRootHash, VoteMessage, KEY_TYPE};
use crate::keystore::{tests::Keyring, BeefyKeystore};
use super::*;
#[test]
fn note_round_works() {
let gv = GossipValidator::<Block>::new();
gv.note_round(1u64);
let live = gv.known_votes.read();
assert!(GossipValidator::<Block>::is_live(&live, &1u64));
drop(live);
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(!GossipValidator::<Block>::is_live(&live, &1u64));
assert!(GossipValidator::<Block>::is_live(&live, &3u64));
assert!(GossipValidator::<Block>::is_live(&live, &7u64));
assert!(GossipValidator::<Block>::is_live(&live, &10u64));
}
#[test]
fn keeps_most_recent_max_rounds() {
let gv = GossipValidator::<Block>::new();
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
gv.note_round(1u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(GossipValidator::<Block>::is_live(&live, &3u64));
assert!(!GossipValidator::<Block>::is_live(&live, &1u64));
drop(live);
gv.note_round(23u64);
gv.note_round(15u64);
gv.note_round(20u64);
gv.note_round(2u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(GossipValidator::<Block>::is_live(&live, &15u64));
assert!(GossipValidator::<Block>::is_live(&live, &20u64));
assert!(GossipValidator::<Block>::is_live(&live, &23u64));
}
#[test]
fn note_same_round_twice() {
let gv = GossipValidator::<Block>::new();
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
drop(live);
// note round #7 again -> should not change anything
gv.note_round(7u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(GossipValidator::<Block>::is_live(&live, &3u64));
assert!(GossipValidator::<Block>::is_live(&live, &7u64));
assert!(GossipValidator::<Block>::is_live(&live, &10u64));
}
struct TestContext;
impl<B: sp_runtime::traits::Block> ValidatorContext<B> for TestContext {
fn broadcast_topic(&mut self, _topic: B::Hash, _force: bool) {
todo!()
}
fn broadcast_message(&mut self, _topic: B::Hash, _message: Vec<u8>, _force: bool) {
todo!()
}
fn send_message(&mut self, _who: &sc_network::PeerId, _message: Vec<u8>) {
todo!()
}
fn send_topic(&mut self, _who: &sc_network::PeerId, _topic: B::Hash, _force: bool) {
todo!()
}
}
fn sign_commitment<BN: Encode, P: Encode>(
who: &Keyring,
commitment: &Commitment<BN, P>,
) -> Signature {
let store: SyncCryptoStorePtr = std::sync::Arc::new(LocalKeystore::in_memory());
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&who.to_seed())).unwrap();
let beefy_keystore: BeefyKeystore = Some(store).into();
beefy_keystore.sign(&who.public(), &commitment.encode()).unwrap()
}
#[test]
fn should_avoid_verifying_signatures_twice() {
let gv = GossipValidator::<Block>::new();
let sender = sc_network::PeerId::random();
let mut context = TestContext;
let commitment =
Commitment { payload: MmrRootHash::default(), block_number: 3_u64, validator_set_id: 0 };
let signature = sign_commitment(&Keyring::Alice, &commitment);
let vote = VoteMessage { commitment, id: Keyring::Alice.public(), signature };
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
// first time the cache should be populated.
let res = gv.validate(&mut context, &sender, &vote.encode());
assert!(matches!(res, ValidationResult::ProcessAndKeep(_)));
assert_eq!(gv.known_votes.read().get(&vote.commitment.block_number).map(|x| x.len()), Some(1));
// second time we should hit the cache
let res = gv.validate(&mut context, &sender, &vote.encode());
assert!(matches!(res, ValidationResult::ProcessAndKeep(_)));
// next we should quickly reject if the round is not live.
gv.note_round(11_u64);
gv.note_round(12_u64);
assert!(!GossipValidator::<Block>::is_live(
&*gv.known_votes.read(),
&vote.commitment.block_number
));
let res = gv.validate(&mut context, &sender, &vote.encode());
assert!(matches!(res, ValidationResult::Discard));
}
substrate/client/beefy/src/keystore.rs
+119
View File
@@ -0,0 +1,119 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::convert::{From, TryInto};
use sp_application_crypto::RuntimeAppPublic;
use sp_core::keccak_256;
use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
use log::warn;
use beefy_primitives::{
crypto::{Public, Signature},
KEY_TYPE,
};
use crate::error;
#[cfg(test)]
#[path = "keystore_tests.rs"]
pub mod tests;
/// A BEEFY specific keystore implemented as a `Newtype`. This is basically a
/// wrapper around [`sp_keystore::SyncCryptoStore`] and allows to customize
/// common cryptographic functionality.
pub(crate) struct BeefyKeystore(Option<SyncCryptoStorePtr>);
impl BeefyKeystore {
/// Check if the keystore contains a private key for one of the public keys
/// contained in `keys`. A public key with a matching private key is known
/// as a local authority id.
///
/// Return the public key for which we also do have a private key. If no
/// matching private key is found, `None` will be returned.
pub fn authority_id(&self, keys: &[Public]) -> Option<Public> {
let store = self.0.clone()?;
// we do check for multiple private keys as a key store sanity check.
let public: Vec<Public> = keys
.iter()
.filter(|k| SyncCryptoStore::has_keys(&*store, &[(k.to_raw_vec(), KEY_TYPE)]))
.cloned()
.collect();
if public.len() > 1 {
warn!(target: "beefy", "🥩 Multiple private keys found for: {:?} ({})", public, public.len());
}
public.get(0).cloned()
}
/// Sign `message` with the `public` key.
///
/// Note that `message` usually will be pre-hashed before being signed.
///
/// Return the message signature or an error in case of failure.
pub fn sign(&self, public: &Public, message: &[u8]) -> Result<Signature, error::Error> {
let store = self.0.clone().ok_or_else(|| error::Error::Keystore("no Keystore".into()))?;
let msg = keccak_256(message);
let public = public.as_ref();
let sig = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, public, &msg)
.map_err(|e| error::Error::Keystore(e.to_string()))?
.ok_or_else(|| error::Error::Signature("ecdsa_sign_prehashed() failed".to_string()))?;
// check that `sig` has the expected result type
let sig = sig.clone().try_into().map_err(|_| {
error::Error::Signature(format!("invalid signature {:?} for key {:?}", sig, public))
})?;
Ok(sig)
}
/// Returns a vector of [`beefy_primitives::crypto::Public`] keys which are currently supported
/// (i.e. found in the keystore).
pub fn public_keys(&self) -> Result<Vec<Public>, error::Error> {
let store = self.0.clone().ok_or_else(|| error::Error::Keystore("no Keystore".into()))?;
let pk: Vec<Public> = SyncCryptoStore::ecdsa_public_keys(&*store, KEY_TYPE)
.iter()
.map(|k| Public::from(k.clone()))
.collect();
Ok(pk)
}
/// Use the `public` key to verify that `sig` is a valid signature for `message`.
///
/// Return `true` if the signature is authentic, `false` otherwise.
pub fn verify(public: &Public, sig: &Signature, message: &[u8]) -> bool {
let msg = keccak_256(message);
let sig = sig.as_ref();
let public = public.as_ref();
sp_core::ecdsa::Pair::verify_prehashed(sig, &msg, public)
}
}
impl From<Option<SyncCryptoStorePtr>> for BeefyKeystore {
fn from(store: Option<SyncCryptoStorePtr>) -> BeefyKeystore {
BeefyKeystore(store)
}
}
substrate/client/beefy/src/keystore_tests.rs
+275
View File
@@ -0,0 +1,275 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::sync::Arc;
use sc_keystore::LocalKeystore;
use sp_core::{ecdsa, keccak_256, Pair};
use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
use beefy_primitives::{crypto, KEY_TYPE};
use super::BeefyKeystore;
use crate::error::Error;
/// Set of test accounts using [`beefy_primitives::crypto`] types.
#[allow(missing_docs)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, strum::Display, strum::EnumIter)]
pub(crate) enum Keyring {
Alice,
Bob,
Charlie,
Dave,
Eve,
Ferdie,
One,
Two,
}
impl Keyring {
/// Sign `msg`.
pub fn sign(self, msg: &[u8]) -> crypto::Signature {
let msg = keccak_256(msg);
ecdsa::Pair::from(self).sign_prehashed(&msg).into()
}
/// Return key pair.
pub fn pair(self) -> crypto::Pair {
ecdsa::Pair::from_string(self.to_seed().as_str(), None).unwrap().into()
}
/// Return public key.
pub fn public(self) -> crypto::Public {
self.pair().public()
}
/// Return seed string.
pub fn to_seed(self) -> String {
format!("//{}", self)
}
}
impl From<Keyring> for crypto::Pair {
fn from(k: Keyring) -> Self {
k.pair()
}
}
impl From<Keyring> for ecdsa::Pair {
fn from(k: Keyring) -> Self {
k.pair().into()
}
}
fn keystore() -> SyncCryptoStorePtr {
Arc::new(LocalKeystore::in_memory())
}
#[test]
fn verify_should_work() {
let msg = keccak_256(b"I am Alice!");
let sig = Keyring::Alice.sign(b"I am Alice!");
assert!(ecdsa::Pair::verify_prehashed(
&sig.clone().into(),
&msg,
&Keyring::Alice.public().into(),
));
// different public key -> fail
assert!(!ecdsa::Pair::verify_prehashed(
&sig.clone().into(),
&msg,
&Keyring::Bob.public().into(),
));
let msg = keccak_256(b"I am not Alice!");
// different msg -> fail
assert!(!ecdsa::Pair::verify_prehashed(&sig.into(), &msg, &Keyring::Alice.public().into(),));
}
#[test]
fn pair_works() {
let want = crypto::Pair::from_string("//Alice", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Alice.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Bob", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Bob.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Charlie", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Charlie.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Dave", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Dave.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Eve", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Eve.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Ferdie", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Ferdie.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//One", None).expect("Pair failed").to_raw_vec();
let got = Keyring::One.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Two", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Two.pair().to_raw_vec();
assert_eq!(want, got);
}
#[test]
fn authority_id_works() {
let store = keystore();
let alice: crypto::Public =
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Alice.to_seed()))
.ok()
.unwrap()
.into();
let bob = Keyring::Bob.public();
let charlie = Keyring::Charlie.public();
let store: BeefyKeystore = Some(store).into();
let mut keys = vec![bob, charlie];
let id = store.authority_id(keys.as_slice());
assert!(id.is_none());
keys.push(alice.clone());
let id = store.authority_id(keys.as_slice()).unwrap();
assert_eq!(id, alice);
}
#[test]
fn sign_works() {
let store = keystore();
let alice: crypto::Public =
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Alice.to_seed()))
.ok()
.unwrap()
.into();
let store: BeefyKeystore = Some(store).into();
let msg = b"are you involved or commited?";
let sig1 = store.sign(&alice, msg).unwrap();
let sig2 = Keyring::Alice.sign(msg);
assert_eq!(sig1, sig2);
}
#[test]
fn sign_error() {
let store = keystore();
let _ = SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Bob.to_seed()))
.ok()
.unwrap();
let store: BeefyKeystore = Some(store).into();
let alice = Keyring::Alice.public();
let msg = b"are you involved or commited?";
let sig = store.sign(&alice, msg).err().unwrap();
let err = Error::Signature("ecdsa_sign_prehashed() failed".to_string());
assert_eq!(sig, err);
}
#[test]
fn sign_no_keystore() {
let store: BeefyKeystore = None.into();
let alice = Keyring::Alice.public();
let msg = b"are you involved or commited";
let sig = store.sign(&alice, msg).err().unwrap();
let err = Error::Keystore("no Keystore".to_string());
assert_eq!(sig, err);
}
#[test]
fn verify_works() {
let store = keystore();
let alice: crypto::Public =
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Alice.to_seed()))
.ok()
.unwrap()
.into();
let store: BeefyKeystore = Some(store).into();
// `msg` and `sig` match
let msg = b"are you involved or commited?";
let sig = store.sign(&alice, msg).unwrap();
assert!(BeefyKeystore::verify(&alice, &sig, msg));
// `msg and `sig` don't match
let msg = b"you are just involved";
assert!(!BeefyKeystore::verify(&alice, &sig, msg));
}
// Note that we use keys with and without a seed for this test.
#[test]
fn public_keys_works() {
const TEST_TYPE: sp_application_crypto::KeyTypeId = sp_application_crypto::KeyTypeId(*b"test");
let store = keystore();
let add_key = |key_type, seed: Option<&str>| {
SyncCryptoStore::ecdsa_generate_new(&*store, key_type, seed).unwrap()
};
// test keys
let _ = add_key(TEST_TYPE, Some(Keyring::Alice.to_seed().as_str()));
let _ = add_key(TEST_TYPE, Some(Keyring::Bob.to_seed().as_str()));
let _ = add_key(TEST_TYPE, None);
let _ = add_key(TEST_TYPE, None);
// BEEFY keys
let _ = add_key(KEY_TYPE, Some(Keyring::Dave.to_seed().as_str()));
let _ = add_key(KEY_TYPE, Some(Keyring::Eve.to_seed().as_str()));
let key1: crypto::Public = add_key(KEY_TYPE, None).into();
let key2: crypto::Public = add_key(KEY_TYPE, None).into();
let store: BeefyKeystore = Some(store).into();
let keys = store.public_keys().ok().unwrap();
assert!(keys.len() == 4);
assert!(keys.contains(&Keyring::Dave.public()));
assert!(keys.contains(&Keyring::Eve.public()));
assert!(keys.contains(&key1));
assert!(keys.contains(&key2));
}
substrate/client/beefy/src/lib.rs
+159
View File
@@ -0,0 +1,159 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::sync::Arc;
use log::debug;
use prometheus::Registry;
use sc_client_api::{Backend, BlockchainEvents, Finalizer};
use sc_network_gossip::{GossipEngine, Network as GossipNetwork};
use sp_api::ProvideRuntimeApi;
use sp_blockchain::HeaderBackend;
use sp_keystore::SyncCryptoStorePtr;
use sp_runtime::traits::Block;
use beefy_primitives::BeefyApi;
mod error;
mod gossip;
mod keystore;
mod metrics;
mod round;
mod worker;
pub mod notification;
pub const BEEFY_PROTOCOL_NAME: &str = "/paritytech/beefy/1";
/// Returns the configuration value to put in
/// [`sc_network::config::NetworkConfiguration::extra_sets`].
pub fn beefy_peers_set_config() -> sc_network::config::NonDefaultSetConfig {
let mut cfg =
sc_network::config::NonDefaultSetConfig::new(BEEFY_PROTOCOL_NAME.into(), 1024 * 1024);
cfg.allow_non_reserved(25, 25);
cfg
}
/// A convenience BEEFY client trait that defines all the type bounds a BEEFY client
/// has to satisfy. Ideally that should actually be a trait alias. Unfortunately as
/// of today, Rust does not allow a type alias to be used as a trait bound. Tracking
/// issue is <https://github.com/rust-lang/rust/issues/41517>.
pub trait Client<B, BE>:
BlockchainEvents<B> + HeaderBackend<B> + Finalizer<B, BE> + ProvideRuntimeApi<B> + Send + Sync
where
B: Block,
BE: Backend<B>,
{
// empty
}
impl<B, BE, T> Client<B, BE> for T
where
B: Block,
BE: Backend<B>,
T: BlockchainEvents<B>
+ HeaderBackend<B>
+ Finalizer<B, BE>
+ ProvideRuntimeApi<B>
+ Send
+ Sync,
{
// empty
}
/// BEEFY gadget initialization parameters.
pub struct BeefyParams<B, BE, C, N>
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
N: GossipNetwork<B> + Clone + Send + 'static,
{
/// BEEFY client
pub client: Arc<C>,
/// Client Backend
pub backend: Arc<BE>,
/// Local key store
pub key_store: Option<SyncCryptoStorePtr>,
/// Gossip network
pub network: N,
/// BEEFY signed commitment sender
pub signed_commitment_sender: notification::BeefySignedCommitmentSender<B>,
/// Minimal delta between blocks, BEEFY should vote for
pub min_block_delta: u32,
/// Prometheus metric registry
pub prometheus_registry: Option<Registry>,
}
/// Start the BEEFY gadget.
///
/// This is a thin shim around running and awaiting a BEEFY worker.
pub async fn start_beefy_gadget<B, BE, C, N>(beefy_params: BeefyParams<B, BE, C, N>)
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
N: GossipNetwork<B> + Clone + Send + 'static,
{
let BeefyParams {
client,
backend,
key_store,
network,
signed_commitment_sender,
min_block_delta,
prometheus_registry,
} = beefy_params;
let gossip_validator = Arc::new(gossip::GossipValidator::new());
let gossip_engine =
GossipEngine::new(network, BEEFY_PROTOCOL_NAME, gossip_validator.clone(), None);
let metrics =
prometheus_registry.as_ref().map(metrics::Metrics::register).and_then(
|result| match result {
Ok(metrics) => {
debug!(target: "beefy", "🥩 Registered metrics");
Some(metrics)
},
Err(err) => {
debug!(target: "beefy", "🥩 Failed to register metrics: {:?}", err);
None
},
},
);
let worker_params = worker::WorkerParams {
client,
backend,
key_store: key_store.into(),
signed_commitment_sender,
gossip_engine,
gossip_validator,
min_block_delta,
metrics,
};
let worker = worker::BeefyWorker::<_, _, _>::new(worker_params);
worker.run().await
}
substrate/client/beefy/src/metrics.rs
+93
View File
@@ -0,0 +1,93 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! BEEFY Prometheus metrics definition
use prometheus::{register, Counter, Gauge, PrometheusError, Registry, U64};
/// BEEFY metrics exposed through Prometheus
pub(crate) struct Metrics {
/// Current active validator set id
pub beefy_validator_set_id: Gauge<U64>,
/// Total number of votes sent by this node
pub beefy_votes_sent: Counter<U64>,
/// Most recent concluded voting round
pub beefy_round_concluded: Gauge<U64>,
/// Best block finalized by BEEFY
pub beefy_best_block: Gauge<U64>,
/// Next block BEEFY should vote on
pub beefy_should_vote_on: Gauge<U64>,
/// Number of sessions without a signed commitment
pub beefy_skipped_sessions: Counter<U64>,
}
impl Metrics {
pub(crate) fn register(registry: &Registry) -> Result<Self, PrometheusError> {
Ok(Self {
beefy_validator_set_id: register(
Gauge::new("beefy_validator_set_id", "Current BEEFY active validator set id.")?,
registry,
)?,
beefy_votes_sent: register(
Counter::new("beefy_votes_sent", "Number of votes sent by this node")?,
registry,
)?,
beefy_round_concluded: register(
Gauge::new("beefy_round_concluded", "Voting round, that has been concluded")?,
registry,
)?,
beefy_best_block: register(
Gauge::new("beefy_best_block", "Best block finalized by BEEFY")?,
registry,
)?,
beefy_should_vote_on: register(
Gauge::new("beefy_should_vote_on", "Next block, BEEFY should vote on")?,
registry,
)?,
beefy_skipped_sessions: register(
Counter::new(
"beefy_skipped_sessions",
"Number of sessions without a signed commitment",
)?,
registry,
)?,
})
}
}
// Note: we use the `format` macro to convert an expr into a `u64`. This will fail,
// if expr does not derive `Display`.
#[macro_export]
macro_rules! metric_set {
($self:ident, $m:ident, $v:expr) => {{
let val: u64 = format!("{}", $v).parse().unwrap();
if let Some(metrics) = $self.metrics.as_ref() {
metrics.$m.set(val);
}
}};
}
#[macro_export]
macro_rules! metric_inc {
($self:ident, $m:ident) => {{
if let Some(metrics) = $self.metrics.as_ref() {
metrics.$m.inc();
}
}};
}
substrate/client/beefy/src/notification.rs
+113
View File
@@ -0,0 +1,113 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::sync::Arc;
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver, TracingUnboundedSender};
use sp_runtime::traits::{Block, NumberFor};
use parking_lot::Mutex;
/// Stream of signed commitments returned when subscribing.
pub type SignedCommitment<Block> =
beefy_primitives::SignedCommitment<NumberFor<Block>, beefy_primitives::MmrRootHash>;
/// Stream of signed commitments returned when subscribing.
type SignedCommitmentStream<Block> = TracingUnboundedReceiver<SignedCommitment<Block>>;
/// Sending endpoint for notifying about signed commitments.
type SignedCommitmentSender<Block> = TracingUnboundedSender<SignedCommitment<Block>>;
/// Collection of channel sending endpoints shared with the receiver side so they can register
/// themselves.
type SharedSignedCommitmentSenders<Block> = Arc<Mutex<Vec<SignedCommitmentSender<Block>>>>;
/// The sending half of the signed commitment channel(s).
///
/// Used to send notifications about signed commitments generated at the end of a BEEFY round.
#[derive(Clone)]
pub struct BeefySignedCommitmentSender<B>
where
B: Block,
{
subscribers: SharedSignedCommitmentSenders<B>,
}
impl<B> BeefySignedCommitmentSender<B>
where
B: Block,
{
/// The `subscribers` should be shared with a corresponding `SignedCommitmentSender`.
fn new(subscribers: SharedSignedCommitmentSenders<B>) -> Self {
Self { subscribers }
}
/// Send out a notification to all subscribers that a new signed commitment is available for a
/// block.
pub fn notify(&self, signed_commitment: SignedCommitment<B>) {
let mut subscribers = self.subscribers.lock();
// do an initial prune on closed subscriptions
subscribers.retain(|n| !n.is_closed());
if !subscribers.is_empty() {
subscribers.retain(|n| n.unbounded_send(signed_commitment.clone()).is_ok());
}
}
}
/// The receiving half of the signed commitments channel.
///
/// Used to receive notifications about signed commitments generated at the end of a BEEFY round.
/// The `BeefySignedCommitmentStream` entity stores the `SharedSignedCommitmentSenders` so it can be
/// used to add more subscriptions.
#[derive(Clone)]
pub struct BeefySignedCommitmentStream<B>
where
B: Block,
{
subscribers: SharedSignedCommitmentSenders<B>,
}
impl<B> BeefySignedCommitmentStream<B>
where
B: Block,
{
/// Creates a new pair of receiver and sender of signed commitment notifications.
pub fn channel() -> (BeefySignedCommitmentSender<B>, Self) {
let subscribers = Arc::new(Mutex::new(vec![]));
let receiver = BeefySignedCommitmentStream::new(subscribers.clone());
let sender = BeefySignedCommitmentSender::new(subscribers);
(sender, receiver)
}
/// Create a new receiver of signed commitment notifications.
///
/// The `subscribers` should be shared with a corresponding `BeefySignedCommitmentSender`.
fn new(subscribers: SharedSignedCommitmentSenders<B>) -> Self {
Self { subscribers }
}
/// Subscribe to a channel through which signed commitments are sent at the end of each BEEFY
/// voting round.
pub fn subscribe(&self) -> SignedCommitmentStream<B> {
let (sender, receiver) = tracing_unbounded("mpsc_signed_commitments_notification_stream");
self.subscribers.lock().push(sender);
receiver
}
}
substrate/client/beefy/src/round.rs
+121
View File
@@ -0,0 +1,121 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{collections::BTreeMap, hash::Hash};
use log::{debug, trace};
use beefy_primitives::{
crypto::{Public, Signature},
ValidatorSet, ValidatorSetId,
};
use sp_arithmetic::traits::AtLeast32BitUnsigned;
use sp_runtime::traits::MaybeDisplay;
#[derive(Default)]
struct RoundTracker {
votes: Vec<(Public, Signature)>,
}
impl RoundTracker {
fn add_vote(&mut self, vote: (Public, Signature)) -> bool {
// this needs to handle equivocations in the future
if self.votes.contains(&vote) {
return false
}
self.votes.push(vote);
true
}
fn is_done(&self, threshold: usize) -> bool {
self.votes.len() >= threshold
}
}
fn threshold(authorities: usize) -> usize {
let faulty = authorities.saturating_sub(1) / 3;
authorities - faulty
}
pub(crate) struct Rounds<Hash, Number> {
rounds: BTreeMap<(Hash, Number), RoundTracker>,
validator_set: ValidatorSet<Public>,
}
impl<H, N> Rounds<H, N>
where
H: Ord + Hash,
N: Ord + AtLeast32BitUnsigned + MaybeDisplay,
{
pub(crate) fn new(validator_set: ValidatorSet<Public>) -> Self {
Rounds { rounds: BTreeMap::new(), validator_set }
}
}
impl<H, N> Rounds<H, N>
where
H: Ord + Hash,
N: Ord + AtLeast32BitUnsigned + MaybeDisplay,
{
pub(crate) fn validator_set_id(&self) -> ValidatorSetId {
self.validator_set.id
}
pub(crate) fn validators(&self) -> Vec<Public> {
self.validator_set.validators.clone()
}
pub(crate) fn add_vote(&mut self, round: (H, N), vote: (Public, Signature)) -> bool {
self.rounds.entry(round).or_default().add_vote(vote)
}
pub(crate) fn is_done(&self, round: &(H, N)) -> bool {
let done = self
.rounds
.get(round)
.map(|tracker| tracker.is_done(threshold(self.validator_set.validators.len())))
.unwrap_or(false);
debug!(target: "beefy", "🥩 Round #{} done: {}", round.1, done);
done
}
pub(crate) fn drop(&mut self, round: &(H, N)) -> Option<Vec<Option<Signature>>> {
trace!(target: "beefy", "🥩 About to drop round #{}", round.1);
let signatures = self.rounds.remove(round)?.votes;
Some(
self.validator_set
.validators
.iter()
.map(|authority_id| {
signatures.iter().find_map(|(id, sig)| {
if id == authority_id {
Some(sig.clone())
} else {
None
}
})
})
.collect(),
)
}
}
substrate/client/beefy/src/worker.rs
+534
View File
@@ -0,0 +1,534 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{collections::BTreeSet, fmt::Debug, marker::PhantomData, sync::Arc};
use codec::{Codec, Decode, Encode};
use futures::{future, FutureExt, StreamExt};
use log::{debug, error, info, trace, warn};
use parking_lot::Mutex;
use sc_client_api::{Backend, FinalityNotification, FinalityNotifications};
use sc_network_gossip::GossipEngine;
use sp_api::BlockId;
use sp_arithmetic::traits::AtLeast32Bit;
use sp_runtime::{
generic::OpaqueDigestItemId,
traits::{Block, Header, NumberFor},
SaturatedConversion,
};
use beefy_primitives::{
crypto::{AuthorityId, Public, Signature},
BeefyApi, Commitment, ConsensusLog, MmrRootHash, SignedCommitment, ValidatorSet,
VersionedCommitment, VoteMessage, BEEFY_ENGINE_ID, GENESIS_AUTHORITY_SET_ID,
};
use crate::{
error,
gossip::{topic, GossipValidator},
keystore::BeefyKeystore,
metric_inc, metric_set,
metrics::Metrics,
notification, round, Client,
};
pub(crate) struct WorkerParams<B, BE, C>
where
B: Block,
{
pub client: Arc<C>,
pub backend: Arc<BE>,
pub key_store: BeefyKeystore,
pub signed_commitment_sender: notification::BeefySignedCommitmentSender<B>,
pub gossip_engine: GossipEngine<B>,
pub gossip_validator: Arc<GossipValidator<B>>,
pub min_block_delta: u32,
pub metrics: Option<Metrics>,
}
/// A BEEFY worker plays the BEEFY protocol
pub(crate) struct BeefyWorker<B, C, BE>
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
{
client: Arc<C>,
backend: Arc<BE>,
key_store: BeefyKeystore,
signed_commitment_sender: notification::BeefySignedCommitmentSender<B>,
gossip_engine: Arc<Mutex<GossipEngine<B>>>,
gossip_validator: Arc<GossipValidator<B>>,
/// Min delta in block numbers between two blocks, BEEFY should vote on
min_block_delta: u32,
metrics: Option<Metrics>,
rounds: round::Rounds<MmrRootHash, NumberFor<B>>,
finality_notifications: FinalityNotifications<B>,
/// Best block we received a GRANDPA notification for
best_grandpa_block: NumberFor<B>,
/// Best block a BEEFY voting round has been concluded for
best_beefy_block: Option<NumberFor<B>>,
/// Validator set id for the last signed commitment
last_signed_id: u64,
// keep rustc happy
_backend: PhantomData<BE>,
}
impl<B, C, BE> BeefyWorker<B, C, BE>
where
B: Block + Codec,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
{
/// Return a new BEEFY worker instance.
///
/// Note that a BEEFY worker is only fully functional if a corresponding
/// BEEFY pallet has been deployed on-chain.
///
/// The BEEFY pallet is needed in order to keep track of the BEEFY authority set.
pub(crate) fn new(worker_params: WorkerParams<B, BE, C>) -> Self {
let WorkerParams {
client,
backend,
key_store,
signed_commitment_sender,
gossip_engine,
gossip_validator,
min_block_delta,
metrics,
} = worker_params;
BeefyWorker {
client: client.clone(),
backend,
key_store,
signed_commitment_sender,
gossip_engine: Arc::new(Mutex::new(gossip_engine)),
gossip_validator,
min_block_delta,
metrics,
rounds: round::Rounds::new(ValidatorSet::empty()),
finality_notifications: client.finality_notification_stream(),
best_grandpa_block: client.info().finalized_number,
best_beefy_block: None,
last_signed_id: 0,
_backend: PhantomData,
}
}
}
impl<B, C, BE> BeefyWorker<B, C, BE>
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
{
/// Return `true`, if we should vote on block `number`
fn should_vote_on(&self, number: NumberFor<B>) -> bool {
let best_beefy_block = if let Some(block) = self.best_beefy_block {
block
} else {
debug!(target: "beefy", "🥩 Missing best BEEFY block - won't vote for: {:?}", number);
return false
};
let target = vote_target(self.best_grandpa_block, best_beefy_block, self.min_block_delta);
trace!(target: "beefy", "🥩 should_vote_on: #{:?}, next_block_to_vote_on: #{:?}", number, target);
metric_set!(self, beefy_should_vote_on, target);
number == target
}
/// Return the current active validator set at header `header`.
///
/// Note that the validator set could be `None`. This is the case if we don't find
/// a BEEFY authority set change and we can't fetch the authority set from the
/// BEEFY on-chain state.
///
/// Such a failure is usually an indication that the BEEFY pallet has not been deployed (yet).
fn validator_set(&self, header: &B::Header) -> Option<ValidatorSet<Public>> {
let new = if let Some(new) = find_authorities_change::<B>(header) {
Some(new)
} else {
let at = BlockId::hash(header.hash());
self.client.runtime_api().validator_set(&at).ok()
};
trace!(target: "beefy", "🥩 active validator set: {:?}", new);
new
}
/// Verify `active` validator set for `block` against the key store
///
/// The critical case is, if we do have a public key in the key store which is not
/// part of the active validator set.
///
/// Note that for a non-authority node there will be no keystore, and we will
/// return an error and don't check. The error can usually be ignored.
fn verify_validator_set(
&self,
block: &NumberFor<B>,
mut active: ValidatorSet<Public>,
) -> Result<(), error::Error> {
let active: BTreeSet<Public> = active.validators.drain(..).collect();
let store: BTreeSet<Public> = self.key_store.public_keys()?.drain(..).collect();
let missing: Vec<_> = store.difference(&active).cloned().collect();
if !missing.is_empty() {
debug!(target: "beefy", "🥩 for block {:?} public key missing in validator set: {:?}", block, missing);
}
Ok(())
}
fn handle_finality_notification(&mut self, notification: FinalityNotification<B>) {
trace!(target: "beefy", "🥩 Finality notification: {:?}", notification);
// update best GRANDPA finalized block we have seen
self.best_grandpa_block = *notification.header.number();
if let Some(active) = self.validator_set(&notification.header) {
// Authority set change or genesis set id triggers new voting rounds
//
// TODO: (adoerr) Enacting a new authority set will also implicitly 'conclude'
// the currently active BEEFY voting round by starting a new one. This is
// temporary and needs to be replaced by proper round life cycle handling.
if active.id != self.rounds.validator_set_id() ||
(active.id == GENESIS_AUTHORITY_SET_ID && self.best_beefy_block.is_none())
{
debug!(target: "beefy", "🥩 New active validator set id: {:?}", active);
metric_set!(self, beefy_validator_set_id, active.id);
// BEEFY should produce a signed commitment for each session
if active.id != self.last_signed_id + 1 && active.id != GENESIS_AUTHORITY_SET_ID {
metric_inc!(self, beefy_skipped_sessions);
}
// verify the new validator set
let _ = self.verify_validator_set(notification.header.number(), active.clone());
self.rounds = round::Rounds::new(active.clone());
debug!(target: "beefy", "🥩 New Rounds for id: {:?}", active.id);
self.best_beefy_block = Some(*notification.header.number());
// this metric is kind of 'fake'. Best BEEFY block should only be updated once we
// have a signed commitment for the block. Remove once the above TODO is done.
metric_set!(self, beefy_best_block, *notification.header.number());
}
}
if self.should_vote_on(*notification.header.number()) {
let authority_id = if let Some(id) =
self.key_store.authority_id(self.rounds.validators().as_slice())
{
debug!(target: "beefy", "🥩 Local authority id: {:?}", id);
id
} else {
debug!(target: "beefy", "🥩 Missing validator id - can't vote for: {:?}", notification.header.hash());
return
};
let mmr_root =
if let Some(hash) = find_mmr_root_digest::<B, Public>(&notification.header) {
hash
} else {
warn!(target: "beefy", "🥩 No MMR root digest found for: {:?}", notification.header.hash());
return
};
let commitment = Commitment {
payload: mmr_root,
block_number: notification.header.number(),
validator_set_id: self.rounds.validator_set_id(),
};
let encoded_commitment = commitment.encode();
let signature = match self.key_store.sign(&authority_id, &*encoded_commitment) {
Ok(sig) => sig,
Err(err) => {
warn!(target: "beefy", "🥩 Error signing commitment: {:?}", err);
return
},
};
trace!(
target: "beefy",
"🥩 Produced signature using {:?}, is_valid: {:?}",
authority_id,
BeefyKeystore::verify(&authority_id, &signature, &*encoded_commitment)
);
let message = VoteMessage { commitment, id: authority_id, signature };
let encoded_message = message.encode();
metric_inc!(self, beefy_votes_sent);
debug!(target: "beefy", "🥩 Sent vote message: {:?}", message);
self.handle_vote(
(message.commitment.payload, *message.commitment.block_number),
(message.id, message.signature),
);
self.gossip_engine.lock().gossip_message(topic::<B>(), encoded_message, false);
}
}
fn handle_vote(&mut self, round: (MmrRootHash, NumberFor<B>), vote: (Public, Signature)) {
self.gossip_validator.note_round(round.1);
let vote_added = self.rounds.add_vote(round, vote);
if vote_added && self.rounds.is_done(&round) {
if let Some(signatures) = self.rounds.drop(&round) {
// id is stored for skipped session metric calculation
self.last_signed_id = self.rounds.validator_set_id();
let commitment = Commitment {
payload: round.0,
block_number: round.1,
validator_set_id: self.last_signed_id,
};
let signed_commitment = SignedCommitment { commitment, signatures };
metric_set!(self, beefy_round_concluded, round.1);
info!(target: "beefy", "🥩 Round #{} concluded, committed: {:?}.", round.1, signed_commitment);
if self
.backend
.append_justification(
BlockId::Number(round.1),
(
BEEFY_ENGINE_ID,
VersionedCommitment::V1(signed_commitment.clone()).encode(),
),
)
.is_err()
{
// just a trace, because until the round lifecycle is improved, we will
// conclude certain rounds multiple times.
trace!(target: "beefy", "🥩 Failed to append justification: {:?}", signed_commitment);
}
self.signed_commitment_sender.notify(signed_commitment);
self.best_beefy_block = Some(round.1);
metric_set!(self, beefy_best_block, round.1);
}
}
}
pub(crate) async fn run(mut self) {
let mut votes = Box::pin(self.gossip_engine.lock().messages_for(topic::<B>()).filter_map(
|notification| async move {
debug!(target: "beefy", "🥩 Got vote message: {:?}", notification);
VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(
&mut &notification.message[..],
)
.ok()
},
));
loop {
let engine = self.gossip_engine.clone();
let gossip_engine = future::poll_fn(|cx| engine.lock().poll_unpin(cx));
futures::select! {
notification = self.finality_notifications.next().fuse() => {
if let Some(notification) = notification {
self.handle_finality_notification(notification);
} else {
return;
}
},
vote = votes.next().fuse() => {
if let Some(vote) = vote {
self.handle_vote(
(vote.commitment.payload, vote.commitment.block_number),
(vote.id, vote.signature),
);
} else {
return;
}
},
_ = gossip_engine.fuse() => {
error!(target: "beefy", "🥩 Gossip engine has terminated.");
return;
}
}
}
}
}
/// Extract the MMR root hash from a digest in the given header, if it exists.
fn find_mmr_root_digest<B, Id>(header: &B::Header) -> Option<MmrRootHash>
where
B: Block,
Id: Codec,
{
header.digest().logs().iter().find_map(|log| {
match log.try_to::<ConsensusLog<Id>>(OpaqueDigestItemId::Consensus(&BEEFY_ENGINE_ID)) {
Some(ConsensusLog::MmrRoot(root)) => Some(root),
_ => None,
}
})
}
/// Scan the `header` digest log for a BEEFY validator set change. Return either the new
/// validator set or `None` in case no validator set change has been signaled.
fn find_authorities_change<B>(header: &B::Header) -> Option<ValidatorSet<AuthorityId>>
where
B: Block,
{
let id = OpaqueDigestItemId::Consensus(&BEEFY_ENGINE_ID);
let filter = |log: ConsensusLog<AuthorityId>| match log {
ConsensusLog::AuthoritiesChange(validator_set) => Some(validator_set),
_ => None,
};
header.digest().convert_first(|l| l.try_to(id).and_then(filter))
}
/// Calculate next block number to vote on
fn vote_target<N>(best_grandpa: N, best_beefy: N, min_delta: u32) -> N
where
N: AtLeast32Bit + Copy + Debug,
{
let diff = best_grandpa.saturating_sub(best_beefy);
let diff = diff.saturated_into::<u32>();
let target = best_beefy + min_delta.max(diff.next_power_of_two()).into();
trace!(
target: "beefy",
"🥩 vote target - diff: {:?}, next_power_of_two: {:?}, target block: #{:?}",
diff,
diff.next_power_of_two(),
target,
);
target
}
#[cfg(test)]
mod tests {
use super::vote_target;
#[test]
fn vote_on_min_block_delta() {
let t = vote_target(1u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(2u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(3u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(4u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(4u32, 4, 4);
assert_eq!(8, t);
let t = vote_target(10u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(11u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(12u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(13u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(10u32, 10, 8);
assert_eq!(18, t);
let t = vote_target(11u32, 10, 8);
assert_eq!(18, t);
let t = vote_target(12u32, 10, 8);
assert_eq!(18, t);
let t = vote_target(13u32, 10, 8);
assert_eq!(18, t);
}
#[test]
fn vote_on_power_of_two() {
let t = vote_target(1008u32, 1000, 4);
assert_eq!(1008, t);
let t = vote_target(1016u32, 1000, 4);
assert_eq!(1016, t);
let t = vote_target(1032u32, 1000, 4);
assert_eq!(1032, t);
let t = vote_target(1064u32, 1000, 4);
assert_eq!(1064, t);
let t = vote_target(1128u32, 1000, 4);
assert_eq!(1128, t);
let t = vote_target(1256u32, 1000, 4);
assert_eq!(1256, t);
let t = vote_target(1512u32, 1000, 4);
assert_eq!(1512, t);
let t = vote_target(1024u32, 0, 4);
assert_eq!(1024, t);
}
#[test]
fn vote_on_target_block() {
let t = vote_target(1008u32, 1002, 4);
assert_eq!(1010, t);
let t = vote_target(1010u32, 1002, 4);
assert_eq!(1010, t);
let t = vote_target(1016u32, 1006, 4);
assert_eq!(1022, t);
let t = vote_target(1022u32, 1006, 4);
assert_eq!(1022, t);
let t = vote_target(1032u32, 1012, 4);
assert_eq!(1044, t);
let t = vote_target(1044u32, 1012, 4);
assert_eq!(1044, t);
let t = vote_target(1064u32, 1014, 4);
assert_eq!(1078, t);
let t = vote_target(1078u32, 1014, 4);
assert_eq!(1078, t);
let t = vote_target(1128u32, 1008, 4);
assert_eq!(1136, t);
let t = vote_target(1136u32, 1008, 4);
assert_eq!(1136, t);
}
}