Integrate BEEFY (#9833)

* Initial project setup and skeleton (#4)

* initial project setup for beefy gadget client

* update editorconfig

* update gitignore

* add initial skeleton for beefy gadget worker

* add skeleton for gossip processing

* add app crypto

* move around some code

* add basic flow for voting

* add logic for picking blocks to sign

* add rustfmt config

* add example node with beefy gadget

* use u32::next_power_of_two

* make maximum periodicity configurable

* add copyright header

* rename max_periodicity to min_interval

* CI stuff (#5)

* CI stuff.

* Fix workspace.

* cargo fmt --all

* Add license for beefy-gadget

* One toolchain to rule them all.

* Clippy.

* Fix clippy.

* Clippy in the runtime.

* Fix clippy grumbles.

* cargo fmt --all

* Primitives & Light Client examples (#8)

* Primitives.

* Docs.

* Document primitives.

* Simple tests.

* Light client examples.

* Fix stuff.

* cargo fmt --all

* Add a bunch of tests for imports.

* Add more examples.

* cargo fmt --all

* Fix clippy.

* cargo fmt --all

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Add GRANDPA / FG clarifications.

* Fix min number of signatures.

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update to substrate master (#22)

* update to substrate master

* update dependencies

* fix clippy issues

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Add beefy pallet (#25)

* move beefy application crypto to primitives

* make primitives compile under no_std

* add beefy pallet that maintains authority set

* add beefy pallet to node example runtime

* tabify node-example cargo.toml files

* use double quotes in Cargo.toml files

* add missing hex-literal dependency

* add runtime api to fetch BEEFY authorities

* fix clippy warnings

* rename beefy-pallet to pallet-beefy

* sort dependencies in node-example/runtime/Cargo.toml

* Signed commitments rpc pubsub (#26)

* move beefy application crypto to primitives

* make primitives compile under no_std

* add beefy pallet that maintains authority set

* add beefy pallet to node example runtime

* tabify node-example cargo.toml files

* use double quotes in Cargo.toml files

* add missing hex-literal dependency

* add runtime api to fetch BEEFY authorities

* fix clippy warnings

* gadget: use commitment and signedcommitment

* gadget: send notifications for signed commitments

* gadget: add rpc pubsub for signed commitments

* node-example: enable beefy rpc

* gadget: fix clippy warnings

* rename beefy-pallet to pallet-beefy

* sort dependencies in node-example/runtime/Cargo.toml

* gadget: add documentation on SignedCommitment rpc wrapper type

* gadget: add todos about dummy beefy commitments

* gadget: remove redundant closure

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Integrate MMR and deposit root into the digest. (#24)

* Add basic MMR.

* Deposit digest item.

* cargo fmt --all

* Merge with primitives.

* cargo fmt --all

* Fix extra spaces.

* cargo fmt --all

* Switch branch.

* remove stray whitespace

* update to latest td-mmr commit

* fix clippy error

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* use new mmr root as commitment payload (#27)

* use new mmr root as commitment payload

* fix mmr root codec index

* warn on MMR root digest not found

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* add type alias for MMR root hash

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Bump serde_json from 1.0.59 to 1.0.60 (#28)

* Update to latest substrate. (#32)

* Update to latest substrate.

* Fix tests.

* cargo fmt --all

* Switch to master.

* Bump serde from 1.0.117 to 1.0.118 (#29)

* Bump serde from 1.0.117 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.117 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.117...v1.0.118)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump arc-swap.

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Remove transition flag (#35)

* Get rid of is_set_transition_flag

* Fix tests.

* cargo fmt --all

* Bump futures from 0.3.9 to 0.3.12 (#50)

* Bump log from 0.4.11 to 0.4.13 (#52)

* Bump Substrate and Deps (#57)

* Update README (#58)

* Update README

* Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* address review comments

* missed a typo

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Add validator set to the pallet. (#65)

* Bump Substrate and Deps (#71)

* Bump Substrate and Deps

* pin serde and syn

* bump Substrate again for '__Nonexhaustive' fix

* add cargo deny ignore

* Beefy pallet test (#74)

* setup mock

* test session change

* silence beefy

* clippy still

* no change - no log

* clippy again

* Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* code review changes, added additional test

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Beefy node cleanup (#75)

* bump serde

* bump substrate, scale-codec 2.0.0

* we need a proper beefy node

* rename primitives as well

* Sort members.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Migrate beefy-pallet to FRAMEv2 (#76)

* migrate beefy-pallet to FRAMEv2

* Code review

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Run BEEFY worker as non-validator (#77)

* run BEEFY worker as non-validator

* don't check for roloe.is_authority

* change enum type name

* Bump Substrate and Deps (#79)

* Add BEEFY gadget as extra peer set (#80)

* Add BEEFY gadget as extra peer set

* use BEEFY protocol

* Add ValidatorSetId to BEEFY digest (#85)

* add ValidatorSetId to BEEFY digest

* apply review changes

* Bump Substrate and Deps (#91)

* Bump Substrate and Deps

* Bump Substrate again in order to include a hot-fix

* redo again

* use CryptoStore issue

* cargo fmt

* Bump serde_json from 1.0.63 to 1.0.64 (#93)

* Track BEEFY validator set (#94)

* Track BEEFY validator set

* Add validator_set_id to BeefyWorker

* Make validattor_set_id optional

* Ad 92 (#97)

* sign_commitment()

* Error handling todo

* Add error type (#99)

* Add error type

* Address review

* Extract worker and round logic (#104)

* Bump serde from 1.0.123 to 1.0.124 (#106)

* Rework BeefyAPI (#110)

* Initialize BeefyWorker with current validator set (#111)

* Update toolchain (#115)

* Use nightly toolchain

* dongradde to latest clippy stable

* GH workflow trail and error

* next try

* use stable for clippy

* update wasm builder

* yet another try

* fun with CI

* no env var

* and one more

* allow from_over_into bco contruct_runtime

* back to start

* well ...

* full circle

* old version was still used

* Bump Substrate and Deps (#117)

* Bump Substrate and Deps

* cargo fmt should enforce uniform imports

* merge some imports

* Delayed BEEFY worker initialization (#121)

* lifecycle state

* add Client convenience trait

* rework trait identifiers

* WIP

* rework BeefyWorker::new() signature

* Delayed BEEFY gadget initialization

* address review

* Bump substrate. (#123)

* Bump substrate.

* Fix tests.

* Lower log-level for a missing validator set (#124)

* lower log-level for a missing validator set

* move best_finalized_block initialization

* Setup Prometheus metrics (#125)

* setup Prometheus metrics

* expose validator set id

* cargo fmt

* Update beefy-gadget/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* add vote messages gossiped metric

* track authorities change, before checking for MMR root digest

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Make Client convenience trait public (#126)

* Bump serde from 1.0.124 to 1.0.125 (#131)

* Reset rounds on new validator set. (#133)

* Re-set rounds on new validator set.

* Fix docs.

* Bump Substrate and Deps (#134)

* beefy: authority set changes fixes (#139)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type (#140)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* gadget: reduce gossip spam (#141)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type

* gadget: only gossip last 5 rounds

* gadget: note round to gossip validator before gossiping message

* gadget: fix clippy warnings

* gadget: update docs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* gadget: verify SignedCommitment message signature (#142)

* gadget: verify SignedCommitment message signature

* gadget: log messages with bad sigs

* gadget: move todo comment

* Bump futures from 0.3.13 to 0.3.14 (#145)

* Milestone 1 (#144)

* use best_finalized, prevent race

* make best_finalized_block an Option, should_vote_on bails on None

* Bump futures from 0.3.13 to 0.3.14

* Revert futures bump

* Revert "Revert futures bump"

This reverts commit a1b5e7e9bac526f2897ebfdfee7f02dd29a13ac5.

* Revert "Bump futures from 0.3.13 to 0.3.14"

This reverts commit a4e508b118ad2c4b52909d24143c284073961458.

* debug msg if the bail voting

* validator_set()

* local_id()

* get rid of worker state

* Apply review suggestions

* fix should_vote_on()

* Extract BeefyGossipValidator (#147)

* Extract BeefyGossipValidator

* Apply review suggestions

* Add block_delta parameter to start_beefy_gadget (#151)

* Add block_delta parameter

* rename to min_block_delta

* Add additional metrics (#152)

* Add additional metrics

* add skipped session metric

* add some comment for temp metric

* don't log under info for every concluded round (#156)

* don't log error on missing validator keys (#157)

* don't log error on missing validator keys

* remove unused import

* Fix validator set change handling (#158)

* reduce some logs from debug to trace

* fix validator set changes handling

* rename validator module to gossip

* run rustfmt

* Fix should_vote_on() (#160)

* Fix should_vote_on()

* by the textbook

* fix the algorithm

* Apply review suggestions

* don't use NumberFor in vote_target

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* Make KeyStore optional (#173)

* Use builder pattern for NonDefaultSetConfig (#178)

Co-authored-by: adoerr <0xad@gmx.net>

* Append SignedCommitment to block justifications (#177)

* Append SignedCommitment

* add BeefyParams

* add WorkerParams

* use warn

* versioned variant for SignedCommitment

* Bump serde from 1.0.125 to 1.0.126 (#184)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.125 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.125...v1.0.126)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump strum from 0.20.0 to 0.21.0 (#195)

* Bump strum from 0.20.0 to 0.21.0

Bumps [strum](https://github.com/Peternator7/strum) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use dervie feature for strum; clippy and deny housekeeping

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Make concluded round an info log (#200)

* Remove external crypto trait bounds (#207)

* BeefyKeystore newtype

* WIP

* remove mod ecdsa

* WIP

* fix tests

* some polishing

* Rename AuthorityId to BeefyId to avoid type conflict in UI (#211)

* Add trace points; Reduce MAX_LIVE_GOSSIP_ROUNDS (#210)

* Add trace points; Reduce MAX_LIVE_GOSSIP_ROUNDS

* log local authority id

* Additional initial authority id's (#217)

* Scratch concluded rounds

* adjust testnet doc

* fix authority key typo

* We don't want no scratches

* address review comments

* Fix note_round() (#219)

* rename BeefyGossipValidator

* Fix note_round()

* use const for assert

* put message trace points back in

* test case note_same_round_twice()

* address review comments

* remove redundant check

* Use LocalKeystore for tests (#224)

* private_keys()

* Use LocalKeystore for tests

* Use keystore helper

* Address review

* some reformatting

* Cache known votes in gossip (#227)

* Implement known messages cache.

* Add tests.

* Appease clippy.

* More clippy

Co-authored-by: adoerr <0xad@gmx.net>

* Some key store sanity checks (#232)

* verify vote message

* verify_validator_set()

* rework logging

* some rework

* Tone down warnings.

* Add signature verification.

* Tone down more.

* Fix clippy

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Use Binary Merkle Tree instead of a trie (#225)

* Binary tree merkle root.

* Add proofs and verification.

* Clean up debug.

* Use BEEFY addresses instead of pubkeys.

* Use new merkle tree.

* Optimize allocations.

* Add test for larger trees.

* Add tests for larger cases.

* Appease clippy

* Appease clippy2.

* Fix proof generation & verification.

* Add more test data.

* Fix CLI.

* Update README

* Bump version.

* Update docs.

* Rename beefy-merkle-root to beefy-merkle-tree

Co-authored-by: adoerr <0xad@gmx.net>

* Bump Substrate and Deps (#235)

* BEEFY+MMR pallet (#236)

* Add MMR leaf format to primitives.

* Fix tests

* Initial work on the BEEFY-MMR pallet.

* Add tests to MMR pallet.

* Use eth addresses.

* Use binary merkle tree.

* Bump libsecp256k1

* Fix compilation.

* Bump deps.

* Appease cargo deny.

* Re-format.

* Module-level docs.

* no-std fix.

* update README

Co-authored-by: adoerr <0xad@gmx.net>

* Fix noting rounds for non-authorities (#238)

* Bump env_logger from 0.8.4 to 0.9.0 (#242)

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.4 to 0.9.0.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.4...v0.9.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* gadget: add global timeout for rebroadcasting messages (#243)

* gadget: add global timeout for rebroadcasting messages

* update rustfmt.toml

* make message_allowed() a debug trace

Co-authored-by: adoerr <0xad@gmx.net>

* Bump Substrate and Deps (#245)

* Bump Substrate and Deps

* Bump Substrate again

* Bump futures from 0.3.15 to 0.3.16 (#247)

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump libsecp256k1 from 0.5.0 to 0.6.0 (#249)

* Bump libsecp256k1 from 0.5.0 to 0.6.0

Bumps [libsecp256k1](https://github.com/paritytech/libsecp256k1) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/paritytech/libsecp256k1/releases)
- [Changelog](https://github.com/paritytech/libsecp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/libsecp256k1/commits)

---
updated-dependencies:
- dependency-name: libsecp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use correct crate name

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Derive `scale_info::TypeInfo` for types used in polkadot (#218)

* Add scale-info TypeInfo derives

* Update scale-info

* Add crates.io patches

* Use substrate aj-metadata-vnext branch

* Revert master branch substrate deps

* Add scale-info to beefy-pallet

* scale-info v0.9.0

* Remove github dependencies and patches

* More TypeInfo derives

* Update scale-info to 0.10.0

* Add missing scale-info dependency

* Add missing TypeInfo derive

* Hide TypeInfo under a feature.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Bump serde from 1.0.126 to 1.0.127 (#260)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Substrate and Deps (#262)

* Update jsonrpc (#265)

* Update jsonrpc

* Update Substrate

* bump Substrate and Deps (#268)

* Bump serde from 1.0.127 to 1.0.128 (#272)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix spelling (#271)

* Bump serde from 1.0.128 to 1.0.130 (#276)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.128 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.130)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump scale-info from 0.10.0 to 0.12.0 (#275)

Bumps [scale-info](https://github.com/paritytech/scale-info) from 0.10.0 to 0.12.0.
- [Release notes](https://github.com/paritytech/scale-info/releases)
- [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/scale-info/commits)

---
updated-dependencies:
- dependency-name: scale-info
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Update to scale-info 1.0 (#278)

* bump substrate (#282)

* bump Substrate and Deps

* cargo fmt

Co-authored-by: Wenfeng Wang <kalot.wang@gmail.com>

* Update worker.rs (#287)

* Bump anyhow from 1.0.43 to 1.0.44 (#290)

* Bump anyhow from 1.0.43 to 1.0.44

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.43 to 1.0.44.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.43...1.0.44)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* derive Default

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Remove optional `scale-info` feature (#292)

* Make scale-info dependency non-optional

* Remove feature gated TypeInfo derives

* Import TypeInfo

* Update substrate

* Fix up runtime

* prune .git suffix (#294)

* remove unused deps (#295)

* remove unused deps

* update lock file

* Bump libsecp256k1 from 0.6.0 to 0.7.0 (#296)

* Bump libsecp256k1 from 0.6.0 to 0.7.0

Bumps [libsecp256k1](https://github.com/paritytech/libsecp256k1) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/paritytech/libsecp256k1/releases)
- [Changelog](https://github.com/paritytech/libsecp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/libsecp256k1/commits)

---
updated-dependencies:
- dependency-name: libsecp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update sec advisories

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* clean compile

* use path dependencies

* beefy-gadget license header

* pallet-beefy license header

* pallet-beefy-mmr license header

* beefy-primitves license header

* carg fmt

* more formatting

* shorten line

* downgrade parity-scale-codec to 2.2.0

* use path dependency for Prometheus endpoint

* remove clippy annotations

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: drewstone <drewstone329@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Wenfeng Wang <kalot.wang@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
This commit is contained in:
Andreas Doerr
2021-09-23 21:02:30 +02:00
committed by GitHub
parent 6845666c5c
commit 283c8daa81
31 changed files with 4992 additions and 12 deletions
+179 -12
View File
@@ -492,6 +492,80 @@ dependencies = [
"serde",
]
[[package]]
name = "beefy-gadget"
version = "4.0.0-dev"
dependencies = [
"beefy-primitives",
"fnv",
"futures 0.3.16",
"log 0.4.14",
"parity-scale-codec",
"parking_lot 0.11.1",
"sc-client-api",
"sc-keystore",
"sc-network",
"sc-network-gossip",
"sc-network-test",
"sc-utils",
"sp-api",
"sp-application-crypto",
"sp-arithmetic",
"sp-blockchain",
"sp-core",
"sp-keystore",
"sp-runtime",
"strum 0.21.0",
"substrate-prometheus-endpoint",
"thiserror",
"wasm-timer",
]
[[package]]
name = "beefy-gadget-rpc"
version = "4.0.0-dev"
dependencies = [
"beefy-gadget",
"beefy-primitives",
"futures 0.3.16",
"jsonrpc-core",
"jsonrpc-core-client",
"jsonrpc-derive",
"jsonrpc-pubsub",
"log 0.4.14",
"parity-scale-codec",
"sc-rpc",
"serde",
"sp-core",
"sp-runtime",
]
[[package]]
name = "beefy-merkle-tree"
version = "4.0.0-dev"
dependencies = [
"env_logger 0.9.0",
"hex",
"hex-literal",
"log 0.4.14",
"tiny-keccak",
]
[[package]]
name = "beefy-primitives"
version = "4.0.0-dev"
dependencies = [
"hex-literal",
"parity-scale-codec",
"scale-info",
"sp-api",
"sp-application-crypto",
"sp-core",
"sp-keystore",
"sp-runtime",
"sp-std",
]
[[package]]
name = "bincode"
version = "1.3.2"
@@ -3751,9 +3825,9 @@ dependencies = [
"base64 0.12.3",
"digest 0.9.0",
"hmac-drbg 0.3.0",
"libsecp256k1-core",
"libsecp256k1-gen-ecmult",
"libsecp256k1-gen-genmult",
"libsecp256k1-core 0.2.2",
"libsecp256k1-gen-ecmult 0.2.1",
"libsecp256k1-gen-genmult 0.2.1",
"rand 0.7.3",
"serde",
"sha2 0.9.3",
@@ -3770,15 +3844,32 @@ dependencies = [
"base64 0.12.3",
"digest 0.9.0",
"hmac-drbg 0.3.0",
"libsecp256k1-core",
"libsecp256k1-gen-ecmult",
"libsecp256k1-gen-genmult",
"libsecp256k1-core 0.2.2",
"libsecp256k1-gen-ecmult 0.2.1",
"libsecp256k1-gen-genmult 0.2.1",
"rand 0.7.3",
"serde",
"sha2 0.9.3",
"typenum",
]
[[package]]
name = "libsecp256k1"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0452aac8bab02242429380e9b2f94ea20cea2b37e2c1777a1358799bbe97f37"
dependencies = [
"arrayref",
"base64 0.13.0",
"digest 0.9.0",
"libsecp256k1-core 0.3.0",
"libsecp256k1-gen-ecmult 0.3.0",
"libsecp256k1-gen-genmult 0.3.0",
"rand 0.8.4",
"serde",
"sha2 0.9.3",
]
[[package]]
name = "libsecp256k1-core"
version = "0.2.2"
@@ -3790,13 +3881,33 @@ dependencies = [
"subtle 2.4.0",
]
[[package]]
name = "libsecp256k1-core"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5be9b9bb642d8522a44d533eab56c16c738301965504753b03ad1de3425d5451"
dependencies = [
"crunchy",
"digest 0.9.0",
"subtle 2.4.0",
]
[[package]]
name = "libsecp256k1-gen-ecmult"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccab96b584d38fac86a83f07e659f0deafd0253dc096dab5a36d53efe653c5c3"
dependencies = [
"libsecp256k1-core",
"libsecp256k1-core 0.2.2",
]
[[package]]
name = "libsecp256k1-gen-ecmult"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3038c808c55c87e8a172643a7d87187fc6c4174468159cb3090659d55bcb4809"
dependencies = [
"libsecp256k1-core 0.3.0",
]
[[package]]
@@ -3805,7 +3916,16 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67abfe149395e3aa1c48a2beb32b068e2334402df8181f818d3aee2b304c4f5d"
dependencies = [
"libsecp256k1-core",
"libsecp256k1-core 0.2.2",
]
[[package]]
name = "libsecp256k1-gen-genmult"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3db8d6ba2cec9eacc40e6e8ccc98931840301f1006e95647ceb2dd5c3aa06f7c"
dependencies = [
"libsecp256k1-core 0.3.0",
]
[[package]]
@@ -5114,6 +5234,50 @@ dependencies = [
"sp-std",
]
[[package]]
name = "pallet-beefy"
version = "4.0.0-dev"
dependencies = [
"beefy-primitives",
"frame-support",
"frame-system",
"pallet-session",
"parity-scale-codec",
"scale-info",
"serde",
"sp-core",
"sp-io",
"sp-runtime",
"sp-staking",
"sp-std",
]
[[package]]
name = "pallet-beefy-mmr"
version = "4.0.0-dev"
dependencies = [
"beefy-merkle-tree",
"beefy-primitives",
"frame-support",
"frame-system",
"hex",
"hex-literal",
"libsecp256k1 0.7.0",
"log 0.4.14",
"pallet-beefy",
"pallet-mmr",
"pallet-mmr-primitives",
"pallet-session",
"parity-scale-codec",
"scale-info",
"serde",
"sp-core",
"sp-io",
"sp-runtime",
"sp-staking",
"sp-std",
]
[[package]]
name = "pallet-bounties"
version = "4.0.0-dev"
@@ -8629,9 +8793,9 @@ dependencies = [
[[package]]
name = "serde"
version = "1.0.126"
version = "1.0.130"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec7505abeacaec74ae4778d9d9328fe5a5d04253220a85c4ee022239fc996d03"
checksum = "f12d06de37cf59146fbdecab66aa99f9fe4f78722e3607577a5375d66bd0c913"
dependencies = [
"serde_derive",
]
@@ -8657,9 +8821,9 @@ dependencies = [
[[package]]
name = "serde_derive"
version = "1.0.126"
version = "1.0.130"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "963a7dbc9895aeac7ac90e74f34a5d5261828f79df35cbed41e10189d3804d43"
checksum = "d7bc1a1ab1961464eae040d96713baa5a724a8152c1222492465b54322ec508b"
dependencies = [
"proc-macro2",
"quote",
@@ -9779,6 +9943,9 @@ name = "strum"
version = "0.21.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aaf86bbcfd1fa9670b7a129f64fc0c9fcbbfe4f1bc4210e9e98fe71ffc12cde2"
dependencies = [
"strum_macros 0.21.1",
]
[[package]]
name = "strum_macros"
+6
View File
@@ -19,6 +19,8 @@ members = [
"client/api",
"client/authority-discovery",
"client/basic-authorship",
"client/beefy",
"client/beefy/rpc",
"client/block-builder",
"client/chain-spec",
"client/chain-spec/derive",
@@ -69,6 +71,9 @@ members = [
"frame/authorship",
"frame/babe",
"frame/balances",
"frame/beefy",
"frame/beefy-mmr",
"frame/beefy-mmr/primitives",
"frame/benchmarking",
"frame/bounties",
"frame/collective",
@@ -138,6 +143,7 @@ members = [
"primitives/arithmetic/fuzzer",
"primitives/authority-discovery",
"primitives/authorship",
"primitives/beefy",
"primitives/block-builder",
"primitives/blockchain",
"primitives/consensus/aura",
+38
View File
@@ -0,0 +1,38 @@
[package]
name = "beefy-gadget"
version = "4.0.0-dev"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
license = "GPL-3.0-or-later WITH Classpath-exception-2.0"
[dependencies]
fnv = "1.0.6"
futures = "0.3"
log = "0.4"
parking_lot = "0.11"
thiserror = "1.0"
wasm-timer = "0.2.5"
codec = { version = "2.2.0", package = "parity-scale-codec", features = ["derive"] }
prometheus = { version = "0.9.0", package = "substrate-prometheus-endpoint", path = "../../utils/prometheus" }
sp-api = { version = "4.0.0-dev", path = "../../primitives/api" }
sp-application-crypto = { version = "4.0.0-dev", path = "../../primitives/application-crypto" }
sp-arithmetic = { version = "4.0.0-dev", path = "../../primitives/arithmetic" }
sp-blockchain = { version = "4.0.0-dev", path = "../../primitives/blockchain" }
sp-core = { version = "4.0.0-dev", path = "../../primitives/core" }
sp-keystore = { version = "0.10.0-dev", path = "../../primitives/keystore" }
sp-runtime = { version = "4.0.0-dev", path = "../../primitives/runtime" }
sc-utils = { version = "4.0.0-dev", path = "../utils" }
sc-client-api = { version = "4.0.0-dev", path = "../api" }
sc-keystore = { version = "4.0.0-dev", path = "../keystore" }
sc-network = { version = "0.10.0-dev", path = "../network" }
sc-network-gossip = { version = "0.10.0-dev", path = "../network-gossip" }
beefy-primitives = { version = "4.0.0-dev", path = "../../primitives/beefy" }
[dev-dependencies]
sc-network-test = { version = "0.8.0", path = "../network/test" }
strum = { version = "0.21", features = ["derive"] }
+26
View File
@@ -0,0 +1,26 @@
[package]
name = "beefy-gadget-rpc"
version = "4.0.0-dev"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
license = "GPL-3.0-or-later WITH Classpath-exception-2.0"
[dependencies]
futures = "0.3.16"
log = "0.4"
serde = { version = "1.0.130", features = ["derive"] }
jsonrpc-core = "18.0.0"
jsonrpc-core-client = "18.0.0"
jsonrpc-derive = "18.0.0"
jsonrpc-pubsub = "18.0.0"
codec = { version = "2.2.0", package = "parity-scale-codec", features = ["derive"] }
sc-rpc = { version = "4.0.0-dev", path = "../../rpc" }
sp-core = { version = "4.0.0-dev", path = "../../../primitives/core" }
sp-runtime = { versin = "4.0.0-dev", path = "../../../primitives/runtime" }
beefy-gadget = { version = "4.0.0-dev", path = "../." }
beefy-primitives = { version = "4.0.0-dev", path = "../../../primitives/beefy" }
+114
View File
@@ -0,0 +1,114 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! RPC API for BEEFY.
#![warn(missing_docs)]
use std::sync::Arc;
use sp_runtime::traits::Block as BlockT;
use futures::{FutureExt, SinkExt, StreamExt};
use jsonrpc_derive::rpc;
use jsonrpc_pubsub::{manager::SubscriptionManager, typed::Subscriber, SubscriptionId};
use log::warn;
use beefy_gadget::notification::BeefySignedCommitmentStream;
mod notification;
/// Provides RPC methods for interacting with BEEFY.
#[rpc]
pub trait BeefyApi<Notification, Hash> {
/// RPC Metadata
type Metadata;
/// Returns the block most recently finalized by BEEFY, alongside side its justification.
#[pubsub(
subscription = "beefy_justifications",
subscribe,
name = "beefy_subscribeJustifications"
)]
fn subscribe_justifications(
&self,
metadata: Self::Metadata,
subscriber: Subscriber<Notification>,
);
/// Unsubscribe from receiving notifications about recently finalized blocks.
#[pubsub(
subscription = "beefy_justifications",
unsubscribe,
name = "beefy_unsubscribeJustifications"
)]
fn unsubscribe_justifications(
&self,
metadata: Option<Self::Metadata>,
id: SubscriptionId,
) -> jsonrpc_core::Result<bool>;
}
/// Implements the BeefyApi RPC trait for interacting with BEEFY.
pub struct BeefyRpcHandler<Block: BlockT> {
signed_commitment_stream: BeefySignedCommitmentStream<Block>,
manager: SubscriptionManager,
}
impl<Block: BlockT> BeefyRpcHandler<Block> {
/// Creates a new BeefyRpcHandler instance.
pub fn new<E>(signed_commitment_stream: BeefySignedCommitmentStream<Block>, executor: E) -> Self
where
E: futures::task::Spawn + Send + Sync + 'static,
{
let manager = SubscriptionManager::new(Arc::new(executor));
Self { signed_commitment_stream, manager }
}
}
impl<Block> BeefyApi<notification::SignedCommitment, Block> for BeefyRpcHandler<Block>
where
Block: BlockT,
{
type Metadata = sc_rpc::Metadata;
fn subscribe_justifications(
&self,
_metadata: Self::Metadata,
subscriber: Subscriber<notification::SignedCommitment>,
) {
let stream = self
.signed_commitment_stream
.subscribe()
.map(|x| Ok::<_, ()>(Ok(notification::SignedCommitment::new::<Block>(x))));
self.manager.add(subscriber, |sink| {
stream
.forward(sink.sink_map_err(|e| warn!("Error sending notifications: {:?}", e)))
.map(|_| ())
});
}
fn unsubscribe_justifications(
&self,
_metadata: Option<Self::Metadata>,
id: SubscriptionId,
) -> jsonrpc_core::Result<bool> {
Ok(self.manager.cancel(id))
}
}
@@ -0,0 +1,39 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use codec::Encode;
use serde::{Deserialize, Serialize};
use sp_runtime::traits::Block as BlockT;
/// An encoded signed commitment proving that the given header has been finalized.
/// The given bytes should be the SCALE-encoded representation of a
/// `beefy_primitives::SignedCommitment`.
#[derive(Clone, Serialize, Deserialize)]
pub struct SignedCommitment(sp_core::Bytes);
impl SignedCommitment {
pub fn new<Block>(
signed_commitment: beefy_gadget::notification::SignedCommitment<Block>,
) -> Self
where
Block: BlockT,
{
SignedCommitment(signed_commitment.encode().into())
}
}
+31
View File
@@ -0,0 +1,31 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! BEEFY gadget specific errors
//!
//! Used for BEEFY gadget interal error handling only
use std::fmt::Debug;
#[derive(Debug, thiserror::Error, PartialEq)]
pub enum Error {
#[error("Keystore error: {0}")]
Keystore(String),
#[error("Signature error: {0}")]
Signature(String),
}
+236
View File
@@ -0,0 +1,236 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{collections::BTreeMap, time::Duration};
use sc_network::PeerId;
use sc_network_gossip::{MessageIntent, ValidationResult, Validator, ValidatorContext};
use sp_core::hashing::twox_64;
use sp_runtime::traits::{Block, Hash, Header, NumberFor};
use codec::{Decode, Encode};
use log::{debug, trace};
use parking_lot::{Mutex, RwLock};
use wasm_timer::Instant;
use beefy_primitives::{
crypto::{Public, Signature},
MmrRootHash, VoteMessage,
};
use crate::keystore::BeefyKeystore;
#[cfg(test)]
#[path = "gossip_tests.rs"]
mod tests;
// Limit BEEFY gossip by keeping only a bound number of voting rounds alive.
const MAX_LIVE_GOSSIP_ROUNDS: usize = 3;
// Timeout for rebroadcasting messages.
const REBROADCAST_AFTER: Duration = Duration::from_secs(60 * 5);
/// Gossip engine messages topic
pub(crate) fn topic<B: Block>() -> B::Hash
where
B: Block,
{
<<B::Header as Header>::Hashing as Hash>::hash(b"beefy")
}
/// A type that represents hash of the message.
pub type MessageHash = [u8; 8];
type KnownVotes<B> = BTreeMap<NumberFor<B>, fnv::FnvHashSet<MessageHash>>;
/// BEEFY gossip validator
///
/// Validate BEEFY gossip messages and limit the number of live BEEFY voting rounds.
///
/// Allows messages from last [`MAX_LIVE_GOSSIP_ROUNDS`] to flow, everything else gets
/// rejected/expired.
///
///All messaging is handled in a single BEEFY global topic.
pub(crate) struct GossipValidator<B>
where
B: Block,
{
topic: B::Hash,
known_votes: RwLock<KnownVotes<B>>,
next_rebroadcast: Mutex<Instant>,
}
impl<B> GossipValidator<B>
where
B: Block,
{
pub fn new() -> GossipValidator<B> {
GossipValidator {
topic: topic::<B>(),
known_votes: RwLock::new(BTreeMap::new()),
next_rebroadcast: Mutex::new(Instant::now() + REBROADCAST_AFTER),
}
}
/// Note a voting round.
///
/// Noting `round` will keep `round` live.
///
/// We retain the [`MAX_LIVE_GOSSIP_ROUNDS`] most **recent** voting rounds as live.
/// As long as a voting round is live, it will be gossiped to peer nodes.
pub(crate) fn note_round(&self, round: NumberFor<B>) {
debug!(target: "beefy", "🥩 About to note round #{}", round);
let mut live = self.known_votes.write();
if !live.contains_key(&round) {
live.insert(round, Default::default());
}
if live.len() > MAX_LIVE_GOSSIP_ROUNDS {
let to_remove = live.iter().next().map(|x| x.0).copied();
if let Some(first) = to_remove {
live.remove(&first);
}
}
}
fn add_known(known_votes: &mut KnownVotes<B>, round: &NumberFor<B>, hash: MessageHash) {
known_votes.get_mut(round).map(|known| known.insert(hash));
}
// Note that we will always keep the most recent unseen round alive.
//
// This is a preliminary fix and the detailed description why we are
// doing this can be found as part of the issue below
//
// https://github.com/paritytech/grandpa-bridge-gadget/issues/237
//
fn is_live(known_votes: &KnownVotes<B>, round: &NumberFor<B>) -> bool {
let unseen_round = if let Some(max_known_round) = known_votes.keys().last() {
round > max_known_round
} else {
known_votes.is_empty()
};
known_votes.contains_key(round) || unseen_round
}
fn is_known(known_votes: &KnownVotes<B>, round: &NumberFor<B>, hash: &MessageHash) -> bool {
known_votes.get(round).map(|known| known.contains(hash)).unwrap_or(false)
}
}
impl<B> Validator<B> for GossipValidator<B>
where
B: Block,
{
fn validate(
&self,
_context: &mut dyn ValidatorContext<B>,
sender: &PeerId,
mut data: &[u8],
) -> ValidationResult<B::Hash> {
if let Ok(msg) =
VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(&mut data)
{
let msg_hash = twox_64(data);
let round = msg.commitment.block_number;
// Verify general usefulness of the message.
// We are going to discard old votes right away (without verification)
// Also we keep track of already received votes to avoid verifying duplicates.
{
let known_votes = self.known_votes.read();
if !GossipValidator::<B>::is_live(&known_votes, &round) {
return ValidationResult::Discard
}
if GossipValidator::<B>::is_known(&known_votes, &round, &msg_hash) {
return ValidationResult::ProcessAndKeep(self.topic)
}
}
if BeefyKeystore::verify(&msg.id, &msg.signature, &msg.commitment.encode()) {
GossipValidator::<B>::add_known(&mut *self.known_votes.write(), &round, msg_hash);
return ValidationResult::ProcessAndKeep(self.topic)
} else {
// TODO: report peer
debug!(target: "beefy", "🥩 Bad signature on message: {:?}, from: {:?}", msg, sender);
}
}
ValidationResult::Discard
}
fn message_expired<'a>(&'a self) -> Box<dyn FnMut(B::Hash, &[u8]) -> bool + 'a> {
let known_votes = self.known_votes.read();
Box::new(move |_topic, mut data| {
let msg = match VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(
&mut data,
) {
Ok(vote) => vote,
Err(_) => return true,
};
let round = msg.commitment.block_number;
let expired = !GossipValidator::<B>::is_live(&known_votes, &round);
trace!(target: "beefy", "🥩 Message for round #{} expired: {}", round, expired);
expired
})
}
fn message_allowed<'a>(
&'a self,
) -> Box<dyn FnMut(&PeerId, MessageIntent, &B::Hash, &[u8]) -> bool + 'a> {
let do_rebroadcast = {
let now = Instant::now();
let mut next_rebroadcast = self.next_rebroadcast.lock();
if now >= *next_rebroadcast {
*next_rebroadcast = now + REBROADCAST_AFTER;
true
} else {
false
}
};
let known_votes = self.known_votes.read();
Box::new(move |_who, intent, _topic, mut data| {
if let MessageIntent::PeriodicRebroadcast = intent {
return do_rebroadcast
}
let msg = match VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(
&mut data,
) {
Ok(vote) => vote,
Err(_) => return true,
};
let round = msg.commitment.block_number;
let allowed = GossipValidator::<B>::is_live(&known_votes, &round);
debug!(target: "beefy", "🥩 Message for round #{} allowed: {}", round, allowed);
allowed
})
}
}
+182
View File
@@ -0,0 +1,182 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use sc_keystore::LocalKeystore;
use sc_network_test::Block;
use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
use beefy_primitives::{crypto::Signature, Commitment, MmrRootHash, VoteMessage, KEY_TYPE};
use crate::keystore::{tests::Keyring, BeefyKeystore};
use super::*;
#[test]
fn note_round_works() {
let gv = GossipValidator::<Block>::new();
gv.note_round(1u64);
let live = gv.known_votes.read();
assert!(GossipValidator::<Block>::is_live(&live, &1u64));
drop(live);
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(!GossipValidator::<Block>::is_live(&live, &1u64));
assert!(GossipValidator::<Block>::is_live(&live, &3u64));
assert!(GossipValidator::<Block>::is_live(&live, &7u64));
assert!(GossipValidator::<Block>::is_live(&live, &10u64));
}
#[test]
fn keeps_most_recent_max_rounds() {
let gv = GossipValidator::<Block>::new();
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
gv.note_round(1u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(GossipValidator::<Block>::is_live(&live, &3u64));
assert!(!GossipValidator::<Block>::is_live(&live, &1u64));
drop(live);
gv.note_round(23u64);
gv.note_round(15u64);
gv.note_round(20u64);
gv.note_round(2u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(GossipValidator::<Block>::is_live(&live, &15u64));
assert!(GossipValidator::<Block>::is_live(&live, &20u64));
assert!(GossipValidator::<Block>::is_live(&live, &23u64));
}
#[test]
fn note_same_round_twice() {
let gv = GossipValidator::<Block>::new();
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
drop(live);
// note round #7 again -> should not change anything
gv.note_round(7u64);
let live = gv.known_votes.read();
assert_eq!(live.len(), MAX_LIVE_GOSSIP_ROUNDS);
assert!(GossipValidator::<Block>::is_live(&live, &3u64));
assert!(GossipValidator::<Block>::is_live(&live, &7u64));
assert!(GossipValidator::<Block>::is_live(&live, &10u64));
}
struct TestContext;
impl<B: sp_runtime::traits::Block> ValidatorContext<B> for TestContext {
fn broadcast_topic(&mut self, _topic: B::Hash, _force: bool) {
todo!()
}
fn broadcast_message(&mut self, _topic: B::Hash, _message: Vec<u8>, _force: bool) {
todo!()
}
fn send_message(&mut self, _who: &sc_network::PeerId, _message: Vec<u8>) {
todo!()
}
fn send_topic(&mut self, _who: &sc_network::PeerId, _topic: B::Hash, _force: bool) {
todo!()
}
}
fn sign_commitment<BN: Encode, P: Encode>(
who: &Keyring,
commitment: &Commitment<BN, P>,
) -> Signature {
let store: SyncCryptoStorePtr = std::sync::Arc::new(LocalKeystore::in_memory());
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&who.to_seed())).unwrap();
let beefy_keystore: BeefyKeystore = Some(store).into();
beefy_keystore.sign(&who.public(), &commitment.encode()).unwrap()
}
#[test]
fn should_avoid_verifying_signatures_twice() {
let gv = GossipValidator::<Block>::new();
let sender = sc_network::PeerId::random();
let mut context = TestContext;
let commitment =
Commitment { payload: MmrRootHash::default(), block_number: 3_u64, validator_set_id: 0 };
let signature = sign_commitment(&Keyring::Alice, &commitment);
let vote = VoteMessage { commitment, id: Keyring::Alice.public(), signature };
gv.note_round(3u64);
gv.note_round(7u64);
gv.note_round(10u64);
// first time the cache should be populated.
let res = gv.validate(&mut context, &sender, &vote.encode());
assert!(matches!(res, ValidationResult::ProcessAndKeep(_)));
assert_eq!(gv.known_votes.read().get(&vote.commitment.block_number).map(|x| x.len()), Some(1));
// second time we should hit the cache
let res = gv.validate(&mut context, &sender, &vote.encode());
assert!(matches!(res, ValidationResult::ProcessAndKeep(_)));
// next we should quickly reject if the round is not live.
gv.note_round(11_u64);
gv.note_round(12_u64);
assert!(!GossipValidator::<Block>::is_live(
&*gv.known_votes.read(),
&vote.commitment.block_number
));
let res = gv.validate(&mut context, &sender, &vote.encode());
assert!(matches!(res, ValidationResult::Discard));
}
+119
View File
@@ -0,0 +1,119 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::convert::{From, TryInto};
use sp_application_crypto::RuntimeAppPublic;
use sp_core::keccak_256;
use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
use log::warn;
use beefy_primitives::{
crypto::{Public, Signature},
KEY_TYPE,
};
use crate::error;
#[cfg(test)]
#[path = "keystore_tests.rs"]
pub mod tests;
/// A BEEFY specific keystore implemented as a `Newtype`. This is basically a
/// wrapper around [`sp_keystore::SyncCryptoStore`] and allows to customize
/// common cryptographic functionality.
pub(crate) struct BeefyKeystore(Option<SyncCryptoStorePtr>);
impl BeefyKeystore {
/// Check if the keystore contains a private key for one of the public keys
/// contained in `keys`. A public key with a matching private key is known
/// as a local authority id.
///
/// Return the public key for which we also do have a private key. If no
/// matching private key is found, `None` will be returned.
pub fn authority_id(&self, keys: &[Public]) -> Option<Public> {
let store = self.0.clone()?;
// we do check for multiple private keys as a key store sanity check.
let public: Vec<Public> = keys
.iter()
.filter(|k| SyncCryptoStore::has_keys(&*store, &[(k.to_raw_vec(), KEY_TYPE)]))
.cloned()
.collect();
if public.len() > 1 {
warn!(target: "beefy", "🥩 Multiple private keys found for: {:?} ({})", public, public.len());
}
public.get(0).cloned()
}
/// Sign `message` with the `public` key.
///
/// Note that `message` usually will be pre-hashed before being signed.
///
/// Return the message signature or an error in case of failure.
pub fn sign(&self, public: &Public, message: &[u8]) -> Result<Signature, error::Error> {
let store = self.0.clone().ok_or_else(|| error::Error::Keystore("no Keystore".into()))?;
let msg = keccak_256(message);
let public = public.as_ref();
let sig = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, public, &msg)
.map_err(|e| error::Error::Keystore(e.to_string()))?
.ok_or_else(|| error::Error::Signature("ecdsa_sign_prehashed() failed".to_string()))?;
// check that `sig` has the expected result type
let sig = sig.clone().try_into().map_err(|_| {
error::Error::Signature(format!("invalid signature {:?} for key {:?}", sig, public))
})?;
Ok(sig)
}
/// Returns a vector of [`beefy_primitives::crypto::Public`] keys which are currently supported
/// (i.e. found in the keystore).
pub fn public_keys(&self) -> Result<Vec<Public>, error::Error> {
let store = self.0.clone().ok_or_else(|| error::Error::Keystore("no Keystore".into()))?;
let pk: Vec<Public> = SyncCryptoStore::ecdsa_public_keys(&*store, KEY_TYPE)
.iter()
.map(|k| Public::from(k.clone()))
.collect();
Ok(pk)
}
/// Use the `public` key to verify that `sig` is a valid signature for `message`.
///
/// Return `true` if the signature is authentic, `false` otherwise.
pub fn verify(public: &Public, sig: &Signature, message: &[u8]) -> bool {
let msg = keccak_256(message);
let sig = sig.as_ref();
let public = public.as_ref();
sp_core::ecdsa::Pair::verify_prehashed(sig, &msg, public)
}
}
impl From<Option<SyncCryptoStorePtr>> for BeefyKeystore {
fn from(store: Option<SyncCryptoStorePtr>) -> BeefyKeystore {
BeefyKeystore(store)
}
}
@@ -0,0 +1,275 @@
// This file is part of Substrate.
// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::sync::Arc;
use sc_keystore::LocalKeystore;
use sp_core::{ecdsa, keccak_256, Pair};
use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
use beefy_primitives::{crypto, KEY_TYPE};
use super::BeefyKeystore;
use crate::error::Error;
/// Set of test accounts using [`beefy_primitives::crypto`] types.
#[allow(missing_docs)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, strum::Display, strum::EnumIter)]
pub(crate) enum Keyring {
Alice,
Bob,
Charlie,
Dave,
Eve,
Ferdie,
One,
Two,
}
impl Keyring {
/// Sign `msg`.
pub fn sign(self, msg: &[u8]) -> crypto::Signature {
let msg = keccak_256(msg);
ecdsa::Pair::from(self).sign_prehashed(&msg).into()
}
/// Return key pair.
pub fn pair(self) -> crypto::Pair {
ecdsa::Pair::from_string(self.to_seed().as_str(), None).unwrap().into()
}
/// Return public key.
pub fn public(self) -> crypto::Public {
self.pair().public()
}
/// Return seed string.
pub fn to_seed(self) -> String {
format!("//{}", self)
}
}
impl From<Keyring> for crypto::Pair {
fn from(k: Keyring) -> Self {
k.pair()
}
}
impl From<Keyring> for ecdsa::Pair {
fn from(k: Keyring) -> Self {
k.pair().into()
}
}
fn keystore() -> SyncCryptoStorePtr {
Arc::new(LocalKeystore::in_memory())
}
#[test]
fn verify_should_work() {
let msg = keccak_256(b"I am Alice!");
let sig = Keyring::Alice.sign(b"I am Alice!");
assert!(ecdsa::Pair::verify_prehashed(
&sig.clone().into(),
&msg,
&Keyring::Alice.public().into(),
));
// different public key -> fail
assert!(!ecdsa::Pair::verify_prehashed(
&sig.clone().into(),
&msg,
&Keyring::Bob.public().into(),
));
let msg = keccak_256(b"I am not Alice!");
// different msg -> fail
assert!(!ecdsa::Pair::verify_prehashed(&sig.into(), &msg, &Keyring::Alice.public().into(),));
}
#[test]
fn pair_works() {
let want = crypto::Pair::from_string("//Alice", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Alice.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Bob", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Bob.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Charlie", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Charlie.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Dave", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Dave.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Eve", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Eve.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Ferdie", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Ferdie.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//One", None).expect("Pair failed").to_raw_vec();
let got = Keyring::One.pair().to_raw_vec();
assert_eq!(want, got);
let want = crypto::Pair::from_string("//Two", None).expect("Pair failed").to_raw_vec();
let got = Keyring::Two.pair().to_raw_vec();
assert_eq!(want, got);
}
#[test]
fn authority_id_works() {
let store = keystore();
let alice: crypto::Public =
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Alice.to_seed()))
.ok()
.unwrap()
.into();
let bob = Keyring::Bob.public();
let charlie = Keyring::Charlie.public();
let store: BeefyKeystore = Some(store).into();
let mut keys = vec![bob, charlie];
let id = store.authority_id(keys.as_slice());
assert!(id.is_none());
keys.push(alice.clone());
let id = store.authority_id(keys.as_slice()).unwrap();
assert_eq!(id, alice);
}
#[test]
fn sign_works() {
let store = keystore();
let alice: crypto::Public =
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Alice.to_seed()))
.ok()
.unwrap()
.into();
let store: BeefyKeystore = Some(store).into();
let msg = b"are you involved or commited?";
let sig1 = store.sign(&alice, msg).unwrap();
let sig2 = Keyring::Alice.sign(msg);
assert_eq!(sig1, sig2);
}
#[test]
fn sign_error() {
let store = keystore();
let _ = SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Bob.to_seed()))
.ok()
.unwrap();
let store: BeefyKeystore = Some(store).into();
let alice = Keyring::Alice.public();
let msg = b"are you involved or commited?";
let sig = store.sign(&alice, msg).err().unwrap();
let err = Error::Signature("ecdsa_sign_prehashed() failed".to_string());
assert_eq!(sig, err);
}
#[test]
fn sign_no_keystore() {
let store: BeefyKeystore = None.into();
let alice = Keyring::Alice.public();
let msg = b"are you involved or commited";
let sig = store.sign(&alice, msg).err().unwrap();
let err = Error::Keystore("no Keystore".to_string());
assert_eq!(sig, err);
}
#[test]
fn verify_works() {
let store = keystore();
let alice: crypto::Public =
SyncCryptoStore::ecdsa_generate_new(&*store, KEY_TYPE, Some(&Keyring::Alice.to_seed()))
.ok()
.unwrap()
.into();
let store: BeefyKeystore = Some(store).into();
// `msg` and `sig` match
let msg = b"are you involved or commited?";
let sig = store.sign(&alice, msg).unwrap();
assert!(BeefyKeystore::verify(&alice, &sig, msg));
// `msg and `sig` don't match
let msg = b"you are just involved";
assert!(!BeefyKeystore::verify(&alice, &sig, msg));
}
// Note that we use keys with and without a seed for this test.
#[test]
fn public_keys_works() {
const TEST_TYPE: sp_application_crypto::KeyTypeId = sp_application_crypto::KeyTypeId(*b"test");
let store = keystore();
let add_key = |key_type, seed: Option<&str>| {
SyncCryptoStore::ecdsa_generate_new(&*store, key_type, seed).unwrap()
};
// test keys
let _ = add_key(TEST_TYPE, Some(Keyring::Alice.to_seed().as_str()));
let _ = add_key(TEST_TYPE, Some(Keyring::Bob.to_seed().as_str()));
let _ = add_key(TEST_TYPE, None);
let _ = add_key(TEST_TYPE, None);
// BEEFY keys
let _ = add_key(KEY_TYPE, Some(Keyring::Dave.to_seed().as_str()));
let _ = add_key(KEY_TYPE, Some(Keyring::Eve.to_seed().as_str()));
let key1: crypto::Public = add_key(KEY_TYPE, None).into();
let key2: crypto::Public = add_key(KEY_TYPE, None).into();
let store: BeefyKeystore = Some(store).into();
let keys = store.public_keys().ok().unwrap();
assert!(keys.len() == 4);
assert!(keys.contains(&Keyring::Dave.public()));
assert!(keys.contains(&Keyring::Eve.public()));
assert!(keys.contains(&key1));
assert!(keys.contains(&key2));
}
+159
View File
@@ -0,0 +1,159 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::sync::Arc;
use log::debug;
use prometheus::Registry;
use sc_client_api::{Backend, BlockchainEvents, Finalizer};
use sc_network_gossip::{GossipEngine, Network as GossipNetwork};
use sp_api::ProvideRuntimeApi;
use sp_blockchain::HeaderBackend;
use sp_keystore::SyncCryptoStorePtr;
use sp_runtime::traits::Block;
use beefy_primitives::BeefyApi;
mod error;
mod gossip;
mod keystore;
mod metrics;
mod round;
mod worker;
pub mod notification;
pub const BEEFY_PROTOCOL_NAME: &str = "/paritytech/beefy/1";
/// Returns the configuration value to put in
/// [`sc_network::config::NetworkConfiguration::extra_sets`].
pub fn beefy_peers_set_config() -> sc_network::config::NonDefaultSetConfig {
let mut cfg =
sc_network::config::NonDefaultSetConfig::new(BEEFY_PROTOCOL_NAME.into(), 1024 * 1024);
cfg.allow_non_reserved(25, 25);
cfg
}
/// A convenience BEEFY client trait that defines all the type bounds a BEEFY client
/// has to satisfy. Ideally that should actually be a trait alias. Unfortunately as
/// of today, Rust does not allow a type alias to be used as a trait bound. Tracking
/// issue is <https://github.com/rust-lang/rust/issues/41517>.
pub trait Client<B, BE>:
BlockchainEvents<B> + HeaderBackend<B> + Finalizer<B, BE> + ProvideRuntimeApi<B> + Send + Sync
where
B: Block,
BE: Backend<B>,
{
// empty
}
impl<B, BE, T> Client<B, BE> for T
where
B: Block,
BE: Backend<B>,
T: BlockchainEvents<B>
+ HeaderBackend<B>
+ Finalizer<B, BE>
+ ProvideRuntimeApi<B>
+ Send
+ Sync,
{
// empty
}
/// BEEFY gadget initialization parameters.
pub struct BeefyParams<B, BE, C, N>
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
N: GossipNetwork<B> + Clone + Send + 'static,
{
/// BEEFY client
pub client: Arc<C>,
/// Client Backend
pub backend: Arc<BE>,
/// Local key store
pub key_store: Option<SyncCryptoStorePtr>,
/// Gossip network
pub network: N,
/// BEEFY signed commitment sender
pub signed_commitment_sender: notification::BeefySignedCommitmentSender<B>,
/// Minimal delta between blocks, BEEFY should vote for
pub min_block_delta: u32,
/// Prometheus metric registry
pub prometheus_registry: Option<Registry>,
}
/// Start the BEEFY gadget.
///
/// This is a thin shim around running and awaiting a BEEFY worker.
pub async fn start_beefy_gadget<B, BE, C, N>(beefy_params: BeefyParams<B, BE, C, N>)
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
N: GossipNetwork<B> + Clone + Send + 'static,
{
let BeefyParams {
client,
backend,
key_store,
network,
signed_commitment_sender,
min_block_delta,
prometheus_registry,
} = beefy_params;
let gossip_validator = Arc::new(gossip::GossipValidator::new());
let gossip_engine =
GossipEngine::new(network, BEEFY_PROTOCOL_NAME, gossip_validator.clone(), None);
let metrics =
prometheus_registry.as_ref().map(metrics::Metrics::register).and_then(
|result| match result {
Ok(metrics) => {
debug!(target: "beefy", "🥩 Registered metrics");
Some(metrics)
},
Err(err) => {
debug!(target: "beefy", "🥩 Failed to register metrics: {:?}", err);
None
},
},
);
let worker_params = worker::WorkerParams {
client,
backend,
key_store: key_store.into(),
signed_commitment_sender,
gossip_engine,
gossip_validator,
min_block_delta,
metrics,
};
let worker = worker::BeefyWorker::<_, _, _>::new(worker_params);
worker.run().await
}
+93
View File
@@ -0,0 +1,93 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! BEEFY Prometheus metrics definition
use prometheus::{register, Counter, Gauge, PrometheusError, Registry, U64};
/// BEEFY metrics exposed through Prometheus
pub(crate) struct Metrics {
/// Current active validator set id
pub beefy_validator_set_id: Gauge<U64>,
/// Total number of votes sent by this node
pub beefy_votes_sent: Counter<U64>,
/// Most recent concluded voting round
pub beefy_round_concluded: Gauge<U64>,
/// Best block finalized by BEEFY
pub beefy_best_block: Gauge<U64>,
/// Next block BEEFY should vote on
pub beefy_should_vote_on: Gauge<U64>,
/// Number of sessions without a signed commitment
pub beefy_skipped_sessions: Counter<U64>,
}
impl Metrics {
pub(crate) fn register(registry: &Registry) -> Result<Self, PrometheusError> {
Ok(Self {
beefy_validator_set_id: register(
Gauge::new("beefy_validator_set_id", "Current BEEFY active validator set id.")?,
registry,
)?,
beefy_votes_sent: register(
Counter::new("beefy_votes_sent", "Number of votes sent by this node")?,
registry,
)?,
beefy_round_concluded: register(
Gauge::new("beefy_round_concluded", "Voting round, that has been concluded")?,
registry,
)?,
beefy_best_block: register(
Gauge::new("beefy_best_block", "Best block finalized by BEEFY")?,
registry,
)?,
beefy_should_vote_on: register(
Gauge::new("beefy_should_vote_on", "Next block, BEEFY should vote on")?,
registry,
)?,
beefy_skipped_sessions: register(
Counter::new(
"beefy_skipped_sessions",
"Number of sessions without a signed commitment",
)?,
registry,
)?,
})
}
}
// Note: we use the `format` macro to convert an expr into a `u64`. This will fail,
// if expr does not derive `Display`.
#[macro_export]
macro_rules! metric_set {
($self:ident, $m:ident, $v:expr) => {{
let val: u64 = format!("{}", $v).parse().unwrap();
if let Some(metrics) = $self.metrics.as_ref() {
metrics.$m.set(val);
}
}};
}
#[macro_export]
macro_rules! metric_inc {
($self:ident, $m:ident) => {{
if let Some(metrics) = $self.metrics.as_ref() {
metrics.$m.inc();
}
}};
}
+113
View File
@@ -0,0 +1,113 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::sync::Arc;
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver, TracingUnboundedSender};
use sp_runtime::traits::{Block, NumberFor};
use parking_lot::Mutex;
/// Stream of signed commitments returned when subscribing.
pub type SignedCommitment<Block> =
beefy_primitives::SignedCommitment<NumberFor<Block>, beefy_primitives::MmrRootHash>;
/// Stream of signed commitments returned when subscribing.
type SignedCommitmentStream<Block> = TracingUnboundedReceiver<SignedCommitment<Block>>;
/// Sending endpoint for notifying about signed commitments.
type SignedCommitmentSender<Block> = TracingUnboundedSender<SignedCommitment<Block>>;
/// Collection of channel sending endpoints shared with the receiver side so they can register
/// themselves.
type SharedSignedCommitmentSenders<Block> = Arc<Mutex<Vec<SignedCommitmentSender<Block>>>>;
/// The sending half of the signed commitment channel(s).
///
/// Used to send notifications about signed commitments generated at the end of a BEEFY round.
#[derive(Clone)]
pub struct BeefySignedCommitmentSender<B>
where
B: Block,
{
subscribers: SharedSignedCommitmentSenders<B>,
}
impl<B> BeefySignedCommitmentSender<B>
where
B: Block,
{
/// The `subscribers` should be shared with a corresponding `SignedCommitmentSender`.
fn new(subscribers: SharedSignedCommitmentSenders<B>) -> Self {
Self { subscribers }
}
/// Send out a notification to all subscribers that a new signed commitment is available for a
/// block.
pub fn notify(&self, signed_commitment: SignedCommitment<B>) {
let mut subscribers = self.subscribers.lock();
// do an initial prune on closed subscriptions
subscribers.retain(|n| !n.is_closed());
if !subscribers.is_empty() {
subscribers.retain(|n| n.unbounded_send(signed_commitment.clone()).is_ok());
}
}
}
/// The receiving half of the signed commitments channel.
///
/// Used to receive notifications about signed commitments generated at the end of a BEEFY round.
/// The `BeefySignedCommitmentStream` entity stores the `SharedSignedCommitmentSenders` so it can be
/// used to add more subscriptions.
#[derive(Clone)]
pub struct BeefySignedCommitmentStream<B>
where
B: Block,
{
subscribers: SharedSignedCommitmentSenders<B>,
}
impl<B> BeefySignedCommitmentStream<B>
where
B: Block,
{
/// Creates a new pair of receiver and sender of signed commitment notifications.
pub fn channel() -> (BeefySignedCommitmentSender<B>, Self) {
let subscribers = Arc::new(Mutex::new(vec![]));
let receiver = BeefySignedCommitmentStream::new(subscribers.clone());
let sender = BeefySignedCommitmentSender::new(subscribers);
(sender, receiver)
}
/// Create a new receiver of signed commitment notifications.
///
/// The `subscribers` should be shared with a corresponding `BeefySignedCommitmentSender`.
fn new(subscribers: SharedSignedCommitmentSenders<B>) -> Self {
Self { subscribers }
}
/// Subscribe to a channel through which signed commitments are sent at the end of each BEEFY
/// voting round.
pub fn subscribe(&self) -> SignedCommitmentStream<B> {
let (sender, receiver) = tracing_unbounded("mpsc_signed_commitments_notification_stream");
self.subscribers.lock().push(sender);
receiver
}
}
+121
View File
@@ -0,0 +1,121 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{collections::BTreeMap, hash::Hash};
use log::{debug, trace};
use beefy_primitives::{
crypto::{Public, Signature},
ValidatorSet, ValidatorSetId,
};
use sp_arithmetic::traits::AtLeast32BitUnsigned;
use sp_runtime::traits::MaybeDisplay;
#[derive(Default)]
struct RoundTracker {
votes: Vec<(Public, Signature)>,
}
impl RoundTracker {
fn add_vote(&mut self, vote: (Public, Signature)) -> bool {
// this needs to handle equivocations in the future
if self.votes.contains(&vote) {
return false
}
self.votes.push(vote);
true
}
fn is_done(&self, threshold: usize) -> bool {
self.votes.len() >= threshold
}
}
fn threshold(authorities: usize) -> usize {
let faulty = authorities.saturating_sub(1) / 3;
authorities - faulty
}
pub(crate) struct Rounds<Hash, Number> {
rounds: BTreeMap<(Hash, Number), RoundTracker>,
validator_set: ValidatorSet<Public>,
}
impl<H, N> Rounds<H, N>
where
H: Ord + Hash,
N: Ord + AtLeast32BitUnsigned + MaybeDisplay,
{
pub(crate) fn new(validator_set: ValidatorSet<Public>) -> Self {
Rounds { rounds: BTreeMap::new(), validator_set }
}
}
impl<H, N> Rounds<H, N>
where
H: Ord + Hash,
N: Ord + AtLeast32BitUnsigned + MaybeDisplay,
{
pub(crate) fn validator_set_id(&self) -> ValidatorSetId {
self.validator_set.id
}
pub(crate) fn validators(&self) -> Vec<Public> {
self.validator_set.validators.clone()
}
pub(crate) fn add_vote(&mut self, round: (H, N), vote: (Public, Signature)) -> bool {
self.rounds.entry(round).or_default().add_vote(vote)
}
pub(crate) fn is_done(&self, round: &(H, N)) -> bool {
let done = self
.rounds
.get(round)
.map(|tracker| tracker.is_done(threshold(self.validator_set.validators.len())))
.unwrap_or(false);
debug!(target: "beefy", "🥩 Round #{} done: {}", round.1, done);
done
}
pub(crate) fn drop(&mut self, round: &(H, N)) -> Option<Vec<Option<Signature>>> {
trace!(target: "beefy", "🥩 About to drop round #{}", round.1);
let signatures = self.rounds.remove(round)?.votes;
Some(
self.validator_set
.validators
.iter()
.map(|authority_id| {
signatures.iter().find_map(|(id, sig)| {
if id == authority_id {
Some(sig.clone())
} else {
None
}
})
})
.collect(),
)
}
}
+534
View File
@@ -0,0 +1,534 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{collections::BTreeSet, fmt::Debug, marker::PhantomData, sync::Arc};
use codec::{Codec, Decode, Encode};
use futures::{future, FutureExt, StreamExt};
use log::{debug, error, info, trace, warn};
use parking_lot::Mutex;
use sc_client_api::{Backend, FinalityNotification, FinalityNotifications};
use sc_network_gossip::GossipEngine;
use sp_api::BlockId;
use sp_arithmetic::traits::AtLeast32Bit;
use sp_runtime::{
generic::OpaqueDigestItemId,
traits::{Block, Header, NumberFor},
SaturatedConversion,
};
use beefy_primitives::{
crypto::{AuthorityId, Public, Signature},
BeefyApi, Commitment, ConsensusLog, MmrRootHash, SignedCommitment, ValidatorSet,
VersionedCommitment, VoteMessage, BEEFY_ENGINE_ID, GENESIS_AUTHORITY_SET_ID,
};
use crate::{
error,
gossip::{topic, GossipValidator},
keystore::BeefyKeystore,
metric_inc, metric_set,
metrics::Metrics,
notification, round, Client,
};
pub(crate) struct WorkerParams<B, BE, C>
where
B: Block,
{
pub client: Arc<C>,
pub backend: Arc<BE>,
pub key_store: BeefyKeystore,
pub signed_commitment_sender: notification::BeefySignedCommitmentSender<B>,
pub gossip_engine: GossipEngine<B>,
pub gossip_validator: Arc<GossipValidator<B>>,
pub min_block_delta: u32,
pub metrics: Option<Metrics>,
}
/// A BEEFY worker plays the BEEFY protocol
pub(crate) struct BeefyWorker<B, C, BE>
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
{
client: Arc<C>,
backend: Arc<BE>,
key_store: BeefyKeystore,
signed_commitment_sender: notification::BeefySignedCommitmentSender<B>,
gossip_engine: Arc<Mutex<GossipEngine<B>>>,
gossip_validator: Arc<GossipValidator<B>>,
/// Min delta in block numbers between two blocks, BEEFY should vote on
min_block_delta: u32,
metrics: Option<Metrics>,
rounds: round::Rounds<MmrRootHash, NumberFor<B>>,
finality_notifications: FinalityNotifications<B>,
/// Best block we received a GRANDPA notification for
best_grandpa_block: NumberFor<B>,
/// Best block a BEEFY voting round has been concluded for
best_beefy_block: Option<NumberFor<B>>,
/// Validator set id for the last signed commitment
last_signed_id: u64,
// keep rustc happy
_backend: PhantomData<BE>,
}
impl<B, C, BE> BeefyWorker<B, C, BE>
where
B: Block + Codec,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
{
/// Return a new BEEFY worker instance.
///
/// Note that a BEEFY worker is only fully functional if a corresponding
/// BEEFY pallet has been deployed on-chain.
///
/// The BEEFY pallet is needed in order to keep track of the BEEFY authority set.
pub(crate) fn new(worker_params: WorkerParams<B, BE, C>) -> Self {
let WorkerParams {
client,
backend,
key_store,
signed_commitment_sender,
gossip_engine,
gossip_validator,
min_block_delta,
metrics,
} = worker_params;
BeefyWorker {
client: client.clone(),
backend,
key_store,
signed_commitment_sender,
gossip_engine: Arc::new(Mutex::new(gossip_engine)),
gossip_validator,
min_block_delta,
metrics,
rounds: round::Rounds::new(ValidatorSet::empty()),
finality_notifications: client.finality_notification_stream(),
best_grandpa_block: client.info().finalized_number,
best_beefy_block: None,
last_signed_id: 0,
_backend: PhantomData,
}
}
}
impl<B, C, BE> BeefyWorker<B, C, BE>
where
B: Block,
BE: Backend<B>,
C: Client<B, BE>,
C::Api: BeefyApi<B>,
{
/// Return `true`, if we should vote on block `number`
fn should_vote_on(&self, number: NumberFor<B>) -> bool {
let best_beefy_block = if let Some(block) = self.best_beefy_block {
block
} else {
debug!(target: "beefy", "🥩 Missing best BEEFY block - won't vote for: {:?}", number);
return false
};
let target = vote_target(self.best_grandpa_block, best_beefy_block, self.min_block_delta);
trace!(target: "beefy", "🥩 should_vote_on: #{:?}, next_block_to_vote_on: #{:?}", number, target);
metric_set!(self, beefy_should_vote_on, target);
number == target
}
/// Return the current active validator set at header `header`.
///
/// Note that the validator set could be `None`. This is the case if we don't find
/// a BEEFY authority set change and we can't fetch the authority set from the
/// BEEFY on-chain state.
///
/// Such a failure is usually an indication that the BEEFY pallet has not been deployed (yet).
fn validator_set(&self, header: &B::Header) -> Option<ValidatorSet<Public>> {
let new = if let Some(new) = find_authorities_change::<B>(header) {
Some(new)
} else {
let at = BlockId::hash(header.hash());
self.client.runtime_api().validator_set(&at).ok()
};
trace!(target: "beefy", "🥩 active validator set: {:?}", new);
new
}
/// Verify `active` validator set for `block` against the key store
///
/// The critical case is, if we do have a public key in the key store which is not
/// part of the active validator set.
///
/// Note that for a non-authority node there will be no keystore, and we will
/// return an error and don't check. The error can usually be ignored.
fn verify_validator_set(
&self,
block: &NumberFor<B>,
mut active: ValidatorSet<Public>,
) -> Result<(), error::Error> {
let active: BTreeSet<Public> = active.validators.drain(..).collect();
let store: BTreeSet<Public> = self.key_store.public_keys()?.drain(..).collect();
let missing: Vec<_> = store.difference(&active).cloned().collect();
if !missing.is_empty() {
debug!(target: "beefy", "🥩 for block {:?} public key missing in validator set: {:?}", block, missing);
}
Ok(())
}
fn handle_finality_notification(&mut self, notification: FinalityNotification<B>) {
trace!(target: "beefy", "🥩 Finality notification: {:?}", notification);
// update best GRANDPA finalized block we have seen
self.best_grandpa_block = *notification.header.number();
if let Some(active) = self.validator_set(&notification.header) {
// Authority set change or genesis set id triggers new voting rounds
//
// TODO: (adoerr) Enacting a new authority set will also implicitly 'conclude'
// the currently active BEEFY voting round by starting a new one. This is
// temporary and needs to be replaced by proper round life cycle handling.
if active.id != self.rounds.validator_set_id() ||
(active.id == GENESIS_AUTHORITY_SET_ID && self.best_beefy_block.is_none())
{
debug!(target: "beefy", "🥩 New active validator set id: {:?}", active);
metric_set!(self, beefy_validator_set_id, active.id);
// BEEFY should produce a signed commitment for each session
if active.id != self.last_signed_id + 1 && active.id != GENESIS_AUTHORITY_SET_ID {
metric_inc!(self, beefy_skipped_sessions);
}
// verify the new validator set
let _ = self.verify_validator_set(notification.header.number(), active.clone());
self.rounds = round::Rounds::new(active.clone());
debug!(target: "beefy", "🥩 New Rounds for id: {:?}", active.id);
self.best_beefy_block = Some(*notification.header.number());
// this metric is kind of 'fake'. Best BEEFY block should only be updated once we
// have a signed commitment for the block. Remove once the above TODO is done.
metric_set!(self, beefy_best_block, *notification.header.number());
}
}
if self.should_vote_on(*notification.header.number()) {
let authority_id = if let Some(id) =
self.key_store.authority_id(self.rounds.validators().as_slice())
{
debug!(target: "beefy", "🥩 Local authority id: {:?}", id);
id
} else {
debug!(target: "beefy", "🥩 Missing validator id - can't vote for: {:?}", notification.header.hash());
return
};
let mmr_root =
if let Some(hash) = find_mmr_root_digest::<B, Public>(&notification.header) {
hash
} else {
warn!(target: "beefy", "🥩 No MMR root digest found for: {:?}", notification.header.hash());
return
};
let commitment = Commitment {
payload: mmr_root,
block_number: notification.header.number(),
validator_set_id: self.rounds.validator_set_id(),
};
let encoded_commitment = commitment.encode();
let signature = match self.key_store.sign(&authority_id, &*encoded_commitment) {
Ok(sig) => sig,
Err(err) => {
warn!(target: "beefy", "🥩 Error signing commitment: {:?}", err);
return
},
};
trace!(
target: "beefy",
"🥩 Produced signature using {:?}, is_valid: {:?}",
authority_id,
BeefyKeystore::verify(&authority_id, &signature, &*encoded_commitment)
);
let message = VoteMessage { commitment, id: authority_id, signature };
let encoded_message = message.encode();
metric_inc!(self, beefy_votes_sent);
debug!(target: "beefy", "🥩 Sent vote message: {:?}", message);
self.handle_vote(
(message.commitment.payload, *message.commitment.block_number),
(message.id, message.signature),
);
self.gossip_engine.lock().gossip_message(topic::<B>(), encoded_message, false);
}
}
fn handle_vote(&mut self, round: (MmrRootHash, NumberFor<B>), vote: (Public, Signature)) {
self.gossip_validator.note_round(round.1);
let vote_added = self.rounds.add_vote(round, vote);
if vote_added && self.rounds.is_done(&round) {
if let Some(signatures) = self.rounds.drop(&round) {
// id is stored for skipped session metric calculation
self.last_signed_id = self.rounds.validator_set_id();
let commitment = Commitment {
payload: round.0,
block_number: round.1,
validator_set_id: self.last_signed_id,
};
let signed_commitment = SignedCommitment { commitment, signatures };
metric_set!(self, beefy_round_concluded, round.1);
info!(target: "beefy", "🥩 Round #{} concluded, committed: {:?}.", round.1, signed_commitment);
if self
.backend
.append_justification(
BlockId::Number(round.1),
(
BEEFY_ENGINE_ID,
VersionedCommitment::V1(signed_commitment.clone()).encode(),
),
)
.is_err()
{
// just a trace, because until the round lifecycle is improved, we will
// conclude certain rounds multiple times.
trace!(target: "beefy", "🥩 Failed to append justification: {:?}", signed_commitment);
}
self.signed_commitment_sender.notify(signed_commitment);
self.best_beefy_block = Some(round.1);
metric_set!(self, beefy_best_block, round.1);
}
}
}
pub(crate) async fn run(mut self) {
let mut votes = Box::pin(self.gossip_engine.lock().messages_for(topic::<B>()).filter_map(
|notification| async move {
debug!(target: "beefy", "🥩 Got vote message: {:?}", notification);
VoteMessage::<MmrRootHash, NumberFor<B>, Public, Signature>::decode(
&mut &notification.message[..],
)
.ok()
},
));
loop {
let engine = self.gossip_engine.clone();
let gossip_engine = future::poll_fn(|cx| engine.lock().poll_unpin(cx));
futures::select! {
notification = self.finality_notifications.next().fuse() => {
if let Some(notification) = notification {
self.handle_finality_notification(notification);
} else {
return;
}
},
vote = votes.next().fuse() => {
if let Some(vote) = vote {
self.handle_vote(
(vote.commitment.payload, vote.commitment.block_number),
(vote.id, vote.signature),
);
} else {
return;
}
},
_ = gossip_engine.fuse() => {
error!(target: "beefy", "🥩 Gossip engine has terminated.");
return;
}
}
}
}
}
/// Extract the MMR root hash from a digest in the given header, if it exists.
fn find_mmr_root_digest<B, Id>(header: &B::Header) -> Option<MmrRootHash>
where
B: Block,
Id: Codec,
{
header.digest().logs().iter().find_map(|log| {
match log.try_to::<ConsensusLog<Id>>(OpaqueDigestItemId::Consensus(&BEEFY_ENGINE_ID)) {
Some(ConsensusLog::MmrRoot(root)) => Some(root),
_ => None,
}
})
}
/// Scan the `header` digest log for a BEEFY validator set change. Return either the new
/// validator set or `None` in case no validator set change has been signaled.
fn find_authorities_change<B>(header: &B::Header) -> Option<ValidatorSet<AuthorityId>>
where
B: Block,
{
let id = OpaqueDigestItemId::Consensus(&BEEFY_ENGINE_ID);
let filter = |log: ConsensusLog<AuthorityId>| match log {
ConsensusLog::AuthoritiesChange(validator_set) => Some(validator_set),
_ => None,
};
header.digest().convert_first(|l| l.try_to(id).and_then(filter))
}
/// Calculate next block number to vote on
fn vote_target<N>(best_grandpa: N, best_beefy: N, min_delta: u32) -> N
where
N: AtLeast32Bit + Copy + Debug,
{
let diff = best_grandpa.saturating_sub(best_beefy);
let diff = diff.saturated_into::<u32>();
let target = best_beefy + min_delta.max(diff.next_power_of_two()).into();
trace!(
target: "beefy",
"🥩 vote target - diff: {:?}, next_power_of_two: {:?}, target block: #{:?}",
diff,
diff.next_power_of_two(),
target,
);
target
}
#[cfg(test)]
mod tests {
use super::vote_target;
#[test]
fn vote_on_min_block_delta() {
let t = vote_target(1u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(2u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(3u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(4u32, 0, 4);
assert_eq!(4, t);
let t = vote_target(4u32, 4, 4);
assert_eq!(8, t);
let t = vote_target(10u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(11u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(12u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(13u32, 10, 4);
assert_eq!(14, t);
let t = vote_target(10u32, 10, 8);
assert_eq!(18, t);
let t = vote_target(11u32, 10, 8);
assert_eq!(18, t);
let t = vote_target(12u32, 10, 8);
assert_eq!(18, t);
let t = vote_target(13u32, 10, 8);
assert_eq!(18, t);
}
#[test]
fn vote_on_power_of_two() {
let t = vote_target(1008u32, 1000, 4);
assert_eq!(1008, t);
let t = vote_target(1016u32, 1000, 4);
assert_eq!(1016, t);
let t = vote_target(1032u32, 1000, 4);
assert_eq!(1032, t);
let t = vote_target(1064u32, 1000, 4);
assert_eq!(1064, t);
let t = vote_target(1128u32, 1000, 4);
assert_eq!(1128, t);
let t = vote_target(1256u32, 1000, 4);
assert_eq!(1256, t);
let t = vote_target(1512u32, 1000, 4);
assert_eq!(1512, t);
let t = vote_target(1024u32, 0, 4);
assert_eq!(1024, t);
}
#[test]
fn vote_on_target_block() {
let t = vote_target(1008u32, 1002, 4);
assert_eq!(1010, t);
let t = vote_target(1010u32, 1002, 4);
assert_eq!(1010, t);
let t = vote_target(1016u32, 1006, 4);
assert_eq!(1022, t);
let t = vote_target(1022u32, 1006, 4);
assert_eq!(1022, t);
let t = vote_target(1032u32, 1012, 4);
assert_eq!(1044, t);
let t = vote_target(1044u32, 1012, 4);
assert_eq!(1044, t);
let t = vote_target(1064u32, 1014, 4);
assert_eq!(1078, t);
let t = vote_target(1078u32, 1014, 4);
assert_eq!(1078, t);
let t = vote_target(1128u32, 1008, 4);
assert_eq!(1136, t);
let t = vote_target(1136u32, 1008, 4);
assert_eq!(1136, t);
}
}
+56
View File
@@ -0,0 +1,56 @@
[package]
name = "pallet-beefy-mmr"
version = "4.0.0-dev"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
license = "Apache-2.0"
description = "BEEFY + MMR runtime utilities"
[dependencies]
hex = { version = "0.4", optional = true }
codec = { version = "2.2.0", package = "parity-scale-codec", default-features = false, features = ["derive"] }
libsecp256k1 = { version = "0.7.0", default-features = false }
log = { version = "0.4.13", default-features = false }
scale-info = { version = "1.0", default-features = false, features = ["derive"] }
serde = { version = "1.0.130", optional = true }
frame-support = { version = "4.0.0-dev", path = "../support", default-features = false }
frame-system = { version = "4.0.0-dev", path = "../system", default-features = false }
pallet-mmr = { version = "4.0.0-dev", path = "../merkle-mountain-range", default-features = false }
pallet-mmr-primitives = { version = "4.0.0-dev", path = "../merkle-mountain-range/primitives", default-features = false }
pallet-session = { version = "4.0.0-dev", path = "../session", default-features = false }
sp-core = { version = "4.0.0-dev", path = "../../primitives/core", default-features = false }
sp-io = { version = "4.0.0-dev", path = "../../primitives/io", default-features = false }
sp-runtime = { version = "4.0.0-dev", path = "../../primitives/runtime", default-features = false }
sp-std = { version = "4.0.0-dev", path = "../../primitives/std", default-features = false }
beefy-merkle-tree = { version = "4.0.0-dev", path = "./primitives", default-features = false }
beefy-primitives = { version = "4.0.0-dev", path = "../../primitives/beefy", default-features = false }
pallet-beefy = { version = "4.0.0-dev", path = "../beefy", default-features = false }
[dev-dependencies]
sp-staking = { version = "4.0.0-dev", path = "../../primitives/staking" }
hex-literal = "0.3"
[features]
default = ["std"]
std = [
"beefy-merkle-tree/std",
"beefy-primitives/std",
"codec/std",
"frame-support/std",
"frame-system/std",
"hex",
"libsecp256k1/std",
"log/std",
"pallet-beefy/std",
"pallet-mmr-primitives/std",
"pallet-mmr/std",
"pallet-session/std",
"serde",
"sp-core/std",
"sp-io/std",
"sp-runtime/std",
"sp-std/std",
]
@@ -0,0 +1,23 @@
[package]
name = "beefy-merkle-tree"
version = "4.0.0-dev"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
license = "Apache-2.0"
description = "A no-std/Substrate compatible library to construct binary merkle tree."
[dependencies]
hex = { version = "0.4", optional = true, default-features = false }
log = { version = "0.4", optional = true, default-features = false }
tiny-keccak = { version = "2.0.2", features = ["keccak"], optional = true }
[dev-dependencies]
env_logger = "0.9"
hex = "0.4"
hex-literal = "0.3"
[features]
debug = ["hex", "log"]
default = ["std", "debug", "keccak"]
keccak = ["tiny-keccak"]
std = []
@@ -0,0 +1,806 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#![cfg_attr(not(feature = "std"), no_std)]
#![warn(missing_docs)]
//! This crate implements a simple binary Merkle Tree utilities required for inter-op with Ethereum
//! bridge & Solidity contract.
//!
//! The implementation is optimised for usage within Substrate Runtime and supports no-std
//! compilation targets.
//!
//! Merkle Tree is constructed from arbitrary-length leaves, that are initially hashed using the
//! same [Hasher] as the inner nodes.
//! Inner nodes are created by concatenating child hashes and hashing again. The implementation
//! does not perform any sorting of the input data (leaves) nor when inner nodes are created.
//!
//! If the number of leaves is not even, last leave (hash of) is promoted to the upper layer.
#[cfg(not(feature = "std"))]
extern crate alloc;
#[cfg(not(feature = "std"))]
use alloc::vec::Vec;
/// Supported hashing output size.
///
/// The size is restricted to 32 bytes to allow for a more optimised implementation.
pub type Hash = [u8; 32];
/// Generic hasher trait.
///
/// Implement the function to support custom way of hashing data.
/// The implementation must return a [Hash] type, so only 32-byte output hashes are supported.
pub trait Hasher {
/// Hash given arbitrary-length piece of data.
fn hash(data: &[u8]) -> Hash;
}
#[cfg(feature = "keccak")]
mod keccak256 {
use tiny_keccak::{Hasher as _, Keccak};
/// Keccak256 hasher implementation.
pub struct Keccak256;
impl Keccak256 {
/// Hash given data.
pub fn hash(data: &[u8]) -> super::Hash {
<Keccak256 as super::Hasher>::hash(data)
}
}
impl super::Hasher for Keccak256 {
fn hash(data: &[u8]) -> super::Hash {
let mut keccak = Keccak::v256();
keccak.update(data);
let mut output = [0_u8; 32];
keccak.finalize(&mut output);
output
}
}
}
#[cfg(feature = "keccak")]
pub use keccak256::Keccak256;
/// Construct a root hash of a Binary Merkle Tree created from given leaves.
///
/// See crate-level docs for details about Merkle Tree construction.
///
/// In case an empty list of leaves is passed the function returns a 0-filled hash.
pub fn merkle_root<H, I, T>(leaves: I) -> Hash
where
H: Hasher,
I: IntoIterator<Item = T>,
T: AsRef<[u8]>,
{
let iter = leaves.into_iter().map(|l| H::hash(l.as_ref()));
merkelize::<H, _, _>(iter, &mut ())
}
fn merkelize<H, V, I>(leaves: I, visitor: &mut V) -> Hash
where
H: Hasher,
V: Visitor,
I: Iterator<Item = Hash>,
{
let upper = Vec::with_capacity(leaves.size_hint().0);
let mut next = match merkelize_row::<H, _, _>(leaves, upper, visitor) {
Ok(root) => return root,
Err(next) if next.is_empty() => return Hash::default(),
Err(next) => next,
};
let mut upper = Vec::with_capacity((next.len() + 1) / 2);
loop {
visitor.move_up();
match merkelize_row::<H, _, _>(next.drain(..), upper, visitor) {
Ok(root) => return root,
Err(t) => {
// swap collections to avoid allocations
upper = next;
next = t;
},
};
}
}
/// A generated merkle proof.
///
/// The structure contains all necessary data to later on verify the proof and the leaf itself.
#[derive(Debug, PartialEq, Eq)]
pub struct MerkleProof<T> {
/// Root hash of generated merkle tree.
pub root: Hash,
/// Proof items (does not contain the leaf hash, nor the root obviously).
///
/// This vec contains all inner node hashes necessary to reconstruct the root hash given the
/// leaf hash.
pub proof: Vec<Hash>,
/// Number of leaves in the original tree.
///
/// This is needed to detect a case where we have an odd number of leaves that "get promoted"
/// to upper layers.
pub number_of_leaves: usize,
/// Index of the leaf the proof is for (0-based).
pub leaf_index: usize,
/// Leaf content.
pub leaf: T,
}
/// A trait of object inspecting merkle root creation.
///
/// It can be passed to [`merkelize_row`] or [`merkelize`] functions and will be notified
/// about tree traversal.
trait Visitor {
/// We are moving one level up in the tree.
fn move_up(&mut self);
/// We are creating an inner node from given `left` and `right` nodes.
///
/// Note that in case of last odd node in the row `right` might be empty.
/// The method will also visit the `root` hash (level 0).
///
/// The `index` is an index of `left` item.
fn visit(&mut self, index: usize, left: &Option<Hash>, right: &Option<Hash>);
}
/// No-op implementation of the visitor.
impl Visitor for () {
fn move_up(&mut self) {}
fn visit(&mut self, _index: usize, _left: &Option<Hash>, _right: &Option<Hash>) {}
}
/// Construct a Merkle Proof for leaves given by indices.
///
/// The function constructs a (partial) Merkle Tree first and stores all elements required
/// to prove requested item (leaf) given the root hash.
///
/// Both the Proof and the Root Hash is returned.
///
/// # Panic
///
/// The function will panic if given [`leaf_index`] is greater than the number of leaves.
pub fn merkle_proof<H, I, T>(leaves: I, leaf_index: usize) -> MerkleProof<T>
where
H: Hasher,
I: IntoIterator<Item = T>,
I::IntoIter: ExactSizeIterator,
T: AsRef<[u8]>,
{
let mut leaf = None;
let iter = leaves.into_iter().enumerate().map(|(idx, l)| {
let hash = H::hash(l.as_ref());
if idx == leaf_index {
leaf = Some(l);
}
hash
});
/// The struct collects a proof for single leaf.
struct ProofCollection {
proof: Vec<Hash>,
position: usize,
}
impl ProofCollection {
fn new(position: usize) -> Self {
ProofCollection { proof: Default::default(), position }
}
}
impl Visitor for ProofCollection {
fn move_up(&mut self) {
self.position /= 2;
}
fn visit(&mut self, index: usize, left: &Option<Hash>, right: &Option<Hash>) {
// we are at left branch - right goes to the proof.
if self.position == index {
if let Some(right) = right {
self.proof.push(*right);
}
}
// we are at right branch - left goes to the proof.
if self.position == index + 1 {
if let Some(left) = left {
self.proof.push(*left);
}
}
}
}
let number_of_leaves = iter.len();
let mut collect_proof = ProofCollection::new(leaf_index);
let root = merkelize::<H, _, _>(iter, &mut collect_proof);
let leaf = leaf.expect("Requested `leaf_index` is greater than number of leaves.");
#[cfg(feature = "debug")]
log::debug!(
"[merkle_proof] Proof: {:?}",
collect_proof.proof.iter().map(hex::encode).collect::<Vec<_>>()
);
MerkleProof { root, proof: collect_proof.proof, number_of_leaves, leaf_index, leaf }
}
/// Leaf node for proof verification.
///
/// Can be either a value that needs to be hashed first,
/// or the hash itself.
#[derive(Debug, PartialEq, Eq)]
pub enum Leaf<'a> {
/// Leaf content.
Value(&'a [u8]),
/// Hash of the leaf content.
Hash(Hash),
}
impl<'a, T: AsRef<[u8]>> From<&'a T> for Leaf<'a> {
fn from(v: &'a T) -> Self {
Leaf::Value(v.as_ref())
}
}
impl<'a> From<Hash> for Leaf<'a> {
fn from(v: Hash) -> Self {
Leaf::Hash(v)
}
}
/// Verify Merkle Proof correctness versus given root hash.
///
/// The proof is NOT expected to contain leaf hash as the first
/// element, but only all adjacent nodes required to eventually by process of
/// concatenating and hashing end up with given root hash.
///
/// The proof must not contain the root hash.
pub fn verify_proof<'a, H, P, L>(
root: &'a Hash,
proof: P,
number_of_leaves: usize,
leaf_index: usize,
leaf: L,
) -> bool
where
H: Hasher,
P: IntoIterator<Item = Hash>,
L: Into<Leaf<'a>>,
{
if leaf_index >= number_of_leaves {
return false
}
let leaf_hash = match leaf.into() {
Leaf::Value(content) => H::hash(content),
Leaf::Hash(hash) => hash,
};
let mut combined = [0_u8; 64];
let mut position = leaf_index;
let mut width = number_of_leaves;
let computed = proof.into_iter().fold(leaf_hash, |a, b| {
if position % 2 == 1 || position + 1 == width {
combined[0..32].copy_from_slice(&b);
combined[32..64].copy_from_slice(&a);
} else {
combined[0..32].copy_from_slice(&a);
combined[32..64].copy_from_slice(&b);
}
let hash = H::hash(&combined);
#[cfg(feature = "debug")]
log::debug!(
"[verify_proof]: (a, b) {:?}, {:?} => {:?} ({:?}) hash",
hex::encode(a),
hex::encode(b),
hex::encode(hash),
hex::encode(combined)
);
position /= 2;
width = ((width - 1) / 2) + 1;
hash
});
root == &computed
}
/// Processes a single row (layer) of a tree by taking pairs of elements,
/// concatenating them, hashing and placing into resulting vector.
///
/// In case only one element is provided it is returned via `Ok` result, in any other case (also an
/// empty iterator) an `Err` with the inner nodes of upper layer is returned.
fn merkelize_row<H, V, I>(
mut iter: I,
mut next: Vec<Hash>,
visitor: &mut V,
) -> Result<Hash, Vec<Hash>>
where
H: Hasher,
V: Visitor,
I: Iterator<Item = Hash>,
{
#[cfg(feature = "debug")]
log::debug!("[merkelize_row]");
next.clear();
let mut index = 0;
let mut combined = [0_u8; 64];
loop {
let a = iter.next();
let b = iter.next();
visitor.visit(index, &a, &b);
#[cfg(feature = "debug")]
log::debug!(" {:?}\n {:?}", a.as_ref().map(hex::encode), b.as_ref().map(hex::encode));
index += 2;
match (a, b) {
(Some(a), Some(b)) => {
combined[0..32].copy_from_slice(&a);
combined[32..64].copy_from_slice(&b);
next.push(H::hash(&combined));
},
// Odd number of items. Promote the item to the upper layer.
(Some(a), None) if !next.is_empty() => {
next.push(a);
},
// Last item = root.
(Some(a), None) => return Ok(a),
// Finish up, no more items.
_ => {
#[cfg(feature = "debug")]
log::debug!(
"[merkelize_row] Next: {:?}",
next.iter().map(hex::encode).collect::<Vec<_>>()
);
return Err(next)
},
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use hex_literal::hex;
#[test]
fn should_generate_empty_root() {
// given
let _ = env_logger::try_init();
let data: Vec<[u8; 1]> = Default::default();
// when
let out = merkle_root::<Keccak256, _, _>(data);
// then
assert_eq!(
hex::encode(&out),
"0000000000000000000000000000000000000000000000000000000000000000"
);
}
#[test]
fn should_generate_single_root() {
// given
let _ = env_logger::try_init();
let data = vec![hex!("E04CC55ebEE1cBCE552f250e85c57B70B2E2625b")];
// when
let out = merkle_root::<Keccak256, _, _>(data);
// then
assert_eq!(
hex::encode(&out),
"aeb47a269393297f4b0a3c9c9cfd00c7a4195255274cf39d83dabc2fcc9ff3d7"
);
}
#[test]
fn should_generate_root_pow_2() {
// given
let _ = env_logger::try_init();
let data = vec![
hex!("E04CC55ebEE1cBCE552f250e85c57B70B2E2625b"),
hex!("25451A4de12dcCc2D166922fA938E900fCc4ED24"),
];
// when
let out = merkle_root::<Keccak256, _, _>(data);
// then
assert_eq!(
hex::encode(&out),
"697ea2a8fe5b03468548a7a413424a6292ab44a82a6f5cc594c3fa7dda7ce402"
);
}
#[test]
fn should_generate_root_complex() {
let _ = env_logger::try_init();
let test = |root, data| {
assert_eq!(hex::encode(&merkle_root::<Keccak256, _, _>(data)), root);
};
test(
"aff1208e69c9e8be9b584b07ebac4e48a1ee9d15ce3afe20b77a4d29e4175aa3",
vec!["a", "b", "c"],
);
test(
"b8912f7269068901f231a965adfefbc10f0eedcfa61852b103efd54dac7db3d7",
vec!["a", "b", "a"],
);
test(
"dc8e73fe6903148ff5079baecc043983625c23b39f31537e322cd0deee09fa9c",
vec!["a", "b", "a", "b"],
);
test(
"fb3b3be94be9e983ba5e094c9c51a7d96a4fa2e5d8e891df00ca89ba05bb1239",
vec!["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"],
);
}
#[test]
fn should_generate_and_verify_proof_simple() {
// given
let _ = env_logger::try_init();
let data = vec!["a", "b", "c"];
// when
let proof0 = merkle_proof::<Keccak256, _, _>(data.clone(), 0);
assert!(verify_proof::<Keccak256, _, _>(
&proof0.root,
proof0.proof.clone(),
data.len(),
proof0.leaf_index,
&proof0.leaf,
));
let proof1 = merkle_proof::<Keccak256, _, _>(data.clone(), 1);
assert!(verify_proof::<Keccak256, _, _>(
&proof1.root,
proof1.proof,
data.len(),
proof1.leaf_index,
&proof1.leaf,
));
let proof2 = merkle_proof::<Keccak256, _, _>(data.clone(), 2);
assert!(verify_proof::<Keccak256, _, _>(
&proof2.root,
proof2.proof,
data.len(),
proof2.leaf_index,
&proof2.leaf
));
// then
assert_eq!(hex::encode(proof0.root), hex::encode(proof1.root));
assert_eq!(hex::encode(proof2.root), hex::encode(proof1.root));
assert!(!verify_proof::<Keccak256, _, _>(
&hex!("fb3b3be94be9e983ba5e094c9c51a7d96a4fa2e5d8e891df00ca89ba05bb1239"),
proof0.proof,
data.len(),
proof0.leaf_index,
&proof0.leaf
));
assert!(!verify_proof::<Keccak256, _, _>(
&proof0.root,
vec![],
data.len(),
proof0.leaf_index,
&proof0.leaf
));
}
#[test]
fn should_generate_and_verify_proof_complex() {
// given
let _ = env_logger::try_init();
let data = vec!["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"];
for l in 0..data.len() {
// when
let proof = merkle_proof::<Keccak256, _, _>(data.clone(), l);
// then
assert!(verify_proof::<Keccak256, _, _>(
&proof.root,
proof.proof,
data.len(),
proof.leaf_index,
&proof.leaf
));
}
}
#[test]
fn should_generate_and_verify_proof_large() {
// given
let _ = env_logger::try_init();
let mut data = vec![];
for i in 1..16 {
for c in 'a'..'z' {
if c as usize % i != 0 {
data.push(c.to_string());
}
}
for l in 0..data.len() {
// when
let proof = merkle_proof::<Keccak256, _, _>(data.clone(), l);
// then
assert!(verify_proof::<Keccak256, _, _>(
&proof.root,
proof.proof,
data.len(),
proof.leaf_index,
&proof.leaf
));
}
}
}
#[test]
fn should_generate_and_verify_proof_large_tree() {
// given
let _ = env_logger::try_init();
let mut data = vec![];
for i in 0..6000 {
data.push(format!("{}", i));
}
for l in (0..data.len()).step_by(13) {
// when
let proof = merkle_proof::<Keccak256, _, _>(data.clone(), l);
// then
assert!(verify_proof::<Keccak256, _, _>(
&proof.root,
proof.proof,
data.len(),
proof.leaf_index,
&proof.leaf
));
}
}
#[test]
#[should_panic]
fn should_panic_on_invalid_leaf_index() {
let _ = env_logger::try_init();
merkle_proof::<Keccak256, _, _>(vec!["a"], 5);
}
#[test]
fn should_generate_and_verify_proof_on_test_data() {
let addresses = vec![
"0x9aF1Ca5941148eB6A3e9b9C741b69738292C533f",
"0xDD6ca953fddA25c496165D9040F7F77f75B75002",
"0x60e9C47B64Bc1C7C906E891255EaEC19123E7F42",
"0xfa4859480Aa6D899858DE54334d2911E01C070df",
"0x19B9b128470584F7209eEf65B69F3624549Abe6d",
"0xC436aC1f261802C4494504A11fc2926C726cB83b",
"0xc304C8C2c12522F78aD1E28dD86b9947D7744bd0",
"0xDa0C2Cba6e832E55dE89cF4033affc90CC147352",
"0xf850Fd22c96e3501Aad4CDCBf38E4AEC95622411",
"0x684918D4387CEb5E7eda969042f036E226E50642",
"0x963F0A1bFbb6813C0AC88FcDe6ceB96EA634A595",
"0x39B38ad74b8bCc5CE564f7a27Ac19037A95B6099",
"0xC2Dec7Fdd1fef3ee95aD88EC8F3Cd5bd4065f3C7",
"0x9E311f05c2b6A43C2CCF16fB2209491BaBc2ec01",
"0x927607C30eCE4Ef274e250d0bf414d4a210b16f0",
"0x98882bcf85E1E2DFF780D0eB360678C1cf443266",
"0xFBb50191cd0662049E7C4EE32830a4Cc9B353047",
"0x963854fc2C358c48C3F9F0A598B9572c581B8DEF",
"0xF9D7Bc222cF6e3e07bF66711e6f409E51aB75292",
"0xF2E3fd32D063F8bBAcB9e6Ea8101C2edd899AFe6",
"0x407a5b9047B76E8668570120A96d580589fd1325",
"0xEAD9726FAFB900A07dAd24a43AE941d2eFDD6E97",
"0x42f5C8D9384034A9030313B51125C32a526b6ee8",
"0x158fD2529Bc4116570Eb7C80CC76FEf33ad5eD95",
"0x0A436EE2E4dEF3383Cf4546d4278326Ccc82514E",
"0x34229A215db8FeaC93Caf8B5B255e3c6eA51d855",
"0xEb3B7CF8B1840242CB98A732BA464a17D00b5dDF",
"0x2079692bf9ab2d6dc7D79BBDdEE71611E9aA3B72",
"0x46e2A67e5d450e2Cf7317779f8274a2a630f3C9B",
"0xA7Ece4A5390DAB18D08201aE18800375caD78aab",
"0x15E1c0D24D62057Bf082Cb2253dA11Ef0d469570",
"0xADDEF4C9b5687Eb1F7E55F2251916200A3598878",
"0xe0B16Fb96F936035db2b5A68EB37D470fED2f013",
"0x0c9A84993feaa779ae21E39F9793d09e6b69B62D",
"0x3bc4D5148906F70F0A7D1e2756572655fd8b7B34",
"0xFf4675C26903D5319795cbd3a44b109E7DDD9fDe",
"0xCec4450569A8945C6D2Aba0045e4339030128a92",
"0x85f0584B10950E421A32F471635b424063FD8405",
"0xb38bEe7Bdc0bC43c096e206EFdFEad63869929E3",
"0xc9609466274Fef19D0e58E1Ee3b321D5C141067E",
"0xa08EA868cF75268E7401021E9f945BAe73872ecc",
"0x67C9Cb1A29E964Fe87Ff669735cf7eb87f6868fE",
"0x1B6BEF636aFcdd6085cD4455BbcC93796A12F6E2",
"0x46B37b243E09540b55cF91C333188e7D5FD786dD",
"0x8E719E272f62Fa97da93CF9C941F5e53AA09e44a",
"0xa511B7E7DB9cb24AD5c89fBb6032C7a9c2EfA0a5",
"0x4D11FDcAeD335d839132AD450B02af974A3A66f8",
"0xB8cf790a5090E709B4619E1F335317114294E17E",
"0x7f0f57eA064A83210Cafd3a536866ffD2C5eDCB3",
"0xC03C848A4521356EF800e399D889e9c2A25D1f9E",
"0xC6b03DF05cb686D933DD31fCa5A993bF823dc4FE",
"0x58611696b6a8102cf95A32c25612E4cEF32b910F",
"0x2ed4bC7197AEF13560F6771D930Bf907772DE3CE",
"0x3C5E58f334306be029B0e47e119b8977B2639eb4",
"0x288646a1a4FeeC560B349d210263c609aDF649a6",
"0xb4F4981E0d027Dc2B3c86afA0D0fC03d317e83C0",
"0xaAE4A87F8058feDA3971f9DEd639Ec9189aA2500",
"0x355069DA35E598913d8736E5B8340527099960b8",
"0x3cf5A0F274cd243C0A186d9fCBdADad089821B93",
"0xca55155dCc4591538A8A0ca322a56EB0E4aD03C4",
"0xE824D0268366ec5C4F23652b8eD70D552B1F2b8B",
"0x84C3e9B25AE8a9b39FF5E331F9A597F2DCf27Ca9",
"0xcA0018e278751De10d26539915d9c7E7503432FE",
"0xf13077dE6191D6c1509ac7E088b8BE7Fe656c28b",
"0x7a6bcA1ec9Db506e47ac6FD86D001c2aBc59C531",
"0xeA7f9A2A9dd6Ba9bc93ca615C3Ddf26973146911",
"0x8D0d8577e16F8731d4F8712BAbFa97aF4c453458",
"0xB7a7855629dF104246997e9ACa0E6510df75d0ea",
"0x5C1009BDC70b0C8Ab2e5a53931672ab448C17c89",
"0x40B47D1AfefEF5eF41e0789F0285DE7b1C31631C",
"0x5086933d549cEcEB20652CE00973703CF10Da373",
"0xeb364f6FE356882F92ae9314fa96116Cf65F47d8",
"0xdC4D31516A416cEf533C01a92D9a04bbdb85EE67",
"0x9b36E086E5A274332AFd3D8509e12ca5F6af918d",
"0xBC26394fF36e1673aE0608ce91A53B9768aD0D76",
"0x81B5AB400be9e563fA476c100BE898C09966426c",
"0x9d93C8ae5793054D28278A5DE6d4653EC79e90FE",
"0x3B8E75804F71e121008991E3177fc942b6c28F50",
"0xC6Eb5886eB43dD473f5BB4e21e56E08dA464D9B4",
"0xfdf1277b71A73c813cD0e1a94B800f4B1Db66DBE",
"0xc2ff2cCc98971556670e287Ff0CC39DA795231ad",
"0x76b7E1473f0D0A87E9B4a14E2B179266802740f5",
"0xA7Bc965660a6EF4687CCa4F69A97563163A3C2Ef",
"0xB9C2b47888B9F8f7D03dC1de83F3F55E738CebD3",
"0xEd400162E6Dd6bD2271728FFb04176bF770De94a",
"0xE3E8331156700339142189B6E555DCb2c0962750",
"0xbf62e342Bc7706a448EdD52AE871d9C4497A53b1",
"0xb9d7A1A111eed75714a0AcD2dd467E872eE6B03D",
"0x03942919DFD0383b8c574AB8A701d89fd4bfA69D",
"0x0Ef4C92355D3c8c7050DFeb319790EFCcBE6fe9e",
"0xA6895a3cf0C60212a73B3891948ACEcF1753f25E",
"0x0Ed509239DB59ef3503ded3d31013C983d52803A",
"0xc4CE8abD123BfAFc4deFf37c7D11DeCd5c350EE4",
"0x4A4Bf59f7038eDcd8597004f35d7Ee24a7Bdd2d3",
"0x5769E8e8A2656b5ed6b6e6fa2a2bFAeaf970BB87",
"0xf9E15cCE181332F4F57386687c1776b66C377060",
"0xc98f8d4843D56a46C21171900d3eE538Cc74dbb5",
"0x3605965B47544Ce4302b988788B8195601AE4dEd",
"0xe993BDfdcAac2e65018efeE0F69A12678031c71d",
"0x274fDf8801385D3FAc954BCc1446Af45f5a8304c",
"0xBFb3f476fcD6429F4a475bA23cEFdDdd85c6b964",
"0x806cD16588Fe812ae740e931f95A289aFb4a4B50",
"0xa89488CE3bD9C25C3aF797D1bbE6CA689De79d81",
"0xd412f1AfAcf0Ebf3Cd324593A231Fc74CC488B12",
"0xd1f715b2D7951d54bc31210BbD41852D9BF98Ed1",
"0xf65aD707c344171F467b2ADba3d14f312219cE23",
"0x2971a4b242e9566dEF7bcdB7347f5E484E11919B",
"0x12b113D6827E07E7D426649fBd605f427da52314",
"0x1c6CA45171CDb9856A6C9Dba9c5F1216913C1e97",
"0x11cC6ee1d74963Db23294FCE1E3e0A0555779CeA",
"0x8Aa1C721255CDC8F895E4E4c782D86726b068667",
"0xA2cDC1f37510814485129aC6310b22dF04e9Bbf0",
"0xCf531b71d388EB3f5889F1f78E0d77f6fb109767",
"0xBe703e3545B2510979A0cb0C440C0Fba55c6dCB5",
"0x30a35886F989db39c797D8C93880180Fdd71b0c8",
"0x1071370D981F60c47A9Cd27ac0A61873a372cBB2",
"0x3515d74A11e0Cb65F0F46cB70ecf91dD1712daaa",
"0x50500a3c2b7b1229c6884505D00ac6Be29Aecd0C",
"0x9A223c2a11D4FD3585103B21B161a2B771aDA3d1",
"0xd7218df03AD0907e6c08E707B15d9BD14285e657",
"0x76CfD72eF5f93D1a44aD1F80856797fBE060c70a",
"0x44d093cB745944991EFF5cBa151AA6602d6f5420",
"0x626516DfF43bf09A71eb6fd1510E124F96ED0Cde",
"0x6530824632dfe099304E2DC5701cA99E6d031E08",
"0x57e6c423d6a7607160d6379A0c335025A14DaFC0",
"0x3966D4AD461Ef150E0B10163C81E79b9029E69c3",
"0xF608aCfd0C286E23721a3c347b2b65039f6690F1",
"0xbfB8FAac31A25646681936977837f7740fCd0072",
"0xd80aa634a623a7ED1F069a1a3A28a173061705c7",
"0x9122a77B36363e24e12E1E2D73F87b32926D3dF5",
"0x62562f0d1cD31315bCCf176049B6279B2bfc39C2",
"0x48aBF7A2a7119e5675059E27a7082ba7F38498b2",
"0xb4596983AB9A9166b29517acD634415807569e5F",
"0x52519D16E20BC8f5E96Da6d736963e85b2adA118",
"0x7663893C3dC0850EfC5391f5E5887eD723e51B83",
"0x5FF323a29bCC3B5b4B107e177EccEF4272959e61",
"0xee6e499AdDf4364D75c05D50d9344e9daA5A9AdF",
"0x1631b0BD31fF904aD67dD58994C6C2051CDe4E75",
"0xbc208e9723D44B9811C428f6A55722a26204eEF2",
"0xe76103a222Ee2C7Cf05B580858CEe625C4dc00E1",
"0xC71Bb2DBC51760f4fc2D46D84464410760971B8a",
"0xB4C18811e6BFe564D69E12c224FFc57351f7a7ff",
"0xD11DB0F5b41061A887cB7eE9c8711438844C298A",
"0xB931269934A3D4432c084bAAc3d0de8143199F4f",
"0x070037cc85C761946ec43ea2b8A2d5729908A2a1",
"0x2E34aa8C95Ffdbb37f14dCfBcA69291c55Ba48DE",
"0x052D93e8d9220787c31d6D83f87eC7dB088E998f",
"0x498dAC6C69b8b9ad645217050054840f1D91D029",
"0xE4F7D60f9d84301e1fFFd01385a585F3A11F8E89",
"0xEa637992f30eA06460732EDCBaCDa89355c2a107",
"0x4960d8Da07c27CB6Be48a79B96dD70657c57a6bF",
"0x7e471A003C8C9fdc8789Ded9C3dbe371d8aa0329",
"0xd24265Cc10eecb9e8d355CCc0dE4b11C556E74D7",
"0xDE59C8f7557Af779674f41CA2cA855d571018690",
"0x2fA8A6b3b6226d8efC9d8f6EBDc73Ca33DDcA4d8",
"0xe44102664c6c2024673Ff07DFe66E187Db77c65f",
"0x94E3f4f90a5f7CBF2cc2623e66B8583248F01022",
"0x0383EdBbc21D73DEd039E9C1Ff6bf56017b4CC40",
"0x64C3E49898B88d1E0f0d02DA23E0c00A2Cd0cA99",
"0xF4ccfB67b938d82B70bAb20975acFAe402E812E1",
"0x4f9ee5829e9852E32E7BC154D02c91D8E203e074",
"0xb006312eF9713463bB33D22De60444Ba95609f6B",
"0x7Cbe76ef69B52110DDb2e3b441C04dDb11D63248",
"0x70ADEEa65488F439392B869b1Df7241EF317e221",
"0x64C0bf8AA36Ba590477585Bc0D2BDa7970769463",
"0xA4cDc98593CE52d01Fe5Ca47CB3dA5320e0D7592",
"0xc26B34D375533fFc4c5276282Fa5D660F3d8cbcB",
];
let root = hex!("72b0acd7c302a84f1f6b6cefe0ba7194b7398afb440e1b44a9dbbe270394ca53");
let data = addresses
.into_iter()
.map(|address| hex::decode(&address[2..]).unwrap())
.collect::<Vec<_>>();
for l in 0..data.len() {
// when
let proof = merkle_proof::<Keccak256, _, _>(data.clone(), l);
assert_eq!(hex::encode(&proof.root), hex::encode(&root));
assert_eq!(proof.leaf_index, l);
assert_eq!(&proof.leaf, &data[l]);
// then
assert!(verify_proof::<Keccak256, _, _>(
&proof.root,
proof.proof,
data.len(),
proof.leaf_index,
&proof.leaf
));
}
let proof = merkle_proof::<Keccak256, _, _>(data.clone(), data.len() - 1);
assert_eq!(
proof,
MerkleProof {
root,
proof: vec![
hex!("340bcb1d49b2d82802ddbcf5b85043edb3427b65d09d7f758fbc76932ad2da2f"),
hex!("ba0580e5bd530bc93d61276df7969fb5b4ae8f1864b4a28c280249575198ff1f"),
hex!("d02609d2bbdb28aa25f58b85afec937d5a4c85d37925bce6d0cf802f9d76ba79"),
hex!("ae3f8991955ed884613b0a5f40295902eea0e0abe5858fc520b72959bc016d4e"),
],
number_of_leaves: data.len(),
leaf_index: data.len() - 1,
leaf: hex!("c26B34D375533fFc4c5276282Fa5D660F3d8cbcB").to_vec(),
}
);
}
}
+236
View File
@@ -0,0 +1,236 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#![cfg_attr(not(feature = "std"), no_std)]
#![warn(missing_docs)]
//! A BEEFY+MMR pallet combo.
//!
//! While both BEEFY and Merkle Mountain Range (MMR) can be used separately,
//! these tools were designed to work together in unison.
//!
//! The pallet provides a standardized MMR Leaf format that is can be used
//! to bridge BEEFY+MMR-based networks (both standalone and polkadot-like).
//!
//! The MMR leaf contains:
//! 1. Block number and parent block hash.
//! 2. Merkle Tree Root Hash of next BEEFY validator set.
//! 3. Merkle Tree Root Hash of current parachain heads state.
//!
//! and thanks to versioning can be easily updated in the future.
use sp_runtime::traits::{Convert, Hash};
use sp_std::prelude::*;
use beefy_primitives::mmr::{BeefyNextAuthoritySet, MmrLeaf, MmrLeafVersion};
use pallet_mmr::primitives::LeafDataProvider;
use codec::Encode;
use frame_support::traits::Get;
pub use pallet::*;
#[cfg(test)]
mod mock;
#[cfg(test)]
mod tests;
/// A BEEFY consensus digest item with MMR root hash.
pub struct DepositBeefyDigest<T>(sp_std::marker::PhantomData<T>);
impl<T> pallet_mmr::primitives::OnNewRoot<beefy_primitives::MmrRootHash> for DepositBeefyDigest<T>
where
T: pallet_mmr::Config<Hash = beefy_primitives::MmrRootHash>,
T: pallet_beefy::Config,
{
fn on_new_root(root: &<T as pallet_mmr::Config>::Hash) {
let digest = sp_runtime::generic::DigestItem::Consensus(
beefy_primitives::BEEFY_ENGINE_ID,
codec::Encode::encode(&beefy_primitives::ConsensusLog::<
<T as pallet_beefy::Config>::BeefyId,
>::MmrRoot(*root)),
);
<frame_system::Pallet<T>>::deposit_log(digest);
}
}
/// Convert BEEFY secp256k1 public keys into Ethereum addresses
pub struct BeefyEcdsaToEthereum;
impl Convert<beefy_primitives::crypto::AuthorityId, Vec<u8>> for BeefyEcdsaToEthereum {
fn convert(a: beefy_primitives::crypto::AuthorityId) -> Vec<u8> {
use sp_core::crypto::Public;
let compressed_key = a.as_slice();
libsecp256k1::PublicKey::parse_slice(
compressed_key,
Some(libsecp256k1::PublicKeyFormat::Compressed),
)
// uncompress the key
.map(|pub_key| pub_key.serialize().to_vec())
// now convert to ETH address
.map(|uncompressed| sp_io::hashing::keccak_256(&uncompressed[1..])[12..].to_vec())
.map_err(|_| {
log::error!(target: "runtime::beefy", "Invalid BEEFY PublicKey format!");
})
.unwrap_or_default()
}
}
type MerkleRootOf<T> = <T as pallet_mmr::Config>::Hash;
type ParaId = u32;
type ParaHead = Vec<u8>;
/// A type that is able to return current list of parachain heads that end up in the MMR leaf.
pub trait ParachainHeadsProvider {
/// Return a list of tuples containing a `ParaId` and Parachain Header data (ParaHead).
///
/// The returned data does not have to be sorted.
fn parachain_heads() -> Vec<(ParaId, ParaHead)>;
}
/// A default implementation for runtimes without parachains.
impl ParachainHeadsProvider for () {
fn parachain_heads() -> Vec<(ParaId, ParaHead)> {
Default::default()
}
}
#[frame_support::pallet]
pub mod pallet {
#![allow(missing_docs)]
use super::*;
use frame_support::pallet_prelude::*;
/// BEEFY-MMR pallet.
#[pallet::pallet]
#[pallet::generate_store(pub(super) trait Store)]
pub struct Pallet<T>(_);
/// The module's configuration trait.
#[pallet::config]
#[pallet::disable_frame_system_supertrait_check]
pub trait Config: pallet_mmr::Config + pallet_beefy::Config {
/// Current leaf version.
///
/// Specifies the version number added to every leaf that get's appended to the MMR.
/// Read more in [`MmrLeafVersion`] docs about versioning leaves.
type LeafVersion: Get<MmrLeafVersion>;
/// Convert BEEFY AuthorityId to a form that would end up in the Merkle Tree.
///
/// For instance for ECDSA (secp256k1) we want to store uncompressed public keys (65 bytes)
/// and later to Ethereum Addresses (160 bits) to simplify using them on Ethereum chain,
/// but the rest of the Substrate codebase is storing them compressed (33 bytes) for
/// efficiency reasons.
type BeefyAuthorityToMerkleLeaf: Convert<<Self as pallet_beefy::Config>::BeefyId, Vec<u8>>;
/// Retrieve a list of current parachain heads.
///
/// The trait is implemented for `paras` module, but since not all chains might have
/// parachains, and we want to keep the MMR leaf structure uniform, it's possible to use
/// `()` as well to simply put dummy data to the leaf.
type ParachainHeads: ParachainHeadsProvider;
}
/// Details of next BEEFY authority set.
///
/// This storage entry is used as cache for calls to [`update_beefy_next_authority_set`].
#[pallet::storage]
#[pallet::getter(fn beefy_next_authorities)]
pub type BeefyNextAuthorities<T: Config> =
StorageValue<_, BeefyNextAuthoritySet<MerkleRootOf<T>>, ValueQuery>;
}
impl<T: Config> LeafDataProvider for Pallet<T>
where
MerkleRootOf<T>: From<beefy_merkle_tree::Hash> + Into<beefy_merkle_tree::Hash>,
{
type LeafData = MmrLeaf<
<T as frame_system::Config>::BlockNumber,
<T as frame_system::Config>::Hash,
MerkleRootOf<T>,
>;
fn leaf_data() -> Self::LeafData {
MmrLeaf {
version: T::LeafVersion::get(),
parent_number_and_hash: frame_system::Pallet::<T>::leaf_data(),
parachain_heads: Pallet::<T>::parachain_heads_merkle_root(),
beefy_next_authority_set: Pallet::<T>::update_beefy_next_authority_set(),
}
}
}
impl<T: Config> beefy_merkle_tree::Hasher for Pallet<T>
where
MerkleRootOf<T>: Into<beefy_merkle_tree::Hash>,
{
fn hash(data: &[u8]) -> beefy_merkle_tree::Hash {
<T as pallet_mmr::Config>::Hashing::hash(data).into()
}
}
impl<T: Config> Pallet<T>
where
MerkleRootOf<T>: From<beefy_merkle_tree::Hash> + Into<beefy_merkle_tree::Hash>,
{
/// Returns latest root hash of a merkle tree constructed from all active parachain headers.
///
/// The leafs are sorted by `ParaId` to allow more efficient lookups and non-existence proofs.
///
/// NOTE this does not include parathreads - only parachains are part of the merkle tree.
///
/// NOTE This is an initial and inefficient implementation, which re-constructs
/// the merkle tree every block. Instead we should update the merkle root in
/// [Self::on_initialize] call of this pallet and update the merkle tree efficiently (use
/// on-chain storage to persist inner nodes).
fn parachain_heads_merkle_root() -> MerkleRootOf<T> {
let mut para_heads = T::ParachainHeads::parachain_heads();
para_heads.sort();
let para_heads = para_heads.into_iter().map(|pair| pair.encode());
beefy_merkle_tree::merkle_root::<Self, _, _>(para_heads).into()
}
/// Returns details of the next BEEFY authority set.
///
/// Details contain authority set id, authority set length and a merkle root,
/// constructed from uncompressed secp256k1 public keys converted to Ethereum addresses
/// of the next BEEFY authority set.
///
/// This function will use a storage-cached entry in case the set didn't change, or compute and
/// cache new one in case it did.
fn update_beefy_next_authority_set() -> BeefyNextAuthoritySet<MerkleRootOf<T>> {
let id = pallet_beefy::Pallet::<T>::validator_set_id() + 1;
let current_next = Self::beefy_next_authorities();
// avoid computing the merkle tree if validator set id didn't change.
if id == current_next.id {
return current_next
}
let beefy_addresses = pallet_beefy::Pallet::<T>::next_authorities()
.into_iter()
.map(T::BeefyAuthorityToMerkleLeaf::convert)
.collect::<Vec<_>>();
let len = beefy_addresses.len() as u32;
let root = beefy_merkle_tree::merkle_root::<Self, _, _>(beefy_addresses).into();
let next_set = BeefyNextAuthoritySet { id, len, root };
// cache the result
BeefyNextAuthorities::<T>::put(&next_set);
next_set
}
}
+205
View File
@@ -0,0 +1,205 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::vec;
use beefy_primitives::mmr::MmrLeafVersion;
use frame_support::{
construct_runtime, parameter_types, sp_io::TestExternalities, BasicExternalities,
};
use sp_core::{Hasher, H256};
use sp_runtime::{
app_crypto::ecdsa::Public,
impl_opaque_keys,
testing::Header,
traits::{BlakeTwo256, ConvertInto, IdentityLookup, Keccak256, OpaqueKeys},
Perbill,
};
use crate as pallet_beefy_mmr;
pub use beefy_primitives::{crypto::AuthorityId as BeefyId, ConsensusLog, BEEFY_ENGINE_ID};
impl_opaque_keys! {
pub struct MockSessionKeys {
pub dummy: pallet_beefy::Pallet<Test>,
}
}
type UncheckedExtrinsic = frame_system::mocking::MockUncheckedExtrinsic<Test>;
type Block = frame_system::mocking::MockBlock<Test>;
construct_runtime!(
pub enum Test where
Block = Block,
NodeBlock = Block,
UncheckedExtrinsic = UncheckedExtrinsic,
{
System: frame_system::{Pallet, Call, Config, Storage, Event<T>},
Session: pallet_session::{Pallet, Call, Storage, Event, Config<T>},
Mmr: pallet_mmr::{Pallet, Storage},
Beefy: pallet_beefy::{Pallet, Config<T>, Storage},
BeefyMmr: pallet_beefy_mmr::{Pallet, Storage},
}
);
parameter_types! {
pub const BlockHashCount: u64 = 250;
pub const SS58Prefix: u8 = 42;
}
impl frame_system::Config for Test {
type BaseCallFilter = frame_support::traits::Everything;
type BlockWeights = ();
type BlockLength = ();
type DbWeight = ();
type Origin = Origin;
type Index = u64;
type BlockNumber = u64;
type Hash = H256;
type Call = Call;
type Hashing = BlakeTwo256;
type AccountId = u64;
type Lookup = IdentityLookup<Self::AccountId>;
type Header = Header;
type Event = Event;
type BlockHashCount = BlockHashCount;
type Version = ();
type PalletInfo = PalletInfo;
type AccountData = ();
type OnNewAccount = ();
type OnKilledAccount = ();
type SystemWeightInfo = ();
type SS58Prefix = SS58Prefix;
type OnSetCode = ();
}
parameter_types! {
pub const Period: u64 = 1;
pub const Offset: u64 = 0;
pub const DisabledValidatorsThreshold: Perbill = Perbill::from_percent(33);
}
impl pallet_session::Config for Test {
type Event = Event;
type ValidatorId = u64;
type ValidatorIdOf = ConvertInto;
type ShouldEndSession = pallet_session::PeriodicSessions<Period, Offset>;
type NextSessionRotation = pallet_session::PeriodicSessions<Period, Offset>;
type SessionManager = MockSessionManager;
type SessionHandler = <MockSessionKeys as OpaqueKeys>::KeyTypeIdProviders;
type Keys = MockSessionKeys;
type DisabledValidatorsThreshold = DisabledValidatorsThreshold;
type WeightInfo = ();
}
pub type MmrLeaf = beefy_primitives::mmr::MmrLeaf<
<Test as frame_system::Config>::BlockNumber,
<Test as frame_system::Config>::Hash,
<Test as pallet_mmr::Config>::Hash,
>;
impl pallet_mmr::Config for Test {
const INDEXING_PREFIX: &'static [u8] = b"mmr";
type Hashing = Keccak256;
type Hash = <Keccak256 as Hasher>::Out;
type LeafData = BeefyMmr;
type OnNewRoot = pallet_beefy_mmr::DepositBeefyDigest<Test>;
type WeightInfo = ();
}
impl pallet_beefy::Config for Test {
type BeefyId = BeefyId;
}
parameter_types! {
pub LeafVersion: MmrLeafVersion = MmrLeafVersion::new(1, 5);
}
impl pallet_beefy_mmr::Config for Test {
type LeafVersion = LeafVersion;
type BeefyAuthorityToMerkleLeaf = pallet_beefy_mmr::BeefyEcdsaToEthereum;
type ParachainHeads = DummyParaHeads;
}
pub struct DummyParaHeads;
impl pallet_beefy_mmr::ParachainHeadsProvider for DummyParaHeads {
fn parachain_heads() -> Vec<(pallet_beefy_mmr::ParaId, pallet_beefy_mmr::ParaHead)> {
vec![(15, vec![1, 2, 3]), (5, vec![4, 5, 6])]
}
}
pub struct MockSessionManager;
impl pallet_session::SessionManager<u64> for MockSessionManager {
fn end_session(_: sp_staking::SessionIndex) {}
fn start_session(_: sp_staking::SessionIndex) {}
fn new_session(idx: sp_staking::SessionIndex) -> Option<Vec<u64>> {
if idx == 0 || idx == 1 {
Some(vec![1, 2])
} else if idx == 2 {
Some(vec![3, 4])
} else {
None
}
}
}
// Note, that we can't use `UintAuthorityId` here. Reason is that the implementation
// of `to_public_key()` assumes, that a public key is 32 bytes long. This is true for
// ed25519 and sr25519 but *not* for ecdsa. An ecdsa public key is 33 bytes.
pub fn mock_beefy_id(id: u8) -> BeefyId {
let buf: [u8; 33] = [id; 33];
let pk = Public::from_raw(buf);
BeefyId::from(pk)
}
pub fn mock_authorities(vec: Vec<u8>) -> Vec<(u64, BeefyId)> {
vec.into_iter().map(|id| ((id as u64), mock_beefy_id(id))).collect()
}
pub fn new_test_ext(ids: Vec<u8>) -> TestExternalities {
new_test_ext_raw_authorities(mock_authorities(ids))
}
pub fn new_test_ext_raw_authorities(authorities: Vec<(u64, BeefyId)>) -> TestExternalities {
let mut t = frame_system::GenesisConfig::default().build_storage::<Test>().unwrap();
let session_keys: Vec<_> = authorities
.iter()
.enumerate()
.map(|(_, id)| (id.0 as u64, id.0 as u64, MockSessionKeys { dummy: id.1.clone() }))
.collect();
BasicExternalities::execute_with_storage(&mut t, || {
for (ref id, ..) in &session_keys {
frame_system::Pallet::<Test>::inc_providers(id);
}
});
pallet_session::GenesisConfig::<Test> { keys: session_keys }
.assimilate_storage(&mut t)
.unwrap();
t.into()
}
+148
View File
@@ -0,0 +1,148 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::vec;
use beefy_primitives::{
mmr::{BeefyNextAuthoritySet, MmrLeafVersion},
ValidatorSet,
};
use codec::{Decode, Encode};
use hex_literal::hex;
use sp_core::H256;
use sp_io::TestExternalities;
use sp_runtime::{traits::Keccak256, DigestItem};
use frame_support::traits::OnInitialize;
use crate::mock::*;
fn init_block(block: u64) {
System::set_block_number(block);
Session::on_initialize(block);
Mmr::on_initialize(block);
Beefy::on_initialize(block);
BeefyMmr::on_initialize(block);
}
pub fn beefy_log(log: ConsensusLog<BeefyId>) -> DigestItem<H256> {
DigestItem::Consensus(BEEFY_ENGINE_ID, log.encode())
}
fn offchain_key(pos: usize) -> Vec<u8> {
(<Test as pallet_mmr::Config>::INDEXING_PREFIX, pos as u64).encode()
}
fn read_mmr_leaf(ext: &mut TestExternalities, index: usize) -> MmrLeaf {
type Node = pallet_mmr_primitives::DataOrHash<Keccak256, MmrLeaf>;
ext.persist_offchain_overlay();
let offchain_db = ext.offchain_db();
offchain_db
.get(&offchain_key(index))
.map(|d| Node::decode(&mut &*d).unwrap())
.map(|n| match n {
Node::Data(d) => d,
_ => panic!("Unexpected MMR node."),
})
.unwrap()
}
#[test]
fn should_contain_mmr_digest() {
let mut ext = new_test_ext(vec![1, 2, 3, 4]);
ext.execute_with(|| {
init_block(1);
assert_eq!(
System::digest().logs,
vec![beefy_log(ConsensusLog::MmrRoot(
hex!("f3e3afbfa69e89cd1e99f8d3570155962f3346d1d8758dc079be49ef70387758").into()
))]
);
// unique every time
init_block(2);
assert_eq!(
System::digest().logs,
vec![
beefy_log(ConsensusLog::MmrRoot(
hex!("f3e3afbfa69e89cd1e99f8d3570155962f3346d1d8758dc079be49ef70387758").into()
)),
beefy_log(ConsensusLog::AuthoritiesChange(ValidatorSet {
validators: vec![mock_beefy_id(3), mock_beefy_id(4),],
id: 1,
})),
beefy_log(ConsensusLog::MmrRoot(
hex!("7d4ae4524bae75d52b63f08eab173b0c263eb95ae2c55c3a1d871241bd0cc559").into()
)),
]
);
});
}
#[test]
fn should_contain_valid_leaf_data() {
let mut ext = new_test_ext(vec![1, 2, 3, 4]);
ext.execute_with(|| {
init_block(1);
});
let mmr_leaf = read_mmr_leaf(&mut ext, 0);
assert_eq!(
mmr_leaf,
MmrLeaf {
version: MmrLeafVersion::new(1, 5),
parent_number_and_hash: (0_u64, H256::repeat_byte(0x45)),
beefy_next_authority_set: BeefyNextAuthoritySet {
id: 1,
len: 2,
root: hex!("01b1a742589773fc054c8f5021a456316ffcec0370b25678b0696e116d1ef9ae")
.into(),
},
parachain_heads: hex!(
"ed893c8f8cc87195a5d4d2805b011506322036bcace79642aa3e94ab431e442e"
)
.into(),
}
);
// build second block on top
ext.execute_with(|| {
init_block(2);
});
let mmr_leaf = read_mmr_leaf(&mut ext, 1);
assert_eq!(
mmr_leaf,
MmrLeaf {
version: MmrLeafVersion::new(1, 5),
parent_number_and_hash: (1_u64, H256::repeat_byte(0x45)),
beefy_next_authority_set: BeefyNextAuthoritySet {
id: 2,
len: 2,
root: hex!("9c6b2c1b0d0b25a008e6c882cc7b415f309965c72ad2b944ac0931048ca31cd5")
.into(),
},
parachain_heads: hex!(
"ed893c8f8cc87195a5d4d2805b011506322036bcace79642aa3e94ab431e442e"
)
.into(),
}
);
}
+40
View File
@@ -0,0 +1,40 @@
[package]
name = "pallet-beefy"
version = "4.0.0-dev"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
license = "Apache-2.0"
[dependencies]
codec = { version = "2.2.0", package = "parity-scale-codec", default-features = false, features = ["derive"] }
scale-info = { version = "1.0", default-features = false, features = ["derive"] }
serde = { version = "1.0.130", optional = true }
frame-support = { version = "4.0.0-dev", path = "../support", default-features = false }
frame-system = { version = "4.0.0-dev", path = "../system", default-features = false }
sp-runtime = { version = "4.0.0-dev", path = "../../primitives/runtime", default-features = false }
sp-std = { version = "4.0.0-dev", path = "../../primitives/std", default-features = false }
pallet-session = { version = "4.0.0-dev", path = "../session", default-features = false }
beefy-primitives = { version = "4.0.0-dev", path = "../../primitives/beefy", default-features = false }
[dev-dependencies]
sp-core = { version = "4.0.0-dev", path = "../../primitives/core" }
sp-io = { version = "4.0.0-dev", path = "../../primitives/io" }
sp-staking = { version = "4.0.0-dev", path = "../../primitives/staking" }
[features]
default = ["std"]
std = [
"codec/std",
"scale-info/std",
"serde",
"beefy-primitives/std",
"frame-support/std",
"frame-system/std",
"sp-runtime/std",
"sp-std/std",
"pallet-session/std",
]
+179
View File
@@ -0,0 +1,179 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#![cfg_attr(not(feature = "std"), no_std)]
use codec::Encode;
use frame_support::{traits::OneSessionHandler, Parameter};
use sp_runtime::{
generic::DigestItem,
traits::{IsMember, Member},
RuntimeAppPublic,
};
use sp_std::prelude::*;
use beefy_primitives::{AuthorityIndex, ConsensusLog, ValidatorSet, BEEFY_ENGINE_ID};
#[cfg(test)]
mod mock;
#[cfg(test)]
mod tests;
pub use pallet::*;
#[frame_support::pallet]
pub mod pallet {
use super::*;
use frame_support::pallet_prelude::*;
use frame_system::pallet_prelude::*;
#[pallet::config]
pub trait Config: frame_system::Config {
/// Authority identifier type
type BeefyId: Member + Parameter + RuntimeAppPublic + Default + MaybeSerializeDeserialize;
}
#[pallet::pallet]
pub struct Pallet<T>(PhantomData<T>);
#[pallet::hooks]
impl<T: Config> Hooks<BlockNumberFor<T>> for Pallet<T> {}
#[pallet::call]
impl<T: Config> Pallet<T> {}
/// The current authorities set
#[pallet::storage]
#[pallet::getter(fn authorities)]
pub(super) type Authorities<T: Config> = StorageValue<_, Vec<T::BeefyId>, ValueQuery>;
/// The current validator set id
#[pallet::storage]
#[pallet::getter(fn validator_set_id)]
pub(super) type ValidatorSetId<T: Config> =
StorageValue<_, beefy_primitives::ValidatorSetId, ValueQuery>;
/// Authorities set scheduled to be used with the next session
#[pallet::storage]
#[pallet::getter(fn next_authorities)]
pub(super) type NextAuthorities<T: Config> = StorageValue<_, Vec<T::BeefyId>, ValueQuery>;
#[pallet::genesis_config]
pub struct GenesisConfig<T: Config> {
pub authorities: Vec<T::BeefyId>,
}
#[cfg(feature = "std")]
impl<T: Config> Default for GenesisConfig<T> {
fn default() -> Self {
Self { authorities: Vec::new() }
}
}
#[pallet::genesis_build]
impl<T: Config> GenesisBuild<T> for GenesisConfig<T> {
fn build(&self) {
Pallet::<T>::initialize_authorities(&self.authorities);
}
}
}
impl<T: Config> Pallet<T> {
/// Return the current active BEEFY validator set.
pub fn validator_set() -> ValidatorSet<T::BeefyId> {
ValidatorSet::<T::BeefyId> { validators: Self::authorities(), id: Self::validator_set_id() }
}
fn change_authorities(new: Vec<T::BeefyId>, queued: Vec<T::BeefyId>) {
// As in GRANDPA, we trigger a validator set change only if the the validator
// set has actually changed.
if new != Self::authorities() {
<Authorities<T>>::put(&new);
let next_id = Self::validator_set_id() + 1u64;
<ValidatorSetId<T>>::put(next_id);
let log: DigestItem<T::Hash> = DigestItem::Consensus(
BEEFY_ENGINE_ID,
ConsensusLog::AuthoritiesChange(ValidatorSet { validators: new, id: next_id })
.encode(),
);
<frame_system::Pallet<T>>::deposit_log(log);
}
<NextAuthorities<T>>::put(&queued);
}
fn initialize_authorities(authorities: &[T::BeefyId]) {
if authorities.is_empty() {
return
}
assert!(<Authorities<T>>::get().is_empty(), "Authorities are already initialized!");
<Authorities<T>>::put(authorities);
<ValidatorSetId<T>>::put(0);
// Like `pallet_session`, initialize the next validator set as well.
<NextAuthorities<T>>::put(authorities);
}
}
impl<T: Config> sp_runtime::BoundToRuntimeAppPublic for Pallet<T> {
type Public = T::BeefyId;
}
impl<T: Config> OneSessionHandler<T::AccountId> for Pallet<T> {
type Key = T::BeefyId;
fn on_genesis_session<'a, I: 'a>(validators: I)
where
I: Iterator<Item = (&'a T::AccountId, T::BeefyId)>,
{
let authorities = validators.map(|(_, k)| k).collect::<Vec<_>>();
Self::initialize_authorities(&authorities);
}
fn on_new_session<'a, I: 'a>(changed: bool, validators: I, queued_validators: I)
where
I: Iterator<Item = (&'a T::AccountId, T::BeefyId)>,
{
if changed {
let next_authorities = validators.map(|(_, k)| k).collect::<Vec<_>>();
let next_queued_authorities = queued_validators.map(|(_, k)| k).collect::<Vec<_>>();
Self::change_authorities(next_authorities, next_queued_authorities);
}
}
fn on_disabled(i: usize) {
let log: DigestItem<T::Hash> = DigestItem::Consensus(
BEEFY_ENGINE_ID,
ConsensusLog::<T::BeefyId>::OnDisabled(i as AuthorityIndex).encode(),
);
<frame_system::Pallet<T>>::deposit_log(log);
}
}
impl<T: Config> IsMember<T::BeefyId> for Pallet<T> {
fn is_member(authority_id: &T::BeefyId) -> bool {
Self::authorities().iter().any(|id| id == authority_id)
}
}
+164
View File
@@ -0,0 +1,164 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::vec;
use frame_support::{
construct_runtime, parameter_types, sp_io::TestExternalities, BasicExternalities,
};
use sp_core::H256;
use sp_runtime::{
app_crypto::ecdsa::Public,
impl_opaque_keys,
testing::Header,
traits::{BlakeTwo256, ConvertInto, IdentityLookup, OpaqueKeys},
Perbill,
};
use crate as pallet_beefy;
pub use beefy_primitives::{crypto::AuthorityId as BeefyId, ConsensusLog, BEEFY_ENGINE_ID};
impl_opaque_keys! {
pub struct MockSessionKeys {
pub dummy: pallet_beefy::Pallet<Test>,
}
}
type UncheckedExtrinsic = frame_system::mocking::MockUncheckedExtrinsic<Test>;
type Block = frame_system::mocking::MockBlock<Test>;
construct_runtime!(
pub enum Test where
Block = Block,
NodeBlock = Block,
UncheckedExtrinsic = UncheckedExtrinsic,
{
System: frame_system::{Pallet, Call, Config, Storage, Event<T>},
Beefy: pallet_beefy::{Pallet, Call, Config<T>, Storage},
Session: pallet_session::{Pallet, Call, Storage, Event, Config<T>},
}
);
parameter_types! {
pub const BlockHashCount: u64 = 250;
pub const SS58Prefix: u8 = 42;
}
impl frame_system::Config for Test {
type BaseCallFilter = frame_support::traits::Everything;
type BlockWeights = ();
type BlockLength = ();
type DbWeight = ();
type Origin = Origin;
type Index = u64;
type BlockNumber = u64;
type Hash = H256;
type Call = Call;
type Hashing = BlakeTwo256;
type AccountId = u64;
type Lookup = IdentityLookup<Self::AccountId>;
type Header = Header;
type Event = Event;
type BlockHashCount = BlockHashCount;
type Version = ();
type PalletInfo = PalletInfo;
type AccountData = ();
type OnNewAccount = ();
type OnKilledAccount = ();
type SystemWeightInfo = ();
type SS58Prefix = SS58Prefix;
type OnSetCode = ();
}
impl pallet_beefy::Config for Test {
type BeefyId = BeefyId;
}
parameter_types! {
pub const Period: u64 = 1;
pub const Offset: u64 = 0;
pub const DisabledValidatorsThreshold: Perbill = Perbill::from_percent(33);
}
impl pallet_session::Config for Test {
type Event = Event;
type ValidatorId = u64;
type ValidatorIdOf = ConvertInto;
type ShouldEndSession = pallet_session::PeriodicSessions<Period, Offset>;
type NextSessionRotation = pallet_session::PeriodicSessions<Period, Offset>;
type SessionManager = MockSessionManager;
type SessionHandler = <MockSessionKeys as OpaqueKeys>::KeyTypeIdProviders;
type Keys = MockSessionKeys;
type DisabledValidatorsThreshold = DisabledValidatorsThreshold;
type WeightInfo = ();
}
pub struct MockSessionManager;
impl pallet_session::SessionManager<u64> for MockSessionManager {
fn end_session(_: sp_staking::SessionIndex) {}
fn start_session(_: sp_staking::SessionIndex) {}
fn new_session(idx: sp_staking::SessionIndex) -> Option<Vec<u64>> {
if idx == 0 || idx == 1 {
Some(vec![1, 2])
} else if idx == 2 {
Some(vec![3, 4])
} else {
None
}
}
}
// Note, that we can't use `UintAuthorityId` here. Reason is that the implementation
// of `to_public_key()` assumes, that a public key is 32 bytes long. This is true for
// ed25519 and sr25519 but *not* for ecdsa. An ecdsa public key is 33 bytes.
pub fn mock_beefy_id(id: u8) -> BeefyId {
let buf: [u8; 33] = [id; 33];
let pk = Public::from_raw(buf);
BeefyId::from(pk)
}
pub fn mock_authorities(vec: Vec<u8>) -> Vec<(u64, BeefyId)> {
vec.into_iter().map(|id| ((id as u64), mock_beefy_id(id))).collect()
}
pub fn new_test_ext(ids: Vec<u8>) -> TestExternalities {
new_test_ext_raw_authorities(mock_authorities(ids))
}
pub fn new_test_ext_raw_authorities(authorities: Vec<(u64, BeefyId)>) -> TestExternalities {
let mut t = frame_system::GenesisConfig::default().build_storage::<Test>().unwrap();
let session_keys: Vec<_> = authorities
.iter()
.enumerate()
.map(|(_, id)| (id.0 as u64, id.0 as u64, MockSessionKeys { dummy: id.1.clone() }))
.collect();
BasicExternalities::execute_with_storage(&mut t, || {
for (ref id, ..) in &session_keys {
frame_system::Pallet::<Test>::inc_providers(id);
}
});
pallet_session::GenesisConfig::<Test> { keys: session_keys }
.assimilate_storage(&mut t)
.unwrap();
t.into()
}
+142
View File
@@ -0,0 +1,142 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::vec;
use beefy_primitives::ValidatorSet;
use codec::Encode;
use sp_core::H256;
use sp_runtime::DigestItem;
use frame_support::traits::OnInitialize;
use crate::mock::*;
fn init_block(block: u64) {
System::set_block_number(block);
Session::on_initialize(block);
}
pub fn beefy_log(log: ConsensusLog<BeefyId>) -> DigestItem<H256> {
DigestItem::Consensus(BEEFY_ENGINE_ID, log.encode())
}
#[test]
fn genesis_session_initializes_authorities() {
let want = vec![mock_beefy_id(1), mock_beefy_id(2), mock_beefy_id(3), mock_beefy_id(4)];
new_test_ext(vec![1, 2, 3, 4]).execute_with(|| {
let authorities = Beefy::authorities();
assert!(authorities.len() == 2);
assert_eq!(want[0], authorities[0]);
assert_eq!(want[1], authorities[1]);
assert!(Beefy::validator_set_id() == 0);
let next_authorities = Beefy::next_authorities();
assert!(next_authorities.len() == 2);
assert_eq!(want[0], next_authorities[0]);
assert_eq!(want[1], next_authorities[1]);
});
}
#[test]
fn session_change_updates_authorities() {
new_test_ext(vec![1, 2, 3, 4]).execute_with(|| {
init_block(1);
assert!(0 == Beefy::validator_set_id());
// no change - no log
assert!(System::digest().logs.is_empty());
init_block(2);
assert!(1 == Beefy::validator_set_id());
let want = beefy_log(ConsensusLog::AuthoritiesChange(ValidatorSet {
validators: vec![mock_beefy_id(3), mock_beefy_id(4)],
id: 1,
}));
let log = System::digest().logs[0].clone();
assert_eq!(want, log);
});
}
#[test]
fn session_change_updates_next_authorities() {
let want = vec![mock_beefy_id(1), mock_beefy_id(2), mock_beefy_id(3), mock_beefy_id(4)];
new_test_ext(vec![1, 2, 3, 4]).execute_with(|| {
init_block(1);
let next_authorities = Beefy::next_authorities();
assert!(next_authorities.len() == 2);
assert_eq!(want[0], next_authorities[0]);
assert_eq!(want[1], next_authorities[1]);
init_block(2);
let next_authorities = Beefy::next_authorities();
assert!(next_authorities.len() == 2);
assert_eq!(want[2], next_authorities[0]);
assert_eq!(want[3], next_authorities[1]);
});
}
#[test]
fn validator_set_at_genesis() {
let want = vec![mock_beefy_id(1), mock_beefy_id(2)];
new_test_ext(vec![1, 2, 3, 4]).execute_with(|| {
let vs = Beefy::validator_set();
assert_eq!(vs.id, 0u64);
assert_eq!(vs.validators[0], want[0]);
assert_eq!(vs.validators[1], want[1]);
});
}
#[test]
fn validator_set_updates_work() {
let want = vec![mock_beefy_id(1), mock_beefy_id(2), mock_beefy_id(3), mock_beefy_id(4)];
new_test_ext(vec![1, 2, 3, 4]).execute_with(|| {
init_block(1);
let vs = Beefy::validator_set();
assert_eq!(vs.id, 0u64);
assert_eq!(want[0], vs.validators[0]);
assert_eq!(want[1], vs.validators[1]);
init_block(2);
let vs = Beefy::validator_set();
assert_eq!(vs.id, 1u64);
assert_eq!(want[2], vs.validators[0]);
assert_eq!(want[3], vs.validators[1]);
});
}
+33
View File
@@ -0,0 +1,33 @@
[package]
name = "beefy-primitives"
version = "4.0.0-dev"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2018"
license = "Apache-2.0"
[dependencies]
codec = { version = "2.2.0", package = "parity-scale-codec", default-features = false, features = ["derive"] }
scale-info = { version = "1.0", default-features = false, features = ["derive"] }
sp-api = { version = "4.0.0-dev", path = "../api", default-features = false }
sp-application-crypto = { version = "4.0.0-dev", path = "../application-crypto", default-features = false }
sp-core = { version = "4.0.0-dev", path = "../core", default-features = false }
sp-runtime = { version = "4.0.0-dev", path = "../runtime", default-features = false }
sp-std = { version = "4.0.0-dev", path = "../std", default-features = false }
[dev-dependencies]
hex-literal = "0.3"
sp-keystore = { version = "0.10.0-dev", path = "../keystore" }
[features]
default = ["std"]
std = [
"codec/std",
"scale-info/std",
"sp-api/std",
"sp-application-crypto/std",
"sp-core/std",
"sp-runtime/std",
"sp-std/std",
]
@@ -0,0 +1,264 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use sp_std::{cmp, prelude::*};
use crate::{crypto::Signature, ValidatorSetId};
/// A commitment signed by GRANDPA validators as part of BEEFY protocol.
///
/// The commitment contains a [payload] extracted from the finalized block at height [block_number].
/// GRANDPA validators collect signatures on commitments and a stream of such signed commitments
/// (see [SignedCommitment]) forms the BEEFY protocol.
#[derive(Clone, Debug, PartialEq, Eq, codec::Encode, codec::Decode)]
pub struct Commitment<TBlockNumber, TPayload> {
/// The payload being signed.
///
/// This should be some form of cumulative representation of the chain (think MMR root hash).
/// The payload should also contain some details that allow the light client to verify next
/// validator set. The protocol does not enforce any particular format of this data,
/// nor how often it should be present in commitments, however the light client has to be
/// provided with full validator set whenever it performs the transition (i.e. importing first
/// block with [validator_set_id] incremented).
pub payload: TPayload,
/// Finalized block number this commitment is for.
///
/// GRANDPA validators agree on a block they create a commitment for and start collecting
/// signatures. This process is called a round.
/// There might be multiple rounds in progress (depending on the block choice rule), however
/// since the payload is supposed to be cumulative, it is not required to import all
/// commitments.
/// BEEFY light client is expected to import at least one commitment per epoch,
/// but is free to import as many as it requires.
pub block_number: TBlockNumber,
/// BEEFY validator set supposed to sign this commitment.
///
/// Validator set is changing once per epoch. The Light Client must be provided by details
/// about the validator set whenever it's importing first commitment with a new
/// `validator_set_id`. Validator set data MUST be verifiable, for instance using [payload]
/// information.
pub validator_set_id: ValidatorSetId,
}
impl<TBlockNumber, TPayload> cmp::PartialOrd for Commitment<TBlockNumber, TPayload>
where
TBlockNumber: cmp::Ord,
TPayload: cmp::Eq,
{
fn partial_cmp(&self, other: &Self) -> Option<cmp::Ordering> {
Some(self.cmp(other))
}
}
impl<TBlockNumber, TPayload> cmp::Ord for Commitment<TBlockNumber, TPayload>
where
TBlockNumber: cmp::Ord,
TPayload: cmp::Eq,
{
fn cmp(&self, other: &Self) -> cmp::Ordering {
self.validator_set_id
.cmp(&other.validator_set_id)
.then_with(|| self.block_number.cmp(&other.block_number))
}
}
/// A commitment with matching GRANDPA validators' signatures.
#[derive(Clone, Debug, PartialEq, Eq, codec::Encode, codec::Decode)]
pub struct SignedCommitment<TBlockNumber, TPayload> {
/// The commitment signatures are collected for.
pub commitment: Commitment<TBlockNumber, TPayload>,
/// GRANDPA validators' signatures for the commitment.
///
/// The length of this `Vec` must match number of validators in the current set (see
/// [Commitment::validator_set_id]).
pub signatures: Vec<Option<Signature>>,
}
impl<TBlockNumber, TPayload> SignedCommitment<TBlockNumber, TPayload> {
/// Return the number of collected signatures.
pub fn no_of_signatures(&self) -> usize {
self.signatures.iter().filter(|x| x.is_some()).count()
}
}
/// A [SignedCommitment] with a version number. This variant will be appended
/// to the block justifications for the block for which the signed commitment
/// has been generated.
#[derive(Clone, Debug, PartialEq, codec::Encode, codec::Decode)]
pub enum VersionedCommitment<N, P> {
#[codec(index = 1)]
/// Current active version
V1(SignedCommitment<N, P>),
}
#[cfg(test)]
mod tests {
use sp_core::{keccak_256, Pair};
use sp_keystore::{testing::KeyStore, SyncCryptoStore, SyncCryptoStorePtr};
use super::*;
use codec::Decode;
use crate::{crypto, KEY_TYPE};
type TestCommitment = Commitment<u128, String>;
type TestSignedCommitment = SignedCommitment<u128, String>;
type TestVersionedCommitment = VersionedCommitment<u128, String>;
// The mock signatures are equivalent to the ones produced by the BEEFY keystore
fn mock_signatures() -> (crypto::Signature, crypto::Signature) {
let store: SyncCryptoStorePtr = KeyStore::new().into();
let alice = sp_core::ecdsa::Pair::from_string("//Alice", None).unwrap();
let _ =
SyncCryptoStore::insert_unknown(&*store, KEY_TYPE, "//Alice", alice.public().as_ref())
.unwrap();
let msg = keccak_256(b"This is the first message");
let sig1 = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, &alice.public(), &msg)
.unwrap()
.unwrap();
let msg = keccak_256(b"This is the second message");
let sig2 = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, &alice.public(), &msg)
.unwrap()
.unwrap();
(sig1.into(), sig2.into())
}
#[test]
fn commitment_encode_decode() {
// given
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
// when
let encoded = codec::Encode::encode(&commitment);
let decoded = TestCommitment::decode(&mut &*encoded);
// then
assert_eq!(decoded, Ok(commitment));
assert_eq!(
encoded,
hex_literal::hex!(
"3048656c6c6f20576f726c6421050000000000000000000000000000000000000000000000"
)
);
}
#[test]
fn signed_commitment_encode_decode() {
// given
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
let signed = SignedCommitment {
commitment,
signatures: vec![None, None, Some(sigs.0), Some(sigs.1)],
};
// when
let encoded = codec::Encode::encode(&signed);
let decoded = TestSignedCommitment::decode(&mut &*encoded);
// then
assert_eq!(decoded, Ok(signed));
assert_eq!(
encoded,
hex_literal::hex!(
"3048656c6c6f20576f726c64210500000000000000000000000000000000000000000000001000
0001558455ad81279df0795cc985580e4fb75d72d948d1107b2ac80a09abed4da8480c746cc321f2319a5e99a830e314d
10dd3cd68ce3dc0c33c86e99bcb7816f9ba01012d6e1f8105c337a86cdd9aaacdc496577f3db8c55ef9e6fd48f2c5c05a
2274707491635d8ba3df64f324575b7b2a34487bca2324b6a0046395a71681be3d0c2a00"
)
);
}
#[test]
fn signed_commitment_count_signatures() {
// given
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
let mut signed = SignedCommitment {
commitment,
signatures: vec![None, None, Some(sigs.0), Some(sigs.1)],
};
assert_eq!(signed.no_of_signatures(), 2);
// when
signed.signatures[2] = None;
// then
assert_eq!(signed.no_of_signatures(), 1);
}
#[test]
fn commitment_ordering() {
fn commitment(
block_number: u128,
validator_set_id: crate::ValidatorSetId,
) -> TestCommitment {
Commitment { payload: "Hello World!".into(), block_number, validator_set_id }
}
// given
let a = commitment(1, 0);
let b = commitment(2, 1);
let c = commitment(10, 0);
let d = commitment(10, 1);
// then
assert!(a < b);
assert!(a < c);
assert!(c < b);
assert!(c < d);
assert!(b < d);
}
#[test]
fn versioned_commitment_encode_decode() {
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
let signed = SignedCommitment {
commitment,
signatures: vec![None, None, Some(sigs.0), Some(sigs.1)],
};
let versioned = TestVersionedCommitment::V1(signed.clone());
let encoded = codec::Encode::encode(&versioned);
assert_eq!(1, encoded[0]);
assert_eq!(encoded[1..], codec::Encode::encode(&signed));
let decoded = TestVersionedCommitment::decode(&mut &*encoded);
assert_eq!(decoded, Ok(versioned));
}
}
+137
View File
@@ -0,0 +1,137 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#![cfg_attr(not(feature = "std"), no_std)]
#![warn(missing_docs)]
//! Primitives for BEEFY protocol.
//!
//! The crate contains shared data types used by BEEFY protocol and documentation (in a form of
//! code) for building a BEEFY light client.
//!
//! BEEFY is a gadget that runs alongside another finality gadget (for instance GRANDPA).
//! For simplicity (and the initially intended use case) the documentation says GRANDPA in places
//! where a more abstract "Finality Gadget" term could be used, but there is no reason why BEEFY
//! wouldn't run with some other finality scheme.
//! BEEFY validator set is supposed to be tracking the Finality Gadget validator set, but note that
//! it will use a different set of keys. For Polkadot use case we plan to use `secp256k1` for BEEFY,
//! while GRANDPA uses `ed25519`.
mod commitment;
pub mod mmr;
pub mod witness;
pub use commitment::{Commitment, SignedCommitment, VersionedCommitment};
use codec::{Codec, Decode, Encode};
use scale_info::TypeInfo;
use sp_core::H256;
use sp_std::prelude::*;
/// Key type for BEEFY module.
pub const KEY_TYPE: sp_application_crypto::KeyTypeId = sp_application_crypto::KeyTypeId(*b"beef");
/// BEEFY cryptographic types
///
/// This module basically introduces three crypto types:
/// - `crypto::Pair`
/// - `crypto::Public`
/// - `crypto::Signature`
///
/// Your code should use the above types as concrete types for all crypto related
/// functionality.
///
/// The current underlying crypto scheme used is ECDSA. This can be changed,
/// without affecting code restricted against the above listed crypto types.
pub mod crypto {
use sp_application_crypto::{app_crypto, ecdsa};
app_crypto!(ecdsa, crate::KEY_TYPE);
/// Identity of a BEEFY authority using ECDSA as its crypto.
pub type AuthorityId = Public;
/// Signature for a BEEFY authority using ECDSA as its crypto.
pub type AuthoritySignature = Signature;
}
/// The `ConsensusEngineId` of BEEFY.
pub const BEEFY_ENGINE_ID: sp_runtime::ConsensusEngineId = *b"BEEF";
/// Authority set id starts with zero at genesis
pub const GENESIS_AUTHORITY_SET_ID: u64 = 0;
/// A typedef for validator set id.
pub type ValidatorSetId = u64;
/// A set of BEEFY authorities, a.k.a. validators.
#[derive(Decode, Encode, Debug, PartialEq, Clone, TypeInfo)]
pub struct ValidatorSet<AuthorityId> {
/// Public keys of the validator set elements
pub validators: Vec<AuthorityId>,
/// Identifier of the validator set
pub id: ValidatorSetId,
}
impl<AuthorityId> ValidatorSet<AuthorityId> {
/// Return an empty validator set with id of 0.
pub fn empty() -> Self {
Self { validators: Default::default(), id: Default::default() }
}
}
/// The index of an authority.
pub type AuthorityIndex = u32;
/// The type used to represent an MMR root hash.
pub type MmrRootHash = H256;
/// A consensus log item for BEEFY.
#[derive(Decode, Encode, TypeInfo)]
pub enum ConsensusLog<AuthorityId: Codec> {
/// The authorities have changed.
#[codec(index = 1)]
AuthoritiesChange(ValidatorSet<AuthorityId>),
/// Disable the authority with given index.
#[codec(index = 2)]
OnDisabled(AuthorityIndex),
/// MMR root hash.
#[codec(index = 3)]
MmrRoot(MmrRootHash),
}
/// BEEFY vote message.
///
/// A vote message is a direct vote created by a BEEFY node on every voting round
/// and is gossiped to its peers.
#[derive(Debug, Decode, Encode, TypeInfo)]
pub struct VoteMessage<Hash, Number, Id, Signature> {
/// Commit to information extracted from a finalized block
pub commitment: Commitment<Number, Hash>,
/// Node authority id
pub id: Id,
/// Node signature
pub signature: Signature,
}
sp_api::decl_runtime_apis! {
/// API necessary for BEEFY voters.
pub trait BeefyApi
{
/// Return the current active BEEFY validator set
fn validator_set() -> ValidatorSet<crypto::AuthorityId>;
}
}
+132
View File
@@ -0,0 +1,132 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! BEEFY + MMR utilties.
//!
//! While BEEFY can be used completely indepentently as an additional consensus gadget,
//! it is designed around a main use case of making bridging standalone networks together.
//! For that use case it's common to use some aggregated data structure (like MMR) to be
//! used in conjunction with BEEFY, to be able to efficiently prove any past blockchain data.
//!
//! This module contains primitives used by Polkadot implementation of the BEEFY+MMR bridge,
//! but we imagine they will be useful for other chains that either want to bridge with Polkadot
//! or are completely standalone, but heavily inspired by Polkadot.
use codec::{Decode, Encode};
use scale_info::TypeInfo;
/// A standard leaf that gets added every block to the MMR constructed by Substrate's `pallet_mmr`.
#[derive(Debug, PartialEq, Eq, Clone, Encode, Decode)]
pub struct MmrLeaf<BlockNumber, Hash, MerkleRoot> {
/// Version of the leaf format.
///
/// Can be used to enable future format migrations and compatibility.
/// See [`MmrLeafVersion`] documentation for details.
pub version: MmrLeafVersion,
/// Current block parent number and hash.
pub parent_number_and_hash: (BlockNumber, Hash),
/// A merkle root of the next BEEFY authority set.
pub beefy_next_authority_set: BeefyNextAuthoritySet<MerkleRoot>,
/// A merkle root of all registered parachain heads.
pub parachain_heads: MerkleRoot,
}
/// A MMR leaf versioning scheme.
///
/// Version is a single byte that constist of two components:
/// - `major` - 3 bits
/// - `minor` - 5 bits
///
/// Any change in encoding that adds new items to the structure is considered non-breaking, hence
/// only requires an update of `minor` version. Any backward incompatible change (i.e. decoding to a
/// previous leaf format fails) should be indicated with `major` version bump.
///
/// Given that adding new struct elements in SCALE is backward compatible (i.e. old format can be
/// still decoded, the new fields will simply be ignored). We expect the major version to be bumped
/// very rarely (hopefuly never).
#[derive(Debug, Default, PartialEq, Eq, Clone, Encode, Decode)]
pub struct MmrLeafVersion(u8);
impl MmrLeafVersion {
/// Create new version object from `major` and `minor` components.
///
/// Panics if any of the component occupies more than 4 bits.
pub fn new(major: u8, minor: u8) -> Self {
if major > 0b111 || minor > 0b11111 {
panic!("Version components are too big.");
}
let version = (major << 5) + minor;
Self(version)
}
/// Split the version into `major` and `minor` sub-components.
pub fn split(&self) -> (u8, u8) {
let major = self.0 >> 5;
let minor = self.0 & 0b11111;
(major, minor)
}
}
/// Details of the next BEEFY authority set.
#[derive(Debug, Default, PartialEq, Eq, Clone, Encode, Decode, TypeInfo)]
pub struct BeefyNextAuthoritySet<MerkleRoot> {
/// Id of the next set.
///
/// Id is required to correlate BEEFY signed commitments with the validator set.
/// Light Client can easily verify that the commitment witness it is getting is
/// produced by the latest validator set.
pub id: crate::ValidatorSetId,
/// Number of validators in the set.
///
/// Some BEEFY Light Clients may use an interactive protocol to verify only subset
/// of signatures. We put set length here, so that these clients can verify the minimal
/// number of required signatures.
pub len: u32,
/// Merkle Root Hash build from BEEFY AuthorityIds.
///
/// This is used by Light Clients to confirm that the commitments are signed by the correct
/// validator set. Light Clients using interactive protocol, might verify only subset of
/// signatures, hence don't require the full list here (will receive inclusion proofs).
pub root: MerkleRoot,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn should_construct_version_correctly() {
let tests = vec![(0, 0, 0b00000000), (7, 2, 0b11100010), (7, 31, 0b11111111)];
for (major, minor, version) in tests {
let v = MmrLeafVersion::new(major, minor);
assert_eq!(v.encode(), vec![version], "Encoding does not match.");
assert_eq!(v.split(), (major, minor));
}
}
#[test]
#[should_panic]
fn should_panic_if_major_too_large() {
MmrLeafVersion::new(8, 0);
}
#[test]
#[should_panic]
fn should_panic_if_minor_too_large() {
MmrLeafVersion::new(0, 32);
}
}
+162
View File
@@ -0,0 +1,162 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Primitives for light, 2-phase interactive verification protocol.
//!
//! Instead of submitting full list of signatures, it's possible to submit first a witness
//! form of [SignedCommitment].
//! This can later be verified by the client requesting only some (out of all) signatures for
//! verification. This allows lowering the data and computation cost of verifying the
//! signed commitment.
use sp_std::prelude::*;
use crate::{
commitment::{Commitment, SignedCommitment},
crypto::Signature,
};
/// A light form of [SignedCommitment].
///
/// This is a light ("witness") form of the signed commitment. Instead of containing full list of
/// signatures, which might be heavy and expensive to verify, it only contains a bit vector of
/// validators which signed the original [SignedCommitment] and a merkle root of all signatures.
///
/// This can be used by light clients for 2-phase interactive verification (for instance for
/// Ethereum Mainnet), in a commit-reveal like scheme, where first we submit only the signed
/// commitment witness and later on, the client picks only some signatures to verify at random.
#[derive(Debug, PartialEq, Eq, codec::Encode, codec::Decode)]
pub struct SignedCommitmentWitness<TBlockNumber, TPayload, TMerkleRoot> {
/// The full content of the commitment.
pub commitment: Commitment<TBlockNumber, TPayload>,
/// The bit vector of validators who signed the commitment.
pub signed_by: Vec<bool>, // TODO [ToDr] Consider replacing with bitvec crate
/// A merkle root of signatures in the original signed commitment.
pub signatures_merkle_root: TMerkleRoot,
}
impl<TBlockNumber, TPayload, TMerkleRoot>
SignedCommitmentWitness<TBlockNumber, TPayload, TMerkleRoot>
{
/// Convert [SignedCommitment] into [SignedCommitmentWitness].
///
/// This takes a [SignedCommitment], which contains full signatures
/// and converts it into a witness form, which does not contain full signatures,
/// only a bit vector indicating which validators have signed the original [SignedCommitment]
/// and a merkle root of all signatures.
///
/// Returns the full list of signatures along with the witness.
pub fn from_signed<TMerkelize>(
signed: SignedCommitment<TBlockNumber, TPayload>,
merkelize: TMerkelize,
) -> (Self, Vec<Option<Signature>>)
where
TMerkelize: FnOnce(&[Option<Signature>]) -> TMerkleRoot,
{
let SignedCommitment { commitment, signatures } = signed;
let signed_by = signatures.iter().map(|s| s.is_some()).collect();
let signatures_merkle_root = merkelize(&signatures);
(Self { commitment, signed_by, signatures_merkle_root }, signatures)
}
}
#[cfg(test)]
mod tests {
use sp_core::{keccak_256, Pair};
use sp_keystore::{testing::KeyStore, SyncCryptoStore, SyncCryptoStorePtr};
use super::*;
use codec::Decode;
use crate::{crypto, KEY_TYPE};
type TestCommitment = Commitment<u128, String>;
type TestSignedCommitment = SignedCommitment<u128, String>;
type TestSignedCommitmentWitness =
SignedCommitmentWitness<u128, String, Vec<Option<Signature>>>;
// The mock signatures are equivalent to the ones produced by the BEEFY keystore
fn mock_signatures() -> (crypto::Signature, crypto::Signature) {
let store: SyncCryptoStorePtr = KeyStore::new().into();
let alice = sp_core::ecdsa::Pair::from_string("//Alice", None).unwrap();
let _ =
SyncCryptoStore::insert_unknown(&*store, KEY_TYPE, "//Alice", alice.public().as_ref())
.unwrap();
let msg = keccak_256(b"This is the first message");
let sig1 = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, &alice.public(), &msg)
.unwrap()
.unwrap();
let msg = keccak_256(b"This is the second message");
let sig2 = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, &alice.public(), &msg)
.unwrap()
.unwrap();
(sig1.into(), sig2.into())
}
fn signed_commitment() -> TestSignedCommitment {
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
SignedCommitment { commitment, signatures: vec![None, None, Some(sigs.0), Some(sigs.1)] }
}
#[test]
fn should_convert_signed_commitment_to_witness() {
// given
let signed = signed_commitment();
// when
let (witness, signatures) =
TestSignedCommitmentWitness::from_signed(signed, |sigs| sigs.to_vec());
// then
assert_eq!(witness.signatures_merkle_root, signatures);
}
#[test]
fn should_encode_and_decode_witness() {
// given
let signed = signed_commitment();
let (witness, _) = TestSignedCommitmentWitness::from_signed(signed, |sigs| sigs.to_vec());
// when
let encoded = codec::Encode::encode(&witness);
let decoded = TestSignedCommitmentWitness::decode(&mut &*encoded);
// then
assert_eq!(decoded, Ok(witness));
assert_eq!(
encoded,
hex_literal::hex!(
"3048656c6c6f20576f726c64210500000000000000000000000000000000000000000000001000
00010110000001558455ad81279df0795cc985580e4fb75d72d948d1107b2ac80a09abed4da8480c746cc321f2319a5e9
9a830e314d10dd3cd68ce3dc0c33c86e99bcb7816f9ba01012d6e1f8105c337a86cdd9aaacdc496577f3db8c55ef9e6fd
48f2c5c05a2274707491635d8ba3df64f324575b7b2a34487bca2324b6a0046395a71681be3d0c2a00"
)
);
}
}