Integrate BEEFY (#9833)

* Initial project setup and skeleton (#4)

* initial project setup for beefy gadget client

* update editorconfig

* update gitignore

* add initial skeleton for beefy gadget worker

* add skeleton for gossip processing

* add app crypto

* move around some code

* add basic flow for voting

* add logic for picking blocks to sign

* add rustfmt config

* add example node with beefy gadget

* use u32::next_power_of_two

* make maximum periodicity configurable

* add copyright header

* rename max_periodicity to min_interval

* CI stuff (#5)

* CI stuff.

* Fix workspace.

* cargo fmt --all

* Add license for beefy-gadget

* One toolchain to rule them all.

* Clippy.

* Fix clippy.

* Clippy in the runtime.

* Fix clippy grumbles.

* cargo fmt --all

* Primitives & Light Client examples (#8)

* Primitives.

* Docs.

* Document primitives.

* Simple tests.

* Light client examples.

* Fix stuff.

* cargo fmt --all

* Add a bunch of tests for imports.

* Add more examples.

* cargo fmt --all

* Fix clippy.

* cargo fmt --all

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Add GRANDPA / FG clarifications.

* Fix min number of signatures.

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update to substrate master (#22)

* update to substrate master

* update dependencies

* fix clippy issues

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Add beefy pallet (#25)

* move beefy application crypto to primitives

* make primitives compile under no_std

* add beefy pallet that maintains authority set

* add beefy pallet to node example runtime

* tabify node-example cargo.toml files

* use double quotes in Cargo.toml files

* add missing hex-literal dependency

* add runtime api to fetch BEEFY authorities

* fix clippy warnings

* rename beefy-pallet to pallet-beefy

* sort dependencies in node-example/runtime/Cargo.toml

* Signed commitments rpc pubsub (#26)

* move beefy application crypto to primitives

* make primitives compile under no_std

* add beefy pallet that maintains authority set

* add beefy pallet to node example runtime

* tabify node-example cargo.toml files

* use double quotes in Cargo.toml files

* add missing hex-literal dependency

* add runtime api to fetch BEEFY authorities

* fix clippy warnings

* gadget: use commitment and signedcommitment

* gadget: send notifications for signed commitments

* gadget: add rpc pubsub for signed commitments

* node-example: enable beefy rpc

* gadget: fix clippy warnings

* rename beefy-pallet to pallet-beefy

* sort dependencies in node-example/runtime/Cargo.toml

* gadget: add documentation on SignedCommitment rpc wrapper type

* gadget: add todos about dummy beefy commitments

* gadget: remove redundant closure

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Integrate MMR and deposit root into the digest. (#24)

* Add basic MMR.

* Deposit digest item.

* cargo fmt --all

* Merge with primitives.

* cargo fmt --all

* Fix extra spaces.

* cargo fmt --all

* Switch branch.

* remove stray whitespace

* update to latest td-mmr commit

* fix clippy error

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* use new mmr root as commitment payload (#27)

* use new mmr root as commitment payload

* fix mmr root codec index

* warn on MMR root digest not found

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* add type alias for MMR root hash

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Bump serde_json from 1.0.59 to 1.0.60 (#28)

* Update to latest substrate. (#32)

* Update to latest substrate.

* Fix tests.

* cargo fmt --all

* Switch to master.

* Bump serde from 1.0.117 to 1.0.118 (#29)

* Bump serde from 1.0.117 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.117 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.117...v1.0.118)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump arc-swap.

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Remove transition flag (#35)

* Get rid of is_set_transition_flag

* Fix tests.

* cargo fmt --all

* Bump futures from 0.3.9 to 0.3.12 (#50)

* Bump log from 0.4.11 to 0.4.13 (#52)

* Bump Substrate and Deps (#57)

* Update README (#58)

* Update README

* Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* address review comments

* missed a typo

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Add validator set to the pallet. (#65)

* Bump Substrate and Deps (#71)

* Bump Substrate and Deps

* pin serde and syn

* bump Substrate again for '__Nonexhaustive' fix

* add cargo deny ignore

* Beefy pallet test (#74)

* setup mock

* test session change

* silence beefy

* clippy still

* no change - no log

* clippy again

* Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* code review changes, added additional test

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Beefy node cleanup (#75)

* bump serde

* bump substrate, scale-codec 2.0.0

* we need a proper beefy node

* rename primitives as well

* Sort members.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Migrate beefy-pallet to FRAMEv2 (#76)

* migrate beefy-pallet to FRAMEv2

* Code review

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Run BEEFY worker as non-validator (#77)

* run BEEFY worker as non-validator

* don't check for roloe.is_authority

* change enum type name

* Bump Substrate and Deps (#79)

* Add BEEFY gadget as extra peer set (#80)

* Add BEEFY gadget as extra peer set

* use BEEFY protocol

* Add ValidatorSetId to BEEFY digest (#85)

* add ValidatorSetId to BEEFY digest

* apply review changes

* Bump Substrate and Deps (#91)

* Bump Substrate and Deps

* Bump Substrate again in order to include a hot-fix

* redo again

* use CryptoStore issue

* cargo fmt

* Bump serde_json from 1.0.63 to 1.0.64 (#93)

* Track BEEFY validator set (#94)

* Track BEEFY validator set

* Add validator_set_id to BeefyWorker

* Make validattor_set_id optional

* Ad 92 (#97)

* sign_commitment()

* Error handling todo

* Add error type (#99)

* Add error type

* Address review

* Extract worker and round logic (#104)

* Bump serde from 1.0.123 to 1.0.124 (#106)

* Rework BeefyAPI (#110)

* Initialize BeefyWorker with current validator set (#111)

* Update toolchain (#115)

* Use nightly toolchain

* dongradde to latest clippy stable

* GH workflow trail and error

* next try

* use stable for clippy

* update wasm builder

* yet another try

* fun with CI

* no env var

* and one more

* allow from_over_into bco contruct_runtime

* back to start

* well ...

* full circle

* old version was still used

* Bump Substrate and Deps (#117)

* Bump Substrate and Deps

* cargo fmt should enforce uniform imports

* merge some imports

* Delayed BEEFY worker initialization (#121)

* lifecycle state

* add Client convenience trait

* rework trait identifiers

* WIP

* rework BeefyWorker::new() signature

* Delayed BEEFY gadget initialization

* address review

* Bump substrate. (#123)

* Bump substrate.

* Fix tests.

* Lower log-level for a missing validator set (#124)

* lower log-level for a missing validator set

* move best_finalized_block initialization

* Setup Prometheus metrics (#125)

* setup Prometheus metrics

* expose validator set id

* cargo fmt

* Update beefy-gadget/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* add vote messages gossiped metric

* track authorities change, before checking for MMR root digest

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Make Client convenience trait public (#126)

* Bump serde from 1.0.124 to 1.0.125 (#131)

* Reset rounds on new validator set. (#133)

* Re-set rounds on new validator set.

* Fix docs.

* Bump Substrate and Deps (#134)

* beefy: authority set changes fixes (#139)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type (#140)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* gadget: reduce gossip spam (#141)

* node: fix grandpa peers set config

* gadget: update best finalized number only when finalized with beefy

* gadget: process authorities changes regardless of vote status

* gadget: remove superfluous signature type

* gadget: only gossip last 5 rounds

* gadget: note round to gossip validator before gossiping message

* gadget: fix clippy warnings

* gadget: update docs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* gadget: verify SignedCommitment message signature (#142)

* gadget: verify SignedCommitment message signature

* gadget: log messages with bad sigs

* gadget: move todo comment

* Bump futures from 0.3.13 to 0.3.14 (#145)

* Milestone 1 (#144)

* use best_finalized, prevent race

* make best_finalized_block an Option, should_vote_on bails on None

* Bump futures from 0.3.13 to 0.3.14

* Revert futures bump

* Revert "Revert futures bump"

This reverts commit a1b5e7e9bac526f2897ebfdfee7f02dd29a13ac5.

* Revert "Bump futures from 0.3.13 to 0.3.14"

This reverts commit a4e508b118ad2c4b52909d24143c284073961458.

* debug msg if the bail voting

* validator_set()

* local_id()

* get rid of worker state

* Apply review suggestions

* fix should_vote_on()

* Extract BeefyGossipValidator (#147)

* Extract BeefyGossipValidator

* Apply review suggestions

* Add block_delta parameter to start_beefy_gadget (#151)

* Add block_delta parameter

* rename to min_block_delta

* Add additional metrics (#152)

* Add additional metrics

* add skipped session metric

* add some comment for temp metric

* don't log under info for every concluded round (#156)

* don't log error on missing validator keys (#157)

* don't log error on missing validator keys

* remove unused import

* Fix validator set change handling (#158)

* reduce some logs from debug to trace

* fix validator set changes handling

* rename validator module to gossip

* run rustfmt

* Fix should_vote_on() (#160)

* Fix should_vote_on()

* by the textbook

* fix the algorithm

* Apply review suggestions

* don't use NumberFor in vote_target

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* Make KeyStore optional (#173)

* Use builder pattern for NonDefaultSetConfig (#178)

Co-authored-by: adoerr <0xad@gmx.net>

* Append SignedCommitment to block justifications (#177)

* Append SignedCommitment

* add BeefyParams

* add WorkerParams

* use warn

* versioned variant for SignedCommitment

* Bump serde from 1.0.125 to 1.0.126 (#184)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.125 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.125...v1.0.126)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump strum from 0.20.0 to 0.21.0 (#195)

* Bump strum from 0.20.0 to 0.21.0

Bumps [strum](https://github.com/Peternator7/strum) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use dervie feature for strum; clippy and deny housekeeping

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Make concluded round an info log (#200)

* Remove external crypto trait bounds (#207)

* BeefyKeystore newtype

* WIP

* remove mod ecdsa

* WIP

* fix tests

* some polishing

* Rename AuthorityId to BeefyId to avoid type conflict in UI (#211)

* Add trace points; Reduce MAX_LIVE_GOSSIP_ROUNDS (#210)

* Add trace points; Reduce MAX_LIVE_GOSSIP_ROUNDS

* log local authority id

* Additional initial authority id's (#217)

* Scratch concluded rounds

* adjust testnet doc

* fix authority key typo

* We don't want no scratches

* address review comments

* Fix note_round() (#219)

* rename BeefyGossipValidator

* Fix note_round()

* use const for assert

* put message trace points back in

* test case note_same_round_twice()

* address review comments

* remove redundant check

* Use LocalKeystore for tests (#224)

* private_keys()

* Use LocalKeystore for tests

* Use keystore helper

* Address review

* some reformatting

* Cache known votes in gossip (#227)

* Implement known messages cache.

* Add tests.

* Appease clippy.

* More clippy

Co-authored-by: adoerr <0xad@gmx.net>

* Some key store sanity checks (#232)

* verify vote message

* verify_validator_set()

* rework logging

* some rework

* Tone down warnings.

* Add signature verification.

* Tone down more.

* Fix clippy

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Use Binary Merkle Tree instead of a trie (#225)

* Binary tree merkle root.

* Add proofs and verification.

* Clean up debug.

* Use BEEFY addresses instead of pubkeys.

* Use new merkle tree.

* Optimize allocations.

* Add test for larger trees.

* Add tests for larger cases.

* Appease clippy

* Appease clippy2.

* Fix proof generation & verification.

* Add more test data.

* Fix CLI.

* Update README

* Bump version.

* Update docs.

* Rename beefy-merkle-root to beefy-merkle-tree

Co-authored-by: adoerr <0xad@gmx.net>

* Bump Substrate and Deps (#235)

* BEEFY+MMR pallet (#236)

* Add MMR leaf format to primitives.

* Fix tests

* Initial work on the BEEFY-MMR pallet.

* Add tests to MMR pallet.

* Use eth addresses.

* Use binary merkle tree.

* Bump libsecp256k1

* Fix compilation.

* Bump deps.

* Appease cargo deny.

* Re-format.

* Module-level docs.

* no-std fix.

* update README

Co-authored-by: adoerr <0xad@gmx.net>

* Fix noting rounds for non-authorities (#238)

* Bump env_logger from 0.8.4 to 0.9.0 (#242)

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.4 to 0.9.0.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.4...v0.9.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* gadget: add global timeout for rebroadcasting messages (#243)

* gadget: add global timeout for rebroadcasting messages

* update rustfmt.toml

* make message_allowed() a debug trace

Co-authored-by: adoerr <0xad@gmx.net>

* Bump Substrate and Deps (#245)

* Bump Substrate and Deps

* Bump Substrate again

* Bump futures from 0.3.15 to 0.3.16 (#247)

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump libsecp256k1 from 0.5.0 to 0.6.0 (#249)

* Bump libsecp256k1 from 0.5.0 to 0.6.0

Bumps [libsecp256k1](https://github.com/paritytech/libsecp256k1) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/paritytech/libsecp256k1/releases)
- [Changelog](https://github.com/paritytech/libsecp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/libsecp256k1/commits)

---
updated-dependencies:
- dependency-name: libsecp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* use correct crate name

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Derive `scale_info::TypeInfo` for types used in polkadot (#218)

* Add scale-info TypeInfo derives

* Update scale-info

* Add crates.io patches

* Use substrate aj-metadata-vnext branch

* Revert master branch substrate deps

* Add scale-info to beefy-pallet

* scale-info v0.9.0

* Remove github dependencies and patches

* More TypeInfo derives

* Update scale-info to 0.10.0

* Add missing scale-info dependency

* Add missing TypeInfo derive

* Hide TypeInfo under a feature.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* Bump serde from 1.0.126 to 1.0.127 (#260)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Substrate and Deps (#262)

* Update jsonrpc (#265)

* Update jsonrpc

* Update Substrate

* bump Substrate and Deps (#268)

* Bump serde from 1.0.127 to 1.0.128 (#272)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix spelling (#271)

* Bump serde from 1.0.128 to 1.0.130 (#276)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.128 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.130)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump scale-info from 0.10.0 to 0.12.0 (#275)

Bumps [scale-info](https://github.com/paritytech/scale-info) from 0.10.0 to 0.12.0.
- [Release notes](https://github.com/paritytech/scale-info/releases)
- [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/scale-info/commits)

---
updated-dependencies:
- dependency-name: scale-info
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Update to scale-info 1.0 (#278)

* bump substrate (#282)

* bump Substrate and Deps

* cargo fmt

Co-authored-by: Wenfeng Wang <kalot.wang@gmail.com>

* Update worker.rs (#287)

* Bump anyhow from 1.0.43 to 1.0.44 (#290)

* Bump anyhow from 1.0.43 to 1.0.44

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.43 to 1.0.44.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.43...1.0.44)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* derive Default

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* Remove optional `scale-info` feature (#292)

* Make scale-info dependency non-optional

* Remove feature gated TypeInfo derives

* Import TypeInfo

* Update substrate

* Fix up runtime

* prune .git suffix (#294)

* remove unused deps (#295)

* remove unused deps

* update lock file

* Bump libsecp256k1 from 0.6.0 to 0.7.0 (#296)

* Bump libsecp256k1 from 0.6.0 to 0.7.0

Bumps [libsecp256k1](https://github.com/paritytech/libsecp256k1) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/paritytech/libsecp256k1/releases)
- [Changelog](https://github.com/paritytech/libsecp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/libsecp256k1/commits)

---
updated-dependencies:
- dependency-name: libsecp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update sec advisories

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>

* clean compile

* use path dependencies

* beefy-gadget license header

* pallet-beefy license header

* pallet-beefy-mmr license header

* beefy-primitves license header

* carg fmt

* more formatting

* shorten line

* downgrade parity-scale-codec to 2.2.0

* use path dependency for Prometheus endpoint

* remove clippy annotations

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: drewstone <drewstone329@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Wenfeng Wang <kalot.wang@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
This commit is contained in:
Andreas Doerr
2021-09-23 21:02:30 +02:00
committed by GitHub
parent 6845666c5c
commit 283c8daa81
31 changed files with 4992 additions and 12 deletions
@@ -0,0 +1,264 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use sp_std::{cmp, prelude::*};
use crate::{crypto::Signature, ValidatorSetId};
/// A commitment signed by GRANDPA validators as part of BEEFY protocol.
///
/// The commitment contains a [payload] extracted from the finalized block at height [block_number].
/// GRANDPA validators collect signatures on commitments and a stream of such signed commitments
/// (see [SignedCommitment]) forms the BEEFY protocol.
#[derive(Clone, Debug, PartialEq, Eq, codec::Encode, codec::Decode)]
pub struct Commitment<TBlockNumber, TPayload> {
/// The payload being signed.
///
/// This should be some form of cumulative representation of the chain (think MMR root hash).
/// The payload should also contain some details that allow the light client to verify next
/// validator set. The protocol does not enforce any particular format of this data,
/// nor how often it should be present in commitments, however the light client has to be
/// provided with full validator set whenever it performs the transition (i.e. importing first
/// block with [validator_set_id] incremented).
pub payload: TPayload,
/// Finalized block number this commitment is for.
///
/// GRANDPA validators agree on a block they create a commitment for and start collecting
/// signatures. This process is called a round.
/// There might be multiple rounds in progress (depending on the block choice rule), however
/// since the payload is supposed to be cumulative, it is not required to import all
/// commitments.
/// BEEFY light client is expected to import at least one commitment per epoch,
/// but is free to import as many as it requires.
pub block_number: TBlockNumber,
/// BEEFY validator set supposed to sign this commitment.
///
/// Validator set is changing once per epoch. The Light Client must be provided by details
/// about the validator set whenever it's importing first commitment with a new
/// `validator_set_id`. Validator set data MUST be verifiable, for instance using [payload]
/// information.
pub validator_set_id: ValidatorSetId,
}
impl<TBlockNumber, TPayload> cmp::PartialOrd for Commitment<TBlockNumber, TPayload>
where
TBlockNumber: cmp::Ord,
TPayload: cmp::Eq,
{
fn partial_cmp(&self, other: &Self) -> Option<cmp::Ordering> {
Some(self.cmp(other))
}
}
impl<TBlockNumber, TPayload> cmp::Ord for Commitment<TBlockNumber, TPayload>
where
TBlockNumber: cmp::Ord,
TPayload: cmp::Eq,
{
fn cmp(&self, other: &Self) -> cmp::Ordering {
self.validator_set_id
.cmp(&other.validator_set_id)
.then_with(|| self.block_number.cmp(&other.block_number))
}
}
/// A commitment with matching GRANDPA validators' signatures.
#[derive(Clone, Debug, PartialEq, Eq, codec::Encode, codec::Decode)]
pub struct SignedCommitment<TBlockNumber, TPayload> {
/// The commitment signatures are collected for.
pub commitment: Commitment<TBlockNumber, TPayload>,
/// GRANDPA validators' signatures for the commitment.
///
/// The length of this `Vec` must match number of validators in the current set (see
/// [Commitment::validator_set_id]).
pub signatures: Vec<Option<Signature>>,
}
impl<TBlockNumber, TPayload> SignedCommitment<TBlockNumber, TPayload> {
/// Return the number of collected signatures.
pub fn no_of_signatures(&self) -> usize {
self.signatures.iter().filter(|x| x.is_some()).count()
}
}
/// A [SignedCommitment] with a version number. This variant will be appended
/// to the block justifications for the block for which the signed commitment
/// has been generated.
#[derive(Clone, Debug, PartialEq, codec::Encode, codec::Decode)]
pub enum VersionedCommitment<N, P> {
#[codec(index = 1)]
/// Current active version
V1(SignedCommitment<N, P>),
}
#[cfg(test)]
mod tests {
use sp_core::{keccak_256, Pair};
use sp_keystore::{testing::KeyStore, SyncCryptoStore, SyncCryptoStorePtr};
use super::*;
use codec::Decode;
use crate::{crypto, KEY_TYPE};
type TestCommitment = Commitment<u128, String>;
type TestSignedCommitment = SignedCommitment<u128, String>;
type TestVersionedCommitment = VersionedCommitment<u128, String>;
// The mock signatures are equivalent to the ones produced by the BEEFY keystore
fn mock_signatures() -> (crypto::Signature, crypto::Signature) {
let store: SyncCryptoStorePtr = KeyStore::new().into();
let alice = sp_core::ecdsa::Pair::from_string("//Alice", None).unwrap();
let _ =
SyncCryptoStore::insert_unknown(&*store, KEY_TYPE, "//Alice", alice.public().as_ref())
.unwrap();
let msg = keccak_256(b"This is the first message");
let sig1 = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, &alice.public(), &msg)
.unwrap()
.unwrap();
let msg = keccak_256(b"This is the second message");
let sig2 = SyncCryptoStore::ecdsa_sign_prehashed(&*store, KEY_TYPE, &alice.public(), &msg)
.unwrap()
.unwrap();
(sig1.into(), sig2.into())
}
#[test]
fn commitment_encode_decode() {
// given
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
// when
let encoded = codec::Encode::encode(&commitment);
let decoded = TestCommitment::decode(&mut &*encoded);
// then
assert_eq!(decoded, Ok(commitment));
assert_eq!(
encoded,
hex_literal::hex!(
"3048656c6c6f20576f726c6421050000000000000000000000000000000000000000000000"
)
);
}
#[test]
fn signed_commitment_encode_decode() {
// given
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
let signed = SignedCommitment {
commitment,
signatures: vec![None, None, Some(sigs.0), Some(sigs.1)],
};
// when
let encoded = codec::Encode::encode(&signed);
let decoded = TestSignedCommitment::decode(&mut &*encoded);
// then
assert_eq!(decoded, Ok(signed));
assert_eq!(
encoded,
hex_literal::hex!(
"3048656c6c6f20576f726c64210500000000000000000000000000000000000000000000001000
0001558455ad81279df0795cc985580e4fb75d72d948d1107b2ac80a09abed4da8480c746cc321f2319a5e99a830e314d
10dd3cd68ce3dc0c33c86e99bcb7816f9ba01012d6e1f8105c337a86cdd9aaacdc496577f3db8c55ef9e6fd48f2c5c05a
2274707491635d8ba3df64f324575b7b2a34487bca2324b6a0046395a71681be3d0c2a00"
)
);
}
#[test]
fn signed_commitment_count_signatures() {
// given
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
let mut signed = SignedCommitment {
commitment,
signatures: vec![None, None, Some(sigs.0), Some(sigs.1)],
};
assert_eq!(signed.no_of_signatures(), 2);
// when
signed.signatures[2] = None;
// then
assert_eq!(signed.no_of_signatures(), 1);
}
#[test]
fn commitment_ordering() {
fn commitment(
block_number: u128,
validator_set_id: crate::ValidatorSetId,
) -> TestCommitment {
Commitment { payload: "Hello World!".into(), block_number, validator_set_id }
}
// given
let a = commitment(1, 0);
let b = commitment(2, 1);
let c = commitment(10, 0);
let d = commitment(10, 1);
// then
assert!(a < b);
assert!(a < c);
assert!(c < b);
assert!(c < d);
assert!(b < d);
}
#[test]
fn versioned_commitment_encode_decode() {
let commitment: TestCommitment =
Commitment { payload: "Hello World!".into(), block_number: 5, validator_set_id: 0 };
let sigs = mock_signatures();
let signed = SignedCommitment {
commitment,
signatures: vec![None, None, Some(sigs.0), Some(sigs.1)],
};
let versioned = TestVersionedCommitment::V1(signed.clone());
let encoded = codec::Encode::encode(&versioned);
assert_eq!(1, encoded[0]);
assert_eq!(encoded[1..], codec::Encode::encode(&signed));
let decoded = TestVersionedCommitment::decode(&mut &*encoded);
assert_eq!(decoded, Ok(versioned));
}
}