When BEEFY voter is initialized from scratch (no aux db persistent data present),
it needs to find BEEFY genesis block and all subsequent Mandatory blocks and
sync justifications for them.
The initialization code was getting active validator sets for these older blocks
from state, but in cases such as 'fast sync', state is unavailable.
This commit adds a fallback initialization mechanism when state is unavailable:
parse header Digests looking for validator set change log deposits.
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* WIP
* Add missing checkout
* Add debuggin
* Fix VAR name
* Bug fix
* Rework jobs
* Revert "Rework jobs"
This reverts commit 2bfa79fd3ae633c17403b838f9a5025f0f7fc3f3.
* Add cache
* Add temp default for testing
* Add missing checkout
* Fix patch
* Comment out the GPG check for now
* Rename polkadot_injected_release into a more appropriate polkadot_injected_debian
* Refactoring / renaming
* Introduce a generic image for binary injection
* Flag files to be deleted and changes to be done
* WIP
* Fix multi binaries images
* Add test build scripts
* Remove old file, add polkadot build-injected script
* Fix doc
* Fix tagging
* Add build of the injected container
* Fix for docker
* Remove the need for TTY
* Handling container publishing
* Fix owner and registry
* Fix vars
* Fix repo
* Fix var naming
* Fix case when there is no tag
* Fix case with no tag
* Handle error
* Fix spacings
* Fix tags
* Remove unnecessary grep that may fail
* Add final check
* Clean up and introduce GPG check
* Add doc
* Add doc
* Update doc/docker.md
Co-authored-by: Mira Ressel <mira@parity.io>
* type
Co-authored-by: Mira Ressel <mira@parity.io>
* Fix used VAR
* Improve doc
* ci: Update .build-push-image jobs to use the new build-injected.sh
* ci: fix path to build-injected.sh script
* Rename the release artifacts folder to prevent confusion due to a similar folder in the gitlab CI
* ci: check out polkadot repo in .build-push-image
This seems far cleaner than copying the entire scripts/ folder into our
job artifacts.
* feat(build-injected.sh): make PROJECT_ROOT configurable
This lets us avoid a dependency on git in our CI image.
* ci: build injected images with buildah
* ci: pass full image names to zombienet
* Add missing ignore
---------
Co-authored-by: Mira Ressel <mira@parity.io>
* Allow functions to work over both parachains and relay chains
* additional references
* import
* backup
* refactoring para and relay traits
* use runtime crates to build types
* decouple ProcessMessage
* decouple ProcessMessage 2
* dmp and xcmp handlers decouple
* backup
* refactor done
* common int values working
* added global ext with mutex
* works for two mutex
* single mutex and remove condvar
* global test ext done
* failing moving test_ext because relay block num
* relay_block_number issue fixed
* backup
* Test working with assertions
* assertions get Test as arg
* DispatchArgs as generic
* clean up
* backup
* teleports for asset-hub-kusama done
* improve assert_expected_events macro
* rename Test generics
* check assertions for tuples
* test assertions redone
* reserve_transfer_assets done
* send transact done
* hrmp test for paras
* hrmp channels test done
* hrmp channels test done 2
* before modifying test dispatch
* reserve tests done & Test dispatch fixed
* reserve transfer local asset
* force_create_and_mint_asset
* force create and mint done
* tests done
* fix imports in common
* common events refactored
* add option to events attributes
* asset-hub-polkadot tests done
* asset-hub-westend half done
* relay chain events move to common
* remove failing send tests for asset-hub-westend
* added events to bridge-hub-rococo
* added events to collectives-polkadot
* cargo clean up
* fix asset-hub-westend tests
* ".git/.scripts/commands/fmt/fmt.sh"
* fix clippy
* ".git/.scripts/commands/fmt/fmt.sh"
* Removed unnecessary deps
* Extracted some commonality for Kusama/Polkadot (which will be reused also for BridgeHubs) (#2971)
* Extracted some commonality for Kusama/Polkadot (which will be reused also for BridgeHubs)
* AssetHubRococo should better use AssetHubKusama runtime
* add fund_account
---------
Co-authored-by: NachoPal <ignacio.palacios.santos@gmail.com>
* address comments
* rename event assertion helpers
* clean comments
* address comments 2
* ".git/.scripts/commands/fmt/fmt.sh"
---------
Co-authored-by: Giles Cope <gilescope@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
* Implements dynamic nominations per nominator
* Adds SnapshotBounds and ElectionSizeTracker
* Changes the ElectionDataProvider interface to receive ElectionBounds as input
* Implements get_npos_voters with ElectionBounds
* Implements get_npos_targets with ElectionBounds
* Adds comments
* tests
* Truncates nomninations that exceed nominations quota; Old tests passing
* Uses DataProviderBounds and ElectionBounds (to continue)
* Finishes conversions - tests passing
* Refactor staking in babe mocks
* Replaces MaxElectableTargets and MaxElectingVoters with ElectionBounds; Adds more tests
* Fixes nits; node compiling
* bechmarks
* removes nomination_quota extrinsic to request the nomination quota
* Lazy quota check, ie. at nominate time only
* remove non-working test (for now)
* tests lazy nominations quota when quota is lower than current number of nominated targets
* Adds runtime API and custom RPC call for clients to query the nominations quota for a given balance
* removes old rpc
* Cosmetic touches
* All mocks working
* Fixes benchmarking mocks
* nits
* more tests
* renames trait methods
* nit
* ".git/.scripts/commands/fmt/fmt.sh"
* Fix V2 PoV benchmarking (#13485)
* Bump default 'additional_trie_layers' to two
The default here only works for extremely small runtimes, which have
no more than 16 storage prefices. This is changed to a "sane" default
of 2, which is save for runtimes with up to 4096 storage prefices (eg StorageValue).
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update tests and test weights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix PoV weights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_balances
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_message_queue
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_glutton
* Fix sanity check
>0 would also do as a check, but let's try this.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
* Move BEEFY code to consensus (#13484)
* Move beefy primitives to consensus dir
* Move beefy gadget to client consensus folder
* Rename beefy crates
* chore: move genesis block builder to chain-spec crate. (#13427)
* chore: move genesis block builder to block builder crate.
* add missing file
* chore: move genesis block builder to sc-chain-spec
* Update client/chain-spec/src/genesis.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update test-utils/runtime/src/genesismap.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update test-utils/runtime/client/src/lib.rs
* fix warnings
* fix warnings
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
* Speed up storage iteration from within the runtime (#13479)
* Speed up storage iteration from within the runtime
* Move the cached iterator into an `Option`
* Use `RefCell` in no_std
* Simplify the code slightly
* Use `Option::replace`
* Update doc comment for `next_storage_key_slow`
* Make unbounded channels size warning exact (part 1) (#13490)
* Replace `futures-channel` with `async-channel` in `out_events`
* Apply suggestions from code review
Co-authored-by: Koute <koute@users.noreply.github.com>
* Also print the backtrace of `send()` call
* Switch from `backtrace` crate to `std::backtrace`
* Remove outdated `backtrace` dependency
* Remove `backtrace` from `Cargo.lock`
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
* Removal of Prometheus alerting rules deployment in cloud-infra (#13499)
* sp-consensus: remove unused error variants (#13495)
* Expose `ChargedAmount` (#13488)
* Expose `ChargedAmount`
* Fix imports
* sc-consensus-beefy: fix metrics: use correct names (#13494)
Signed-off-by: acatangiu <adrian@parity.io>
* clippy fix
* removes NominationsQuotaExceeded event
* Update frame/staking/src/lib.rs
Co-authored-by: Ross Bulat <ross@parity.io>
* adds back the npos_max_iter
* remove duplicate imports added after merge
* fmt
* Adds comment in public struct; Refactors CountBound and SizeCount to struct
* addresses various pr comments
* PR comment reviews
* Fixes on-chain election bounds and related code
* EPM checks the size of the voter list returned by the data provider
* cosmetic changes
* updates e2e tests mock
* Adds more tests for size tracker and refactors code
* Adds back only_iterates_max_2_times_max_allowed_len test
* Refactor
* removes unecessary dependency
* empty commit -- restart all stuck CI jobs
* restarts ci jobs
* Renames ElectionBounds -> Bounds in benchmarking mocks et al
* updates mocks
* Update frame/election-provider-support/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/staking/src/pallet/impls.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/election-provider-support/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Update frame/staking/src/tests.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* more checks in api_nominations_quota in tests
* Improves docs
* fixes e2e tests
* Uses size_hint rather than mem::size_of in size tracker; Refactor size tracker to own module
* nits from reviews
* Refactors bounds to own module; improves docs
* More tests and docs
* fixes docs
* Fixes benchmarks
* Fixes rust docs
* fixes bags-list remote-ext-tests
* Simplify bound checks in create_snapshot_external
* Adds target size check in get_npos_targets
* ".git/.scripts/commands/fmt/fmt.sh"
* restart ci
* rust doc fixes and cosmetic nits
* rollback upgrade on parity-scale-codec version (unecessary)
* reset cargo lock, no need to update it
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: acatangiu <adrian@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: yjh <yjh465402634@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Anthony Lazam <lazam@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Piotr Mikołajczyk <piomiko41@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Introduce bandersnatch vrf
* Some documentation
* Fix tests
* Fix docs refs
* Some more docs
* Comments about key derivation
* Make clippy happy
* Fix ring context enc/dec test
* Fix docs
* Switch to upstream ring-vrf
* Use sub-domains to construct VrfInput
* Bandersnatch VRF experimental feature
* Restore upstream dep
* Fix feature flags
* Apply typo fix
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Bump bandersnatch-vrfs
* Weiestrass form has been selected
* Rename bandersnatch testing app crypto id
* Support for seed recovery
* Clarified domain size <-> key size relationship
* cargo fmt
* Trigger CI
* Some required tweaks to crypto types
* Remove leftovers from Cargo.toml
* Remove some TODO notes
* Simplification of structs construction
* Trigger CI
* Apply review suggestion
Co-authored-by: Koute <koute@users.noreply.github.com>
* Docs typo
* Fix keystore tests
* Consistence
* Add ref to git rependency
* Static check of MAX_VRF_IOS value
* Clarify behavior for out of ring keys signatures
* Add test for ring-vrf to the keystore
* Fix docs
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
* chainHead/api: Make storage/body/call pure RPC methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Add mpsc channel between RPC methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscriptions: Extract mpsc::Sender via BlockGuard
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscriptions: Generate and provide the method operation ID
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Generate `chainHead_body` response
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Generate `chainHead_call` response
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Generate `chainHead_storage` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Propagate responses of methods to chainHead_follow
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust `chainHead_body` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust `chainHead_call` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust `chainHead_call` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Ensure unique operation IDs across methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/events: Remove old method events
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Return `InvalidBlock` error if pinning fails
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Wrap subscription IDs
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Ensure separate operation IDs across subscriptions
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>
* allow spinning up dev node in background without binary
* improve comments
* restore rust-toolchain
* remove rust-toolchain
* tweak start_node_without_binary api
* Update test-utils/cli/src/lib.rs
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
* address commends
* use &str
* update example
* update comment
* update docs
* Revert "update docs"
This reverts commit e18677f081247dd9672f64ea34d2c651daf401c2.
* use node-cli instead of node-template
* fix feature
* fix feature
* fix features
---------
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
* Provide basic breakpoints
* Rename to Observer
* Rename feature. Single trait. Borrow-checker
* : frame_system::Config
* Confused type name
* Minor bugs
* pub trait
* No unnecessary cloning
* Make node compile with all features
* Move everything debug-related to a single module
* Add docs and implementation for ()
* fmt
* Make it feature-gated or for tests
* Prepare testing kit
* Testcase
* Fmt
* Use feature in dev-deps
* ?
* feature propagation
* AAAA
* lol, that doesn't make much sense to me
* Turn on
* clippy
* Remove self dep
* fmt, feature-gating test
* Noop to trigger CI
* idk
* add feature to pipeline
* Corrupt test to see if it is actually being run
* Revert change
* Doc for conf type
* Review
* Imports
* ...
* Remove debug for kitchen-sink
* Move test
* Fix imports
* I must have already tried this one...
* Use relative docify paths
Bumping the docify version and using relative paths is needed for
the monorepo to find the test files in the new workspace.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* update docify to fix paths issue
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Sam Johnson <sam@durosoft.com>
* aura: add feature-flagged explicit slot duration type
* fmt
* add some comments
* have node-template use new explicit feature
* fix mock
* fmt
* use the experimental feature flag instead
* checkout master Cargo.lock
* Merged BEEFY primitives with generic signature and keyset commitment support from old pull to current code
* - Add bls-experimental feature to application-crypto and beefy primitives
- Fix remaining crypto -> ecdsa_crypto
- code build but not tests
* Make beefy primitive tests compile
* move bls related beefy primitives code and test behind bls-experimental flag
* Make BEEFY clients complies with BEEFY API depending on AuthorityId
* - Rename `BeefyAuthoritySet.root` → `BeefyAuthoritySet.keyset_commitment`.
- Remove apk proof keyset_commitment from `BeefyAuthoritySet`.
- Fix failing signed commitment and signature to witness test.
- Make client compatible with BeefyAPI generic on AuthorityId.
- `crypto` → `ecdsa_crypto` in BEEFY client and frame.
* Commit Cargo lock remove ark-serialize from BEEFY primitives
* Use Codec instead of Encode + Decode in primitives/consensus/beefy/src/lib.rs
Co-authored-by: Davide Galassi <davxy@datawok.net>
* - Make `BeefyApi` generic over Signature type.
- Make new `BeeyApi` functinos also generic over AuthorityId and Signature
* Unmake BeefyAPI generic over Signature. Recover Signature type from AuthId.
* - dont use hex or hex-literal use array-bytes instead in beefy primitives and bls crypto.
- CamelCase ECDSA and BLS everywhere.
* Move the definition of BEEFY key type from `primitives/beefy` to `crypto.rs` according to new convention.
* - Add bls377_generate_new to `sp-io` and `application_crypto::bls`.
- Add `bls-experimental` to `sp-io`
Does not compile because PassByCodec can not derive PassBy using customly implemented PassByIner.
* Implement PassBy for `bls::Public` manually
* fix Beefy `KEY_TYPE` in `frame/beefy` tests to come from `sp-core::key_types` enum
* specify both generic for `hex2array_unchecked` in `sp-core/bls.rs`
* Rename `crypto`→`ecdsa_crypto` in `primitives/consensus/beefy/src/test_utils.rs` docs
* remove commented-out code in `primitives/consensus/beefy/src/commitment.rs`
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Fix inconsistency in panic message in `primitives/io/src/lib.rs`
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Remove redundant feature activation in `primitives/io/Cargo.toml`
Co-authored-by: Davide Galassi <davxy@datawok.net>
* - make `w3f-bls` a dev-dependancy only for beefy primitives.
- clean up comments.
Co-authored-by: Davide Galassi <davxy@datawok.net>
* export BEEFY KEY_TYPE from primitives/consensus/beefy
make `frame/consensus/beefy` in dependent of sp_crypto_app
use consistent naming in the beefy primitive tests.
* - implement `BeefyAuthorityId` for `bls_crypto::AuthorityId`.
- implement `bls_verify_works` test for BEEFY `bls_crypto`.
* Remove BEEFY `ecdsa_n_bls_crypto` for now for later re-introduction
* Make commitment and witness BEEFY tests not use Keystore.
* put `bls_beefy_verify_works` test under `bls-experimental` flag.
* bump up Runtime `BeefyAPI` to version 3 due to introducing generic AuthorityId.
* reuse code and encapsulate w3f-bls backend in sp-core as most as possible
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Make comments in primities BEEFY `commitment.rs` and `witness.rs``tests convention conforming
* Use master dep versions
* Trivial change. Mostly to trigger CI
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix Cargo.toml
* Trigger CI with cumulus companion
* Trigger CI after polkadot companion change
---------
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Adrian Catangiu