With the *authority-discovery* module an authoritative node makes itself
discoverable and is able to discover other authorities. Once discovered, a node
can directly connect to other authorities instead of multi-hop gossiping
information.
1. **Making itself discoverable**
1. Retrieve its external addresses
2. Adds its network peer id to the addresses
3. Sign the above
4. Put the signature and the addresses on the libp2p Kademlia DHT
2. **Discovering other authorities**
1. Retrieve the current set of authorities
2. Start DHT queries for the ids of the authorities
3. Validate the signatures of the retrieved key value pairs
4. Add the retrieved external addresses as ~reserved~ priority nodes to the
peerset
* node/runtime: Add authority-discovery as session handler
The srml/authority-discovery module implements the OneSessionHandler in
order to keep its authority set in sync. This commit adds the module to
the set of session handlers.
* core/network: Make network worker return Dht events on poll
Instead of network worker implement the Future trait, have it implement
the Stream interface returning Dht events.
For now these events are ignored in build_network_future but will be
used by the core/authority-discovery module in subsequent commits.
* *: Add scaffolding and integration for core/authority-discovery module
* core/authority-discovery: Implement module logic itself
* srml-system checks
* wip
* more modules compiles
* node-runtime checks
* build.sh passes
* include dispatch error in failed event
* revert some unnecessary changes
* refactor based on comments
* more compile error fixes
* avoid unnecessary into
* reorder code
* fixes some tests
* manually implement encode & decode to avoid i8 workaround
* more test fixes
* more fixes
* more error fixes
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* address comments
* test for DispatchError encoding
* tyep alias for democracy
* make error printable
* line width
* fix balances tests
* fix executive test
* fix system tests
* bump version
* ensure consistent method signature
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <github@gavwood.com>
* changes based on review
* Add issue number for TODOs
* fix
* line width
* fix test
* Update core/sr-primitives/src/lib.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update core/sr-primitives/src/traits.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update srml/council/src/motions.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update srml/council/src/motions.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* update based on review
* More concrete macro matching
* fix test build issue
* Update hex-literal dependency version. (#3141)
* Update hex-literal dep version.
* Update lock file.
* Start to rework the new error handling
* More work to get it back compiling
* Start to fix after master merge
* The great transaction error handling refactoring
* Make `decl_error` errors convertible to `&'static str`
* Make srml-executive build again
* Fix `sr-primitives` tests
* More fixes
* Last round of fix ups
* Fix build
* Fix build
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Rename some stuff
* Fixes after master merge
* Adds `CheckBlockGasLimit` signed extension
* Remove debug stuff
* Fix srml-balances test
* Rename `InvalidIndex` to `CannotLookup`
* Remove weird generic parameters
* Rename function again
* Fix import
* Document the signed extension
* Change from `Into` to `From`
* Update srml/contracts/src/lib.rs
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Fix compilation
* Update srml/contracts/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update core/sr-primitives/src/transaction_validity.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Remove unused code
* Fix compilation
* Some cleanups
* Fix compile errors
* Make `TransactionValidity` a `Result`
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <gavin@parity.io>
* Beautify the code a little bit and fix test
* Make `CannotLookup` an inherent error declared by `decl_error!`
* Adds some documentation
* Make `ApplyOutcome` a result
* Up the spec_version
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <gavin@parity.io>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* generalize tree_root to remove client.backend dependency
* replace client.backend.blockchain.header with client.header
* move used_state_cache_size into client info
* Create intermediate Setup State. Fixes#1134
* remove client.backend from finality proof
* update node-template
* move memory backend into test helper mode
* move test helper into client
* starting the big refactor, remove unused functions
* apply_finality
* apply_finality
* replacing more .backend from environment with client directly
* remove .backend from grandpa by using traits
* remove .backend from babe
* remove .backend from tests where it is not needed
* remove .backend from tests
* fixing tests
* fixing tests
* fixing more tests
* fixing tests
* fix all forks test
* fix style
* fixing unnecessary allocation
* remove old test.
* fix service docs
* apply suggestion
* minor clean ups
* turns out the test-helper features actually is being used!
* fixing line length.
* fix line length
* minor cleaning
* Apply suggestions from code review
thanks, @Basti
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* address grumbles
* simplify finalize block on client
* move block back into inner function
* Apply suggestions from code review
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* use as.ref instead of match
* Update core/client/src/backend.rs
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* Move Service::new to a macro
* Move function calls to macros
* Extract offchain_workers and start_rpc in separate function
In follow-up commits, we want to be able to directly call maintain_transaction_pool, offchain_workers, and start_rpc, without having to implement the Components trait.
This commit is a preliminary step: we extract the code to freestanding functions.
* Introduce an AbstractService trait
* Introduce NewService as an implementation detail of Service
* Implement traits on NewService instead
Instead of implementing AbstractService, Future, and Executor on Service, we implement them on NewService instead.
The implementations of AbstractService, Future, and Executor on Service still exist, but they just wrap to the respective implementations for NewService.
* Move components creation back to macro invocation
Instead of having multiple $build_ parameters passed to the macro, let's group them all into one.
This change is necessary for the follow-up commits, because we are going to call new_impl! only after all the components have already been built.
* Add a $block parameter to new_impl
This makes it possible to be explicit as what the generic parameter of the NewServiceis, without relying on type inference.
* Introduce the ServiceBuilder struct
Introduces a new builder-like ServiceBuilder struct that creates a NewService.
* Macro-ify import_blocks, export_blocks and revert_chain
Similar to the introduction of new_impl!, we extract the actual code into a macro, letting us get rid of the Components and Factory traits
* Add export_blocks, import_blocks and revert_chain methods on ServiceBuilder
Can be used as a replacement for the chain_ops::* methods
* Add run_with_builder
Instead of just run, adds run_with_builder to ParseAndPrepareExport/Import/Revert. This lets you run these operations with a ServiceBuilder instead of a ServiceFactory.
* Transition node and node-template to ServiceBuilder
* Transition transaction-factory to the new service factory
This is technically a breaking change, but the transaction-factory crate is only ever used from within substrate-node, which this commit updates as well.
* Remove old service factory
* Adjust the AbstractService trait to be more usable
We slightly change the trait bounds in order to make all the methods usable.
* Make substrate-service-test compile
* Fix the node-cli tests
* Remove the old API
* Remove the components module
* Fix indentation on chain_ops
* Line widths
* Fix bad line widths commit
* Line widths again 🤦
* Fix the sync test
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <i@gavwood.com>
* Address some concerns
* Remove TelemetryOnConnect
* Remove informant::start
* Update jsonrpc
* Rename factory to builder
* Line widths 😩
* Add transaction pool to babe import queue
* Add transaction pool to Babe check header
* Fix tests
* Add tx pool to Aura import_queue
* Fix tests, node-template
* Add comments regarding unused _transaction_pool
* Make tx pool optional in check_header
* node: exit on GRANDPA voter or BABE authoring error
* node: exit process with non-zero return code when service fails
* service: rename infallible task to essential task
* service: revert field name changes
* core: fix service testnet
* Scale trait and move to u32 blocknumbers.
* Fixes
* Cleanups
* Update node/runtime/src/lib.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix up some of the factory stuff.
* Update core/sr-primitives/src/traits.rs
Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>
* Move Nonce/Index to u32 (#3361)
* Force a non-borked version of upstream crate
* Line lengths and runtime version bump
* Implement `is_validator` for offchain-workers
* Introduce `--no-validator` flag
* Don't run babe/grandpa/im-online when `--no-validator` is given
* Fixes compilation
* Bump spec version
* Improve error handling in executor
* Add missing extern function
* Revert making error public
* Remove `--no-validator` CLI
* Remove generic from sign()
* Remove mandatory RuntimeGenesis trait req
* Remove requirement from Configuration
* Relax trait bounds of core/cli
* Move method
* The config field is no longer public
* Remove Components from bounds of functions
* Implement DerefMut for LightComponents
* Implement Executor for Full/LightComponents
* Fix bad merge
* Fix forgotten config()
* Fix build
* Add Call type to extensible transactions.
Cleanup some naming
* Merge Resource and BlockExhausted into just Exhausted
* Fix
* Another fix
* Call
* Some fixes
* Fix srml tests.
* Fix all tests.
* Refactor crypto so each application of it has its own type.
* Introduce new AuthorityProvider API into Aura
This will eventually allow for dynamic determination of authority
keys and avoid having to set them directly on CLI.
* Introduce authority determinator for Babe.
Experiment with modular consensus API.
* Work in progress to introduce KeyTypeId and avoid polluting API
with validator IDs
* Finish up drafting imonline
* Rework offchain workers API.
* Rework API implementation.
* Make it compile for wasm, simplify app_crypto.
* Fix compilation of im-online.
* Fix compilation of im-online.
* Fix more compilation errors.
* Make it compile.
* Fixing tests.
* Rewrite `keystore`
* Fix session tests
* Bring back `TryFrom`'s'
* Fix `srml-grandpa`
* Fix `srml-aura`
* Fix consensus babe
* More fixes
* Make service generate keys from dev_seed
* Build fixes
* Remove offchain tests
* More fixes and cleanups
* Fixes finality grandpa
* Fix `consensus-aura`
* Fix cli
* Fix `node-cli`
* Fix chain_spec builder
* Fix doc tests
* Add authority getter for grandpa.
* Test fix
* Fixes
* Make keystore accessible from the runtime
* Move app crypto to its own crate
* Update `Cargo.lock`
* Make the crypto stuff usable from the runtime
* Adds some runtime crypto tests
* Use last finalized block for grandpa authority
* Fix warning
* Adds `SessionKeys` runtime api
* Remove `FinalityPair` and `ConsensusPair`
* Minor governance tweaks to get it inline with docs.
* Make the governance be up to date with the docs.
* Build fixes.
* Generate the inital session keys
* Failing keystore is a hard error
* Make babe work again
* Fix grandpa
* Fix tests
* Disable `keystore` in consensus critical stuff
* Build fix.
* ImOnline supports multiple authorities at once.
* Update core/application-crypto/src/ed25519.rs
* Merge branch 'master' into gav-in-progress
* Remove unneeded code for now.
* Some `session` testing
* Support querying the public keys
* Cleanup offchain
* Remove warnings
* More cleanup
* Apply suggestions from code review
Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>
* More cleanups
* JSONRPC API for setting keys.
Also, rename traits::KeyStore* -> traits::BareCryptoStore*
* Bad merge
* Fix integration tests
* Fix test build
* Test fix
* Fixes
* Warnings
* Another warning
* Bump version.
* Switch consensus-common to new futures
* Fix tests
* More tests fixing
* Make Proposer, OnSlot and SyncOracle mut
* Make the Environment mut as well
* Fix test
* Fix Babe tests
* Babe fixes
* Fix CLI service tests
* Fix Babe tests
* Remove unnecessary trait bound
* Inline the code of BlockBuilder and AuthoringApi
* Remove warning lint
* Bounds simplification
* Imports simplification
* Don't panic on bad generated block
* Code style
* Add doc example
* Remove dependency on aura
* Order dependencies alphabetically
* Minor style
* Make extrinsics extensible.
Also Remove old extrinsic types.
* Rest of mockup. Add tips.
* Fix some build issues
* Runtiem builds :)
* Substrate builds.
* Fix a doc test
* Compact encoding
* Extract out the era logic into an extension
* Weight Check signed extension. (#3115)
* Weight signed extension.
* Revert a bit + test for check era.
* Update Cargo.toml
* Update node/cli/src/factory_impl.rs
* Update node/executor/src/lib.rs
* Update node/executor/src/lib.rs
* Don't use len for weight - use data.
* Operational Transaction; second attempt (#3138)
* working poc added.
* some fixes.
* Update doc.
* Fix all tests + final logic.
* more refactoring.
* nits.
* System block limit in bytes.
* Silent the storage macro warnings.
* More logic more tests.
* Fix import.
* Refactor names.
* Fix build.
* Update srml/balances/src/lib.rs
* Final refactor.
* Bump transaction version
* Fix weight mult test.
* Fix more tests and improve doc.
* Bump.
* Make some tests work again.
* Fix subkey.
* Remove todos + bump.
* First draft of annotating weights.
* Refactor weight to u64.
* More refactoring and tests.
* New convert for weight to fee
* more tests.
* remove merge redundancy.
* Fix system test.
* Bring back subkey stuff.
* a few stress tests.
* fix some of the grumbles.
* Final nits.
* Update srml/system/src/lib.rs
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* Scale weights by 1000.
* Bump.
* Fix decl_storage test.
* babe: add babe module trait
* babe: track current slot and epoch start slot
* babe: implement ShouldEndSession based on epochs
* babe: rename weight type to avoid ambiguities
* babe: expose epoch start slot in Epoch digest
* babe: use epoch start for validating epoch transitions
* babe: make the epoch duration a parameter type
* babe: remove unused fields from config
* node: update runtime to use babe instead of aura
* node: use babe instead of aura
* core: generate sr25519 keys from seed and add to keystore
* core: remove AuthorityKeyring
* node: remove unused primitive types related to babe crypto
* uniform babe primitives crate import name
* wrap long lines
* babe: fix find_epoch_digest
* fork-tree: fix find_node_where
* node: set babe epoch duration to "10 minutes"
* babe: cleanup import key cache if authorities don't change
* node: make integration test compile (but fail)
* node: bump spec_version
* node: fix import
* babe: don't use constants in storage fields array sizes
* babe: account for first epoch slot way in the past
* babe: signal next epoch change (not current)
* babe: calculate next epoch randomness with next epoch index
* babe: track next epoch in node
* babe: cache current epoch and authorities separately
* babe: generate valid babe vrf proofs in integration test
* babe: cleanup claim_slot
* babe: perform threshold calculation according to spec
* babe: compute relative weight in threshold
* babe: more precise threshold calculation
* babe: use floats for threshold exponent calculation
* babe: update constant c
* Make extrinsics extensible.
Also Remove old extrinsic types.
* Rest of mockup. Add tips.
* Fix some build issues
* Runtiem builds :)
* Substrate builds.
* Fix a doc test
* Compact encoding
* Extract out the era logic into an extension
* Weight Check signed extension. (#3115)
* Weight signed extension.
* Revert a bit + test for check era.
* Update Cargo.toml
* Update node/cli/src/factory_impl.rs
* Update node/executor/src/lib.rs
* Update node/executor/src/lib.rs
* Don't use len for weight - use data.
* Operational Transaction; second attempt (#3138)
* working poc added.
* some fixes.
* Update doc.
* Fix all tests + final logic.
* more refactoring.
* nits.
* System block limit in bytes.
* Silent the storage macro warnings.
* More logic more tests.
* Fix import.
* Refactor names.
* Fix build.
* Update srml/balances/src/lib.rs
* Final refactor.
* Bump transaction version
* Fix weight mult test.
* Fix more tests and improve doc.
* Bump.
* Make some tests work again.
* Fix subkey.
* Remove todos + bump.
* Ignore expensive test.
* Bump.
Given the following situation: A validator 'A' is not supposed to be
connected to the public internet to protect it from e.g. a DoS attack.
Instead it connects to a sentry node 'sentry-A' which connects to the
public internet. Validator 'B' can reach validator 'A' via sentry node
'sentry-A' and vice versa.
A sentry node needs to participate in the grandpa gossip without
becoming a validator itself in order to forward these message to its
validator. This commit adds a new command line flag (`--grandpa-voter`)
forcing a node to participate in the grandpa voting process even though
no `--key` was specified. Due to the fact that it does not have a key,
it does not become a validator in the network.
In order to simulate the above situation this commit also adds a Docker
Compose file (`scripts/sentry-node/docker-compose.yml`) with further
documentation.
* SharedFinalityProofRequestBuilder -> BoxFinalityProofRequestBuilder
* SharedThings -> BoxThings
* Fix tests
* build_request_data now takes &mut self
* The other traits now also take &mut self
* More or less fix tests
* Fix tests
* Fix more tests
* Moar tests
* Don't call make_block_import multiple time
* Fix doctest
* Remove dependencies on tokio
* Make service not depend on tokio
* Fix service tests
* Manually poll the import queue if failed to start
* Spawn all tasks at the end
* Remove executor from TelemetryOnConnect
* Remove TaskExecutor from offchain workers
* Remove TaskExecutor from AuthoritySetup
* Remove TaskExecutor from service
* Remove tokio dependency from RPC
* Remove finality-grandpa from WASM checks
* Fix offchain tests
* Line widths
* Fix RPC tests
* Fix service tests
* Fix bad futures polling
* Address some concerns
* Better error handling
* Is it the connectivity test that's not passing? I don't know, let's try
* Revert "Is it the connectivity test that's not passing? I don't know, let's try"
This reverts commit 28bbe51f0e2e4885fe1f901e11078604604cb212.
* Fix test
* Draft of new sessions
* Reintroduce tuple impls
* Move staking module to new session API
* More work on staking and grandpa.
* Use iterator to avoid cloning and tuple macro
* Make runtime build again
* Polish the OpaqueKeys devex
* Move consensus logic into system & aura.
* Fix up system module
* Get build mostly going. Stuck at service.rs
* Building again
* Update srml/staking/src/lib.rs
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* Refactoring out Consensus module, AuthorityIdOf, &c.
* Refactored out DigestItem::AuthoritiesChanged. Building.
* Remove tentative code
* Remove invalid comment
* Make Seal opaque and introduce nice methods for handling opaque items.
* Start to use proper digest for Aura authorities tracking.
* Fix up grandpa, remove system::Raw/Log
* Refactor Grandpa to use new logging infrastructure.
Also make authorityid/sessionkey static. Switch over to storing
authorities in a straight Vec.
* Building again
* Tidy up some AuthorityIds
* Expunge most of the rest of the AuthorityKey confusion.
Also, de-generify Babe and re-generify Aura.
* Remove cruft
* Untangle last of the `AuthorityId`s.
* Sort out finality_tracker
* Refactor median getting
* Apply suggestions from code review
Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>
* Session tests works
* Update core/sr-primitives/src/generic/digest.rs
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* Session tests works
* Fix for staking from @dvc94ch
* log an error
* fix test runtime build
* Some test fixes
* Staking mock update to new session api.
* Fix build.
* Move OpaqueKeys to primitives.
* Use on_initialize instead of check_rotate_session.
* Update tests to new staking api.
* fixup mock
* Fix bond_extra_and_withdraw_unbonded_works.
* Fix bond_with_little_staked_value_bounded_by_slot_stake.
* Fix bond_with_no_staked_value.
* Fix change_controller_works.
* Fix less_than_needed_candidates_works.
* Fix multi_era_reward_should_work.
* Fix nominating_and_rewards_should_work.
* Fix nominators_also_get_slashed.
* Fix phragmen_large_scale_test.
* Fix phragmen_poc_works.
* Fix phragmen_score_should_be_accurate_on_large_stakes.
* Fix phragmen_should_not_overflow.
* Fix reward_destination_works.
* Fix rewards_should_work.
* Fix sessions_and_eras_should_work.
* Fix slot_stake_is_least_staked_validator.
* Fix too_many_unbond_calls_should_not_work.
* Fix wrong_vote_is_null.
* Fix runtime.
* Fix wasm runtime build.
* Update Cargo.lock
* Fix warnings.
* Fix grandpa tests.
* Fix test-runtime build.
* Fix template node build.
* Fix stuff.
* Update Cargo.lock to fix CI
* Re-add missing AuRa logs
Runtimes are required to know about every digest they receive ― they
panic otherwise. This re-adds support for AuRa pre-runtime digests.
* Update core/consensus/babe/src/digest.rs
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* Kill log trait and all that jazz.
* Refactor staking tests.
* Fix ci runtime wasm check.
* Line length 120.
* Make tests build again
* Remove trailing commas in function declarations
The `extern_functions!` macro doesn’t like them, perhaps due to a bug in
rustc.
* Fix type error
* Fix compilation errors
* Fix a test
* Another couple of fixes
* Fix another test
* More test fixes
* Another test fix
* Bump runtime.
* Wrap long line
* Fix build, remove redundant code.
* Issue to track TODO
* Leave the benchmark code alone.
* Fix missing `std::time::{Instant, Duration}`
* Indentation
* Aura ConsensusLog as enum
* Drop import_lock from client, move it into backend, impl default version via static mutex
* still need to allow depcretion because of client.backend
* additional docs
* Remove default impl of get_import_lock, impl on instances
* Bump parking_lot to 0.8.0 accross the board
* restored test_consensus
* restored test_sync
* run integration tests only on CI
* use single test-thread for integration tests
* post-merge fix
* panic when integration test runs for too long
* add some traces to integration logs
* manual wait -> timeout
* post-merge fix
* post-merge fix
* Remove InherentsPool.
* Introduce transactions that are not propagated over the network.
* Bump spec version.
* Use separate flag for propagation.
* Fix tests.
* Silence known deprecation warnings
1. Prefixes known instances of usages of client.backend and client.import_lock with `#[allow(deprecated)]` to silence the warnings. 2. Remove file-global `#![allow(deprecated)]` used in these cases. Both to prevent us overlooking externally caused deprecation messages.
* fixing missing ;
* fix missing test cases
* move deprecated markers to make CI happy
* move deprecated markers to make CI happy
* attempt to fix the test
* bumping impl_version of node runtime
* Minor cleanup
Bastian Köcher