* client/authority-discovery: Allow to be run by sentry node
When run as a sentry node, the authority discovery module does not
publish any addresses to the dht, but still discovers validators and
sentry nodes of validators.
* client/authority-discovery/src/lib: Wrap lines at 100 characters
* client/authority-discovery: Remove TODO and unused import
* client/authority-discovery: Pass role to new unit tests
* client/authority-discovery: Apply suggestions
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
* bin/node/cli/src/service: Use expressions instead of statements
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* client/authority-discovery: Rework error handling
Instead of `handle_dht_events` returning a `Result<(), Error>`, return
a `Poll<Error>` where `Poll::Pending` signals that there are no more
events to handle and `Poll::Ready(Error)` signals that a fatal error
occured. Non fatal errors are handled within `handle_dht_events`
directly, thus looping in `poll` is not necessary anymore.
* client/authority-discovery: Return () instead of error on termiantion
* Update client/authority-discovery/src/lib.rs
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* client/authority-discovery: Add test ensure termination on network termi
* client/authority-discovery: Terminate when network does
When the dht event stream returns Poll::Ready(None) it is likely due to
the network terminating. When the network terminates due to the node
itself shutting down or due to a fatal error, there is no purpose in
continuing to run the authority discovery module.
* client/authority-discovery/src/lib: Apply suggestions
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:
- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)
refs #4679
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
* Split the Roles bitfield in three
* Forgot to include some changes
* Fix cli test
* More test fixes
* Oh God, merging master broke other tests
* Didn't run the doctests
* Address review
* I'm trying to fix the build blindly because it's taking a good hour to compile on my machine
* Address some review
* Also update the peerset's API to make sense
* Fix peerset tests
* Fix browser node
* client: distinguish between local and network authority
Co-authored-by: André Silva <andre.beat@gmail.com>
* Implements mocking of runtime apis
This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.
* Fix warnings
* Update primitives/api/proc-macro/src/utils.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Review feedback
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Add KEY_KIND_ID to the public trait
This change is being introduced for the purpose of identifying a public
key with it's identifier and algorithm "kind".
* Use `sign_with` as implemented in BareCryptoStore
* Implement `sign_with` in sc_keystore
* Fix inconsistencies, use *_KIND_ID in sp_core testing
* Rename KeyKindId to CryptoTypeId
* Remove pair-returning functions from BareCryptoStore trait
* Define CryptoTypeId in app-crypto macros
* Add functions to get keys supported by keystore
* Fix sign_with signature to include CryptoTypePublicPair
* Add `sign_with_any` and `sign_with_all`
* Use keystore.sign_with in auth_discovery
* Rename get_supported_keys -> supported_keys
* Added headers to function docstrings
* Use chain instead of extending a temp vector
* Fixed some code formatting
* Restrict size of CryptoTypeId
This is to be able to use Encode/Decode derives and the overcome having
the size being unknown at compile-time.
* Implement sign_with in the trait itself
* Remove whitespace
* Use key_type also as a CryptoTypeId in app_crypto macros
* Rename `get_keys` to `keys` in BareCryptoStore
* Remove usage of key_pair funcs in tests
* Adjust docstring for *_CYPTO_ID constants
* Fix failures
* Simplify mapping on keys
* Remove one let
* Fixed typo
* PR feedback
* remove whitespace
* Zip keys and signatures
* Use into_iter & remove cloned
* Pass index to MissingSignature
* Use typed errors instead of strings for BareCryptoStore
* Implement Debug for trait error
* Use hashsets for better performance for supported_keys
* Make sure keys are inserted into the keystore
* Make sign_with_all return type consistent with `sign_with`
* Rename Error to BareCryptoStoreError
* Rename CRYPT_TYPE_ID -> CRYPTO_ID
* Remove unnecessary CRYPTO_ID declaration in Public trait
* Convert pub key to CryptoTypePublicPair
* Fix use
* Fix code style
* Implement From on CryptoTypePublicPair in app_crypto macros
* Change CryptoTypePublicPair to a struct
* Implement Display on CryptoTypePublicPair
* Pass CryptoTypePublicPair to MissingSignature error
* Adjust docs according to function signature
* Unify keys implementation
* Fix RPC author tests
* Fix stackoverflow
* Tabify spaces
* Pass KeyTypeId to error for easier debugging
* Fix asserts
* Use ToHex to format public key
* Use constants from sp_core
* Rename testing KeyTypeId constants
* Please compiler
* Restore KeyTypeId names
apparently, they're not only used in tests
* Use BareCryptoStoreError instead of String
* Document return value
* Fix borrow check
* Convert to hashset internally
* WIP - iter_keys
* Return raw_public_keys
* Address PR feedback
* Address PR Feedback
* Fix hexdisplay import error
* Update primitives/core/src/traits.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This reduces the usage of `Blake2Hasher` in the code base and replaces
it with `BlakeTwo256`. The most important change is the removal of the
custom extern function for `Blake2Hasher`. The runtime `Hash` trait is
now also simplified and directly requires that the implementing type
implements `Hashable`.
* client/authority-discovery/Cargo.toml: Update dependency
* client/authority-discovery: Pass packet payload and addresses as slice
Starting with Bytes 0.5 `Vec<T>` does not implement `Buf`, but `&[T]`
does.
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* client/authority-discovery: Limit number of connections to authorities
Instead of connecting to all sentry nodes of all authorities, with this
patch the authority discovery module does the following:
- Choose one sentry node per authority at random.
- Choose MAX_NUM_AUTHORITY_CONN out of the above at random.
The module uses randomness to prevent hot spots, e.g. all nodes trying
to connect to a single node. If the authority discovery module would
choose the nodes to connect to at random on each new address that it
learns of, the node would go through a lot of connection churn. Instead
it creates a random seed at start up and uses this seed for its RNG on
each update cycle.
* client/authority-discovery: Extract address cache into own module
* client/authority-discovery/src/addr_cache: Add basic unit tests
* client/authority-discovery: Replace unwrap with expect on [u8] cmp
* .maintain/sentry-node/docker-compose.yml: Prefix endpoint flags
* client/authority-discovery/src/addr_cache: Use sort_unstable and cmp
* client/authority-discovery: Use BTreeMap in addr_cache for sorted iter
To reduce connection churn it is preferrable to have `get_subset` of the
`addr_cache` to return the same result on repeated calls. `get_subset`
iterates a map. To make the process of iteration deterministic, use a
`BTreeMap` instead of a `HashMap`.
* take test-client off sp-consensus
* use test primitives rather than test client in authority discovery tests
* move runtime-interface tests
* don't forget to remove the dev-dependency
* remove more unneeded dev deps
* add changes_trie_config to test prrimitives
* Separate network crates from its integration tests
* Fix up consensus crates for networking test changes
* remove unnecessary dependencies
* remove unused addition
* remove unnecessary dev-dependencies
* fixing finality grandpa tests
* removing unnecessary executor dependencies
When receiving more addresses for a given authority than a defined
threshold (5), the authority discovery drops the remaining in order to
prevent a single authority to fill all priority group slots.
* client/authority-discovery: Add smoke tests for intervall_at
* client/authority-discovery: Fix interval_at to fire on start
* .maintain/sentry-node: Update docker compose file
* client: Introduce --sentry-nodes flag
Enable operators to specify the public addresses of sentry nodes infront
of a validator node so that the validator node can announce the sentry
node addresses instead of its own public addresses on the DHT via the
authority discovery module.
* client/authority-discovery: Break lines at 100 characters
Limit line length to 100 instead of 120 characters.
* Adding script for rename, could be applicable for nodes on top of it, too
* add stderr and gitlab ci features
* apply script
* fix now minor details in expected stderr
* Update the Cargo.lock
* fix name: sc-transaction -> sc-tracing
* fix rename in script, too
* Move transaction pool to primitives
* move backend, errors into primitives
* remove unused client depencies
* Move rpc-api into primitives
* Move peerset back to client
* Move rpc/api back to client, move palette/support/rpc into utils
* move support-rpc into subfolder
* move system-rpc into utils
* move transaction-pool and -graph back into client
* fix broken imports
* Clean up test primitives
* Make support test utils independent of frame
* remove unnecessary node dependencies from service
* Reactivate dependency script:
- only enforce the now achieved status quo will remain
- allow for primitives to depend on /client for now without failing
- more discriptive error message so people understand, what it wants
- minor fix to differentiative between ../client and /client (which may be a subfolder)
- don't allow this to fail anylonger.
* fix doc comment
* 'Should not' rather than 'must not'.
* Revert unwanted dependency changes
* fix faulty import
* fixup derive_more version
* fix wrong import path
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Max Inden