* Replace libsecp256k1 with secp256k1
* Wipe ecdsa secret key from memory on drop
* Some comments for a known issue
* Safer core crypto primitives `from_slice` constructor
Previous version panics if slice lenght is not the expected one.
* Unit test fix
* Enable use of global secp256k1 context
* Better comments for ecdsa `Pair` drop
* Replace `libsecp256k1` with `seco256k1` in `beefy-mmr`
Used to convert ecdsa public key to ETH address
* Replace `libsecp256k1` with `secp256k1` in FRAME `contracts`benchmarks
* Temporary rollback of `beefy-mmr` to libsecp256k1
Check for detected build issues
* Cargo fmt
* Rollback of FRAME `contracts` benchmarks to `libsecp256k1`
* Rollback for unrelated changes
* Typo fix
* Add comments for deprecated `ecdsa_verify` and `secp256k1_ecdsa_recover`
* Consolidating test and production code
* Signing/verifying authority discovery records with PeerId
Unsigned records cannot be rejected yet, they just produce
a warning in the log.
* Upgrading to libp2p 0.40
* libp2p::identity and sp_core::crypto Ed25519 are compatible
* Rejecting authority records unsigned by peer id can be configured
* Fixes based on review comments
* No command-line argument needed
* info was still too much spam in the logs
* Added tests for both strict and loose validation
* Fixing based on review comments
* Pierre preferred a signing method
* Ooops, I need to slow down
* Update bin/node/cli/src/service.rs
* Reexport libp2p crypto used in sc-network
* Added proto3 compatibility tests. And import noise.
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* authority-discovery: Support multiple authority ids per peer id
An peer id can be mapped to multiple authority ids, because an authority id is a session key that
could be changed every session. Before this pr the internal authority discovery cache assumed that
each authority id can only be mapped to one peer id. However, this isn't true since we changed the
default implementation of the authority discovery to combine the current and next session authorities.
* Review feedback
* Update client/authority-discovery/src/worker/addr_cache.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Early return on no peer ids
* Update client/authority-discovery/src/worker/addr_cache.rs
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update types in comment
* FMT
* Add warning
* Update client/authority-discovery/src/worker/addr_cache.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Feedback
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Clean up sc-network
- Avoid using clone() for the Copy type `PeerId`.
- Use `find_map` for `filter_map` and `next`.
- Use `Self`.
* More on Copy types
* Cargo +nightly fmt --all
* More ..
* fmt
* Revert vec![default_notif_handshake_message]
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* client/authority-discovery: Remove sentry node logic
The notion of sentry nodes has been deprecated (see [1] for details).
This commit removes support for sentry nodes in the
`client/authority-discovery` module.
While removing `Role::Sentry` this commit also introduces
`Role::Discover`, allowing a node to discover addresses of authorities
without publishing ones own addresses. This will be needed in Polkadot
for collator nodes.
[1] https://github.com/paritytech/substrate/issues/6845
* client/authority-discovery/service: Improve PeerId comment
* Asyncify sign_with
* Asyncify generate/get keys
* Complete BareCryptoStore asyncification
* Cleanup
* Rebase
* Add Proxy
* Inject keystore proxy into extensions
* Implement some methods
* Await on send
* Cleanup
* Send result over the oneshot channel sender
* Process one future at a time
* Fix cargo stuff
* Asyncify sr25519_vrf_sign
* Cherry-pick and fix changes
* Introduce SyncCryptoStore
* SQUASH ME WITH THE first commit
* Implement into SyncCryptoStore
* Implement BareCryptoStore for KeystoreProxyAdapter
* authority-discovery
* AURA
* BABE
* finality-grandpa
* offchain-workers
* benchmarking-cli
* sp_io
* test-utils
* application-crypto
* Extensions and RPC
* Client Service
* bin
* Update cargo.lock
* Implement BareCryptoStore on proxy directly
* Simplify proxy setup
* Fix authority-discover
* Pass async keystore to authority-discovery
* Fix tests
* Use async keystore in authority-discovery
* Rename BareCryptoStore to CryptoStore
* WIP
* Remote mutable borrow in CryptoStore trait
* Implement Keystore with backends
* Remove Proxy implementation
* Fix service builder and keystore user-crates
* Fix tests
* Rework authority-discovery after refactoring
* futures::select!
* Fix multiple mut borrows in authority-discovery
* Merge fixes
* Require sync
* Restore Cargo.lock
* PR feedback - round 1
* Remove Keystore and use LocalKeystore directly
Also renamed KeystoreParams to KeystoreContainer
* Join
* Remove sync requirement
* Fix keystore tests
* Fix tests
* client/authority-discovery: Remove event stream dynamic dispatching
With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.
Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.
* Make it compile
* Fix submit_transaction
* Fix block_on issue
* Use await in async context
* Fix manual seal keystore
* Fix authoring_blocks test
* fix aura authoring_blocks
* Try to fix tests for auth-discovery
* client/authority-discovery: Fix lookup_throttling test
* client/authority-discovery: Fix triggers_dht_get_query test
* Fix epoch_authorship_works
* client/authority-discovery: Remove timing assumption in unit test
* client/authority-discovery: Revert changes to termination test
* PR feedback
* Remove deadcode and mark test code
* Fix test_sync
* Use the correct keyring type
* Return when from_service stream is closed
* Convert SyncCryptoStore to a trait
* Fix line width
* Fix line width - take 2
* Remove unused import
* Fix keystore instantiation
* PR feedback
* Remove KeystoreContainer
* Revert "Remove KeystoreContainer"
This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.
* Take a ref of keystore
* Move keystore to dev-dependencies
* Address some PR feedback
* Missed one
* Pass keystore reference - take 2
* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX
Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:
1. No additional mental overhead of a `SyncCryptoStorePtr`.
2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.
Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.
* Remove SyncCryptoStorePtr
* Remove KeystoreContainer & SyncCryptoStorePtr
* PR feedback
* *: Use CryptoStorePtr whereever possible
* *: Define SyncCryptoStore as a pure extension trait of CryptoStore
* Follow up to SyncCryptoStore extension trait
* Adjust docs for SyncCryptoStore as Ben suggested
* Cleanup unnecessary requirements
* sp-keystore
* Use async_std::task::block_on in keystore
* Fix block_on std requirement
* Update primitives/keystore/src/lib.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Fix wasm build
* Remove unused var
* Fix wasm compilation - take 2
* Revert async-std in keystore
* Fix indent
* Fix version and copyright
* Cleanup feature = "std"
* Auth Discovery: Ignore if from_service is cloed
* Max's suggestion
* Revert async-std usage for block_on
* Address PR feedback
* Fix example offchain worker build
* Address PR feedback
* Update Cargo.lock
* Move unused methods to test helper functions
* Restore accidentally deleted cargo.lock files
* Fix unused imports
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* client/authority-discovery: Rename AuthorityDiscovery to XXXWorker
* client/authority-discovery: Introduce AuthorityDiscoveryService
Add a basic `AuthorityDiscoveryService` implementation which enables
callers to get the addresses for a given `AuthorityId` from the local
cache.
* client/authority-discovery: Split into worker and service mod
Move `Service` and `Worker` to their own Rust modules resulting in the
following file structure.
├── build.rs
├── Cargo.toml
└── src
├── error.rs
├── lib.rs
├── service.rs
├── tests.rs
├── worker
│ ├── addr_cache.rs
│ ├── schema
│ │ └── dht.proto
│ └── tests.rs
└── worker.rs
* client/authority-discovery: Cache PeerId -> AuthorityId mapping
* client/authority-discovery: Update priority group on interval
Instead of updating the authority discovery peerset priority group each
time a new DHT value is found, update it regularly on an interval.
This removes the need for deterministic random selection. Instead of
trying to return a random stable set of `Multiaddr`s, the `AddrCache`
now returns a random set on each call.
* client/authority-discovery: Implement Service::get_authority_id
* client/authority-discovery: Use HashMap instead of BTreeMap
* client/authority-discovery: Rework priority group interval
* client/authority-discovery: Fix comment
* bin/node/cli: Update authority discovery constructor
* client/authority-discovery: Fuse from_service receiver
* client/authority-discovery: Remove Rng import
* client/authority-discovery: Ignore Multiaddr without PeerId
* client/authority-discovery/service: Add note on returned None
* client/authority-discovery/addr_cache: Replace double clone with deref
* client/authority-discovery: Don't add own address to priority group
In the scenario of a validator publishing the address of its sentry node
to the DHT, said sentry node should not add its own Multiaddr to the
peerset "authority" priority group.
Related to 273f31b.
* client/authority-discovery: Remove unused import PeerId
* client/authority-discovery/tests: Add tcp protocol to multiaddresses
Previously, when publishing ones address onto the DHT, the signature
signing those addresses would be SCALE encoded twice.
This commit removes the second encoding and adjusts the tests
to catch future regressions.
* client/authority-discovery: Allow to be run by sentry node
When run as a sentry node, the authority discovery module does not
publish any addresses to the dht, but still discovers validators and
sentry nodes of validators.
* client/authority-discovery/src/lib: Wrap lines at 100 characters
* client/authority-discovery: Remove TODO and unused import
* client/authority-discovery: Pass role to new unit tests
* client/authority-discovery: Apply suggestions
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
* bin/node/cli/src/service: Use expressions instead of statements
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* client/authority-discovery: Rework error handling
Instead of `handle_dht_events` returning a `Result<(), Error>`, return
a `Poll<Error>` where `Poll::Pending` signals that there are no more
events to handle and `Poll::Ready(Error)` signals that a fatal error
occured. Non fatal errors are handled within `handle_dht_events`
directly, thus looping in `poll` is not necessary anymore.
* client/authority-discovery: Return () instead of error on termiantion
* Update client/authority-discovery/src/lib.rs
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* client/authority-discovery: Add test ensure termination on network termi
* client/authority-discovery: Terminate when network does
When the dht event stream returns Poll::Ready(None) it is likely due to
the network terminating. When the network terminates due to the node
itself shutting down or due to a fatal error, there is no purpose in
continuing to run the authority discovery module.
* client/authority-discovery/src/lib: Apply suggestions
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:
- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)
refs #4679
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
* Implements mocking of runtime apis
This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.
* Fix warnings
* Update primitives/api/proc-macro/src/utils.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Review feedback
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
This reduces the usage of `Blake2Hasher` in the code base and replaces
it with `BlakeTwo256`. The most important change is the removal of the
custom extern function for `Blake2Hasher`. The runtime `Hash` trait is
now also simplified and directly requires that the implementing type
implements `Hashable`.
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* client/authority-discovery: Limit number of connections to authorities
Instead of connecting to all sentry nodes of all authorities, with this
patch the authority discovery module does the following:
- Choose one sentry node per authority at random.
- Choose MAX_NUM_AUTHORITY_CONN out of the above at random.
The module uses randomness to prevent hot spots, e.g. all nodes trying
to connect to a single node. If the authority discovery module would
choose the nodes to connect to at random on each new address that it
learns of, the node would go through a lot of connection churn. Instead
it creates a random seed at start up and uses this seed for its RNG on
each update cycle.
* client/authority-discovery: Extract address cache into own module
* client/authority-discovery/src/addr_cache: Add basic unit tests
* client/authority-discovery: Replace unwrap with expect on [u8] cmp
* .maintain/sentry-node/docker-compose.yml: Prefix endpoint flags
* client/authority-discovery/src/addr_cache: Use sort_unstable and cmp
* client/authority-discovery: Use BTreeMap in addr_cache for sorted iter
To reduce connection churn it is preferrable to have `get_subset` of the
`addr_cache` to return the same result on repeated calls. `get_subset`
iterates a map. To make the process of iteration deterministic, use a
`BTreeMap` instead of a `HashMap`.
Davide Galassi