Commit Graph

6 Commits

Author SHA1 Message Date
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00
Wei Tang 9fdd4672b0 Use tracing-based subscriber logging (#6825)
* init_logger: switch from log-based to tracing-based and add compatibility layer

* Move tracing profiling subscriber related config realization

* sp-tracing: change profiling to be a layer instead of a subscriber

* Enable profiling layer in cli

* Change all test env_logger init to sp_tracing::try_init_simple

* Remove all local env_logger dependency

* Add missing tracing-subscriber dependency

* frame-sudo: fix tests

* frame-support: fix tests

* Fix frame/pallet and executor tests

* Fix the remaining tests

* Use subscriber's try_init as recommended by @davidbarsky

* Be explict that the tracing-log feature is needed

* Set subscriber writer to stderr

* Shorter line width

* Update cargo lock tracing version

* Fix sc_tracing crate compile

* Fix sc_authority_discovery crate test

* unremove default-features

* Leave enabled to default true

* Warn if global default cannot be set

* Fix unused import

* Remove unused PROXY_TARGET

* Change all reference from rc5 to rc6

* Change all reference of rc2 to rc6

* Fix styling

* Fix typo

* make logger init error'ing

* re-fixing the test issue

Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-09-17 11:04:43 +02:00
Max Inden f677b40ed7 client/authority-discovery: Throttle DHT requests (#7018)
* client/authority-discovery: Throttle DHT requests

Instead of passing one DHT query for each authority down to the network
every query interval, only pass MAX_IN_FLIGHT_LOOKUPS at a given point
in time, triggering new ones when previous ones return.

* client/authority-discovery/worker/test: Fix wrong constant
2020-09-07 10:17:28 +00:00
Max Inden 9a17d58960 client/authority-discovery: Limit number of addresses per authority (#6947)
* client/authority-discovery: Test addresses per authority limit

* client/authority-discovery: Limit number of addresses per authority
2020-08-24 14:17:39 +00:00
Max Inden 5c500aa783 client/authority-discovery: Append PeerId to Multiaddr at most once (#6933)
* client/authority-discovery/worker: Extract address getter

* client/authority-discovery: Test for no duplicate p2p components

* client/authority-discovery: Append PeerId to Multiaddr at most once

When collecting the addresses to be published for the local node,
`addresses_to_publish` adds the local nodes `PeerId` to each
`Multiaddr`. Before doing so, ensure the `Multiaddr` does not already
contain one.

* client/authority-discovery: Remove explicit return
2020-08-24 09:37:32 +02:00
Max Inden 2d5ec72331 client/authority-discovery: Introduce AuthorityDiscoveryService (#6760)
* client/authority-discovery: Rename AuthorityDiscovery to XXXWorker

* client/authority-discovery: Introduce AuthorityDiscoveryService

Add a basic `AuthorityDiscoveryService` implementation which enables
callers to get the addresses for a given `AuthorityId` from the local
cache.

* client/authority-discovery: Split into worker and service mod

Move `Service` and `Worker` to their own Rust modules resulting in the
following file structure.

├── build.rs
├── Cargo.toml
└── src
    ├── error.rs
    ├── lib.rs
    ├── service.rs
    ├── tests.rs
    ├── worker
    │   ├── addr_cache.rs
    │   ├── schema
    │   │   └── dht.proto
    │   └── tests.rs
    └── worker.rs

* client/authority-discovery: Cache PeerId -> AuthorityId mapping

* client/authority-discovery: Update priority group on interval

Instead of updating the authority discovery peerset priority group each
time a new DHT value is found, update it regularly on an interval.

This removes the need for deterministic random selection. Instead of
trying to return a random stable set of `Multiaddr`s, the `AddrCache`
now returns a random set on each call.

* client/authority-discovery: Implement Service::get_authority_id

* client/authority-discovery: Use HashMap instead of BTreeMap

* client/authority-discovery: Rework priority group interval

* client/authority-discovery: Fix comment

* bin/node/cli: Update authority discovery constructor

* client/authority-discovery: Fuse from_service receiver

* client/authority-discovery: Remove Rng import

* client/authority-discovery: Ignore Multiaddr without PeerId

* client/authority-discovery/service: Add note on returned None

* client/authority-discovery/addr_cache: Replace double clone with deref
2020-08-12 14:16:40 +00:00