* Remove necessity to pass ConsensusEngineId when registering notifications protocol
* Line width
* Fix tests protocol name
* Other renames
* Doc update
* Change issue in TODO
* grandpa: don't send equivocation reports for local identities
* grandpa: add test for self-report
* grandpa: fix test compilation
this works on rust nightly but breaks on ci which is using rust stable
* grandpa: fix early enactment of forced authority set changes
* grandpa: add test for early enactment of forced changes
* grandpa: fix typo in log message
* grandpa: only allow one pending forced change per fork
* grandpa: fix tests
* Initial draft
* Add an iterator that helps us get most items
* Revert changes to grandpa
* Change fields to just be the grandpa authority set and babe epoch changes
* Only use the fields we need from the shared authority set
* Switch to RPC call
* Revert "Only use the fields we need from the shared authority set"
This reverts commit 6ede87b0c5fe53f251d7cb45951006a7dc8f9b83.
* Add babe_finalized_block_weight from `ashley-improve-sync-state-WIP-loading`
* Fix rpc test
* Move sync state rpc stuff into sc-sync-state-rpc
* Remove as_json_value and remove unwraps from sc-sync-state-rpc
* Add clone_inner to SharedAuthoritySet
* Asyncify sign_with
* Asyncify generate/get keys
* Complete BareCryptoStore asyncification
* Cleanup
* Rebase
* Add Proxy
* Inject keystore proxy into extensions
* Implement some methods
* Await on send
* Cleanup
* Send result over the oneshot channel sender
* Process one future at a time
* Fix cargo stuff
* Asyncify sr25519_vrf_sign
* Cherry-pick and fix changes
* Introduce SyncCryptoStore
* SQUASH ME WITH THE first commit
* Implement into SyncCryptoStore
* Implement BareCryptoStore for KeystoreProxyAdapter
* authority-discovery
* AURA
* BABE
* finality-grandpa
* offchain-workers
* benchmarking-cli
* sp_io
* test-utils
* application-crypto
* Extensions and RPC
* Client Service
* bin
* Update cargo.lock
* Implement BareCryptoStore on proxy directly
* Simplify proxy setup
* Fix authority-discover
* Pass async keystore to authority-discovery
* Fix tests
* Use async keystore in authority-discovery
* Rename BareCryptoStore to CryptoStore
* WIP
* Remote mutable borrow in CryptoStore trait
* Implement Keystore with backends
* Remove Proxy implementation
* Fix service builder and keystore user-crates
* Fix tests
* Rework authority-discovery after refactoring
* futures::select!
* Fix multiple mut borrows in authority-discovery
* Merge fixes
* Require sync
* Restore Cargo.lock
* PR feedback - round 1
* Remove Keystore and use LocalKeystore directly
Also renamed KeystoreParams to KeystoreContainer
* Join
* Remove sync requirement
* Fix keystore tests
* Fix tests
* client/authority-discovery: Remove event stream dynamic dispatching
With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.
Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.
* Make it compile
* Fix submit_transaction
* Fix block_on issue
* Use await in async context
* Fix manual seal keystore
* Fix authoring_blocks test
* fix aura authoring_blocks
* Try to fix tests for auth-discovery
* client/authority-discovery: Fix lookup_throttling test
* client/authority-discovery: Fix triggers_dht_get_query test
* Fix epoch_authorship_works
* client/authority-discovery: Remove timing assumption in unit test
* client/authority-discovery: Revert changes to termination test
* PR feedback
* Remove deadcode and mark test code
* Fix test_sync
* Use the correct keyring type
* Return when from_service stream is closed
* Convert SyncCryptoStore to a trait
* Fix line width
* Fix line width - take 2
* Remove unused import
* Fix keystore instantiation
* PR feedback
* Remove KeystoreContainer
* Revert "Remove KeystoreContainer"
This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.
* Take a ref of keystore
* Move keystore to dev-dependencies
* Address some PR feedback
* Missed one
* Pass keystore reference - take 2
* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX
Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:
1. No additional mental overhead of a `SyncCryptoStorePtr`.
2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.
Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.
* Remove SyncCryptoStorePtr
* Remove KeystoreContainer & SyncCryptoStorePtr
* PR feedback
* *: Use CryptoStorePtr whereever possible
* *: Define SyncCryptoStore as a pure extension trait of CryptoStore
* Follow up to SyncCryptoStore extension trait
* Adjust docs for SyncCryptoStore as Ben suggested
* Cleanup unnecessary requirements
* sp-keystore
* Use async_std::task::block_on in keystore
* Fix block_on std requirement
* Update primitives/keystore/src/lib.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Fix wasm build
* Remove unused var
* Fix wasm compilation - take 2
* Revert async-std in keystore
* Fix indent
* Fix version and copyright
* Cleanup feature = "std"
* Auth Discovery: Ignore if from_service is cloed
* Max's suggestion
* Revert async-std usage for block_on
* Address PR feedback
* Fix example offchain worker build
* Address PR feedback
* Update Cargo.lock
* Move unused methods to test helper functions
* Restore accidentally deleted cargo.lock files
* Fix unused imports
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* grandpa-rpc: use FinalityProofProvider to check finality for rpc
* grandpa-rpc: minor tidy
* grandpa-rpc: remove dyn FinalityProofProvider
* grandpa-rpc: remove unused dependencies
* node: move finality_proof_provider setup
* grandpa-rpc: print error reported by finality_proof_provider
* grandpa-rpc: add note about unnecessary encode/decode
* grandpa-rpc: dont encode/decode and use correct hash
* grandpa-rpc: set_id is optional
* grandpa-rpc: create test for prove_finality
* grandpa-rpc: set visibility back to how it was
* grandpa-rpc: remove unused dependency
* grandpa-rpc: minor tidy
* grandpa: doc strings
* grandpa-rpc: rename to prove_finality
* grandpa-rpc: use current set id if none is provided
* grandpa-rpc: remove unnecessary check in test
* node: group finality_proof_provider in rpc_setup
* grandpa: make prove_finality concrete in FinalityProofProvider
* grandpa-rpc: wrap finality output in struct and store as Bytes
* grandpa-rpc: exhaustive error codes and wrap
* grandpa-rpc: let prove_finality take a range instead of a starting point
* grandpa-rpc: fix test for changed API
* grandpa-rpc: fix line length
* grandpa: fix reviewer nits
* node/rpc: fix reviewer comments
* init_logger: switch from log-based to tracing-based and add compatibility layer
* Move tracing profiling subscriber related config realization
* sp-tracing: change profiling to be a layer instead of a subscriber
* Enable profiling layer in cli
* Change all test env_logger init to sp_tracing::try_init_simple
* Remove all local env_logger dependency
* Add missing tracing-subscriber dependency
* frame-sudo: fix tests
* frame-support: fix tests
* Fix frame/pallet and executor tests
* Fix the remaining tests
* Use subscriber's try_init as recommended by @davidbarsky
* Be explict that the tracing-log feature is needed
* Set subscriber writer to stderr
* Shorter line width
* Update cargo lock tracing version
* Fix sc_tracing crate compile
* Fix sc_authority_discovery crate test
* unremove default-features
* Leave enabled to default true
* Warn if global default cannot be set
* Fix unused import
* Remove unused PROXY_TARGET
* Change all reference from rc5 to rc6
* Change all reference of rc2 to rc6
* Fix styling
* Fix typo
* make logger init error'ing
* re-fixing the test issue
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* grandpa: report metrics on prevotes and precommits cast
* Update client/finality-grandpa/src/environment.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Update client/finality-grandpa/src/environment.rs
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Max Inden <mail@max-inden.de>
* Stop sending messages on legacy substream altogether
* Ensure that handshake is sent back even in case of back-pressure
* Update client/network/src/protocol/generic_proto/handler/group.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Also process OpenRequest and Closed
* Also process OpenRequest and Closed
* Fix bad merge
* God I'm so lost with all these merges
* Immediately return Closed
* Add warning for sending on non-registered protocol
* Register GrandPa protocol in tests
* Update client/network/src/protocol/generic_proto/handler/group.rs
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Max Inden <mail@max-inden.de>
* client/*: Treat protocol name as str and not [u8]
Notification protocol names are in practice always valid utf8 strings.
Instead of treating them as such in the type system, thus far they were
casted to a [u8] at creation time.
With this commit protocol names are instead treated as valid utf8
strings throughout the codebase and passed as `Cow<'static, str>`
instead of `Cow<'static, [u8]>`. Among other things this eliminates the
need for string casting when logging.
* client/network: Don't allocate when protocol name is borrowed
* grandpa: use bytes type for justification rpc notification
* grandpa: always create justification if there are rpc subscribers
* grandpa: wording
* grandpa: replace notify_justification macro with function
* grandpa: prefer Option<&T> over &Option<T>
* Rough skeleton for what I think the RPC should look like
* Create channel for sending justifications
Sends finalized header and justification from Grandpa to the
client. This lays the groundwork for hooking into the RPC module.
* WIP: Add subscribers for justifications to Grandpa
Adds the Sender end of a channel into Grandpa, through which notifications
about block finality events can be sent.
* WIP: Add a struct for managing subscriptions
Slightly different approach from the last commit, but same
basic idea. Still a rough sketch, very much doesn't compile yet.
* Make naming more clear and lock data in Arc
* Rough idea of what RPC would look like
* Remove code from previous approach
* Missed some things
* Update client/rpc-api/src/chain/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update client/rpc-api/src/chain/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Split justification subscription into sender and receiver halves
* Replace RwLock with a Mutex
* Add sample usage from the Service's point of view
* Remove code that referred to "chain_" RPC
* Use the Justification sender/receivers from Grandpa LinkHalf
* Add some PubSub boilerplate
* Add guiding comments
* TMP: comment out to fix compilation
* Return MetaIoHandler from PubSubHandler in create_full
* Uncomment pubsub methods in rpc handler (fails to build)
* node/rpc: make Metadata concrete in create_full to fix compilation
* node: pass in SubscriptionManger to grandpa rpc handler
* grandpa-rpc: use SubscriptionManger to add subscriber
* grandpa-rpc: attempt at setting up the justification stream (fails to build)
* grandpa-rpc: fix compilation of connecting stream to sink
* grandpa-rpc: implement unsubscribe
* grandpa-rpc: update older tests
* grandpa-rpc: add full prefix to avoid confusing rust-analyzer
* grandpa-rpc: add test for pubsub not available
* grandpa-rpc: tidy up leftover code
* grandpa-rpc: add test for sub and unsub of justifications
* grandpa-rpc: minor stylistic changes
* grandpa-rpc: split unit test
* grandpa-rpc: minor stylistic changes in test
* grandpa-rpc: skip returning future when cancelling
* grandpa-rpc: reuse testing executor from sc-rpc
* grandpa-rpc: don't need to use PubSubHandler in tests
* node-rpc: use MetaIoHandler rather than PubSubHandler
* grandpa: log if getting header failed
* grandpa: move justification channel creation into factory function
* grandpa: make the justification sender optional
* grandpa: fix compilation warnings
* grandpa: move justification notification types to new file
* grandpa-rpc: move JustificationNotification to grandpa-rpc
* grandpa-rpc: move JustificationNotification to its own file
* grandpa: rename justification channel pairs
* grandpa: rename notifier types
* grandpa: pass justification as GrandpaJustification to the rpc module
* Move Metadata to sc-rpc-api
* grandpa-rpc: remove unsed error code
* grandpa: fix bug for checking if channel is closed before sendind
* grandpa-rpc: unit test for sending justifications
* grandpa-rpc: update comments for the pubsub test
* grandpa-rpc: update pubsub tests with more steps
* grandpa-rpc: fix pubsub test
* grandpa-rpc: minor indendation
* grandpa-rpc: decode instead of encode in test
* grandpa: fix review comments
* grandpa: remove unused serde dependency
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
* Remove service components and add build_network, build_offchain_workers etc
* Improve transaction pool api
* Remove commented out line
* Add PartialComponents
* Add BuildNetworkParams, documentation
* Remove unused imports in tests
* Apply suggestions from code review
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Remove unused imports in node-bench
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* grandpa: remove unused methods to convert digest
* grandpa: add root extrinsic for scheduling forced change
* grandpa: add benchmark for schedule_forced_change
* grandpa: don't take authority weight in schedule_forced_change
* grandpa: add const for default forced change delay
* grandpa: adjust weights after benchmark on ref hardware
* grandpa: fix cleanup of forced changes on standard change application
* grandpa: replace schedule_forced_change with note_stalled
* grandpa: always trigger a session change when the set is stalled
* grandpa: fix bug on set id mutation after failed scheduled change
* grandpa: take delay as parameter in note_stalled
* grandpa: fix tests
* grandpa: fix cleanup of forced changes
* grandpa: add test for forced changes cleanup
* grandpa: add test for session rotation set id
* grandpa: add test for scheduling of forced changes on new session
* Simplify a few chain components creation APIs related to the service
* Fix basic-authorship doc tests
* Remove DefaultQueue
* Update client/service/src/builder.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Move ExecutionExtensions comment around
* Remove unused BlakeTwo256
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* grandpa: replace Result<(), ()> with Option<()>
* grandpa: replace &Option<T> with Option<&T>
* grandpa: cleanup local id and keystore usages
* grandpa: return bool on check_message_signature
* grandpa: fix erroneous log message on startup
* grandpa: fix test
* WIP
* Making progress
* Almost ready
* Get service tests compiling
* Fix node screenshot
* Line widths
* Fix node cli tests
* Fix node cli warning
* ChainComponents -> ServiceComponents, fix tests
* make spawn_handle public
* Remove spawnnamed impl for taskmanager
* Move the keep alive stuff to the task manager
* Move the telemetry, base path, rpc keep_alive to the service builder
* Make the task manager keep alive an internal detail
* Rewrite the browser start_client future
* Remove run_node etc
* Revert my personal changes to browser-demo/build.sh
* use |config|
* Add a runtime_version function to SubstrateCli
* Reexport role and runtime version from sc cli
* Update Cargo.lock
* runtime_version -> native_runtime_version
* Pass chain spec to native_runtime_version for polkadot
* Fix line widths
* Traitify ServiceComponents Client
* Use sign_with and stop using `Pair`
* PR feedback
* Remove clone
* Transfer ownership of public to sign_message
* Use Option
* Simplify code
* Fix error message
* Pass keystore as ref
* Pass keystore properly
* Fix tests
* Make tx pool aware of retracted fork blocks
* Make it compile
* Update client/transaction-pool/src/lib.rs
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Fix doc test
* Simplify the implementation
* Send tree route as arc to prevent heavy clones
* Switch to use `ExtrinsicHash` to make it more clear
* Fix benchmark
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
An `UntilImported` stream wraps a `Stream` of incoming messages and
waits for blocks those messages are based on before passing the messages
on.
The above `Stream` of incoming messages implements `Unpin`, thus there
is no need to use `pin_project` on the `UntilImported` struct. Instead
one only has to add the `Unpin` trait bound on the `I` trait parameter.
* Make `Digest` support `StorageAppend`
This adds support for `StorageAppend` to `Digest`. Digest is just a
wrapper around a `Vec` and we abuse the fact that SCALE does not puts
any special marker into the encoding for structs. So, we can just append
to the encoded Digest. A test is added that ensures, if the `Digest`
format ever changes, we remove this optimization.
* Update weight
* Update frame/support/src/storage/mod.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Update frame/system/src/lib.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* grandpa: wire up basic RPC call
* grandpa: make it compile against GRANDPA with expose round state
* grandpa: use shared voter state to expose RPC endpoint
* grandpa: restructure into nested structs
* grandpa: return background rounds too
* grandpa: return error when endpoint not ready
* grandpa: collect grandpa rpc deps
* grandpa: decide to use concrete AuthorityId in finality-grandpa-rpc
* grandpa: remove unncessary type annotation
* grandpa: move error code to const
* grandpa: remove unnecessary WIP comment
* grandpa: remove Id type parameter for SharedVoterState
* grandpa: update tests to add shared_voter_state in parameters
* grandpa: remove old deprecated test
* grandpa: fix getting the correct set_id
* grandpa: make SharedVoterState a struct
* grandpa: wrap shared_voter_state in rpc_setup
* grandpa: replace spaces with tabs
* grandpa: limit RwLock write attempt to 1 sec
* grandpa: add missing doc comments and remove some pub
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-Authored-By: Hernando Castano <HCastano@users.noreply.github.com>
* grandpa: update function name call after change in finality-grandpa
* grandpa: group pub use and only export voter::report
* grandpa: add missing docs
* grandpa: extract out structs used for json serialization
* grandpa: stick to u32 for fields intended for js
* grandpa: move Error type to its own file
* grandpa: group pub use better
* Apply code review suggestion
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* grandpa: use correct version of finality-granpda in rpc crate
* grandpa: add back basic rpc unit test
* grandpa: replace SharedVoterState::new() with empty()
* node: cleanup grandpa::SharedVoterState usage in macro
* grandpa: remove VoterState error variant
* grandpa: enable missing futures compat feature
* grandpa: fix typo in error variant
* grandpa: remove test_utils
* grandpa: allow mocking rpc handler components
* grandpa: rename serialized to report in rpc module
* grandpa: add proper test for RPC
* grandpa: update to finality-grandpa v0.12.1
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api
* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa
* drop sc-service from sc-rpc
* drop sc-service from sc-consensus-aura
* drop sc-client from manual-seal and babe
* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli
* drop sc-client from bin/node and bin/node-template
* drop sc-client
* fix tests
* remove check -p sc-client from gitlab.yml
* fix warnings
* fixes ui test
* fix light client tests
* adds associated Client type to AbstractService
* adds UsageProvider to Client
* fixed ui test, again
* tried and failed to get node-cli to compile for wasm
* thanks to tomaka for helping me get node-cli to compile for wasmm
* ui test pls pas 🙏🏾
* all tests passing 🪄
* no_run documentation code
* rm -f documentation code
* ClientProvider
* fix mega trait
* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client
* adds license to sc-consensus
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* babe: restrict info logging during initial sync
* grandpa: restrict info logging during initial sync
* grandpa: fix test compilation
* grandpa: remove afg_log macro mod
This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:
- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)
refs #4679
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
André Silva