* Removal of light client from substrate
* add missing import
* These tests relate to there being light and non light clients.
* removing lightnodes from test
* cargo fmt
* not needed
* LightDataChecker not needed any longer
* cargo fmt
* Update client/service/test/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/test/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* cargo fmt
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Use tokio runtime handle instead of TaskExecutor abstraction
Before this pr we had the `TaskExecutor` abstraction which theoretically
allowed that any futures executor could have been used. However, this
was never tested and is currently not really required. Anyone running a
node currently only used tokio and nothing else (because this was hard
coded in CLI). So, this pr removes the `TaskExecutor` abstraction and
relies directly on the tokio runtime handle.
Besides this changes, this pr also makes sure that the http and ws rpc
server use the same tokio runtime. This fixes a panic that occurred when
you drop the rpc servers inside an async function (tokio doesn't like
that a tokio runtime is dropped in the async context of another tokio
runtime).
As we don't use any custom runtime in the http rpc server anymore, this
pr also removes the `rpc-http-threads` cli argument. If external parties
complain that there aren't enough threads for the rpc server, we could
bring support for increasing the thread count of the tokio runtime.
* FMT
* Fix try runtime
* Fix integration tests and some other optimizations
* Remove warnings
* sp-utils => sc-utils
* cargo fmt
* These files are now in the client so should be licensed as GPL3
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Split native executor stuff from wasm executor stuff
* Remove `native_runtime_version` in places
* Fix warning
* Fix test warning
* Remove redundant NativeRuntimeInfo trait
* Add a warning for use_native
* Run cargo fmt
* Revert "Add a warning for use_native"
This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.
* Make choosing an executor (native/wasm) an explicit part of service construction
* Add Cargo.lock
* Rename Executor to ExecutorDispatch
* Update bin/node/executor/src/lib.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Fix tests
* Fix minor node-executor error
* Fix node cli command thing
Co-authored-by: Squirrel <gilescope@gmail.com>
* Comment out browser stuff
* Remove browser stuff
* Remove more wasm transport code
* Remove ExtTransport and rework how telemetry initialises.
* Change (most) wasm-timer using code to use std::time
* Rename CI-job
* Aura does not compile for wasm
* Remove testing in the browser on CI
* Update README
* Leave `StreamSink` be
* fmt
* Decouples light-sync state from chain spec
This decouples the light-sync state from chain spec. First, the
light-sync state currently only works with BABE+Grandpa, so not
all *Substrate* based chains can use this feature. The next problem was
also that this pulled the `sc-consensus-babe` and `sc-finality-grandpa`
crate into `sc-chain-spec`.
If a chain now wants to support the light-sync state, it needs to add
the `LightSyncStateExtension` to the chain spec as an extension. This is
documented in the crate level docs of `sc-sync-state-rpc`. If this
extension is not available, `SyncStateRpc` fails at initialization.
* Fix compilation for browser
* Fmt
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Make it possible to override maximum payload of RPC
* Finish it.
* remove todo.
* Update client/cli/src/commands/run_cmd.rs
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Incorporate suggestions
* Thread rpc_max_payload from configuration to trace_block
* Try obey line gitlab/check_line_width.sh
* update state rpc tests
* Improve readbility
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
This introduces a new field `code_substitute` into the chain spec. This
can be used to substitute the on-chain wasm starting from a given block
until there is another wasm on chain (determined through the
`spec_version`). This can be used to fix broken on chain wasm runtimes.
* Add MMR custom RPC.
* Change RuntimeApi to avoid hardcoding leaf type.
* Properly implement the new RuntimeAPI and wire up RPC.
* Extract Offchain DB as separate execution extension.
* Enable offchain DB access for offchain calls.
* Fix offchain_election tests.
* Skip block initialisation for proof generation.
* Fix integration test setup.
* Fix offchain tests. Not sure how I missed them earlier 🤷.
* Fix long line.
* One more test missing.
* Update mock for multi-phase.
* Address review grumbbles.
* Address review grumbles.
* Fix line width of a comment
* Decrease the peer reputation on invalid block requests
This pr changes the block request handler to decrease the reputation of
peers when they send the same request multiple times or they send us an
invalid block request.
* Review feedback
* Change log target
* Remove unused code
* Fix tracing tests (#8022)
* Fix tracing tests
The tests were not working properly.
1. Some test was setting a global subscriber, this could lead to racy
conditions with other tests.
2. A logging test called `process::exit` which is completly wrong.
* Update client/tracing/src/lib.rs
Co-authored-by: David <dvdplm@gmail.com>
* Review comments
Co-authored-by: David <dvdplm@gmail.com>
* Fix tracing spans are not being forwarded to spawned task (#8009)
* Fix tracing spans are not being forwarded to spawned task
There is a bug that tracing spans are not forwarded to spawned task. The
problem was that only the telemetry span was forwarded. The solution to
this is to use the tracing provided `in_current_span` to capture the
current active span and pass the telemetry span explictely. We will now
always enter the span when the future is polled. This is essentially the
same strategy as tracing is doing with its `Instrumented`, but now
extended for our use case with having multiple spans active.
* More tests
* Proper test for telemetry and prefix span
* WIP
* Fix test (need to create & enter the span at the same time)
* WIP
* Remove telemtry_span from sc_service config
* CLEANUP
* Update comment
* Incorrect indent
* More meaningful name
* Dedent
* Naming XD
* Attempt to make a more complete test
* lint
* Missing licenses
* Remove user data
* CLEANUP
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* CLEANUP
* Apply suggestion
* Update bin/node/cli/tests/telemetry.rs
Co-authored-by: David <dvdplm@gmail.com>
* Wrapping lines
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
* client/network: Re-enable light_client_handler.rs unit tests
* client/network: Add scaffolding for light client using req-resp
* client/network: Make it compile
* client/network: Rename OutEvent SendRequest
* client/network: Restructure light client request client and handler
* client/network: Rename light client request client to sender
* client/network: Remove light client prepare_request
* client/network/src/light: Rework configuration
* client/network: Formatting
* client/network/light: Remove RequestId
* client/network/light: Make request functions methods
* client/network/light: Refactor request wrapping
* client/network/light: Fix warnings
* client/network/light: Serialize request in method
* client/network/light: Make returning response a method
* client/network/light: Depend on request response to timeout requests
* client/network: Fix test compilation
* client/network/light: Re-enable connection test
* client/network/light: Re-enable timeout test
* client/network/light: Re-enable incorrect_response test
* client/network/light: Re-enable wrong_response_type test
* client/network/light: Re-enable retry_count_failures test
* client/network/light: Re-enable issue_request tests
* client/network/light: Re-enable send_receive tests
* client/network/light: Deduplicate test logic
* client/network/light: Remove unused imports
* client/network/light: Handle request failure
* client/network/light: Move generate_protocol_config
* client/network: Fix test compilation
* client/network: Rename light client request client to sender
* client/network: Handle too-many-requests error
* client/network: Update outdated comments
* client/network/light: Choose any peer if none has best block defined
* .maintain: Replace sentry-node with local-docker-test-network
Sentry nodes are deprecated. Thus there is no need for
`.maintain/sentry-node` to spin up a sentry node test environment.
Instead this commit rewrites the setup to contain two full-connected
validators and one light client.
With the steps below one can now spin up a local test network with
two validators, one light-client, Prometheus and Grafana.
- cargo build --release
- sudo docker-compose -f .maintain/local-docker-test-network/docker-compose.yml up
* client/network/light: Handle oneshot cancellation
* client/network/light: Do not reduce retry count on missing peer
* client/network/request-response: Assert in debug request id to be unique
* client/network/light: Choose same limit as block request protocol
* client/network: Report reputation changes via response
Allow request response protocol handlers to issue reputation changes, by
sending them back along with the response payload.
* client/network: Remove resolved TODOs
* Made a start
* So the proof between authority set is phragmen one, this is crazy big,
or is there some signing of the result : that is the storage key, damn?
* ok getting from header digest seems doable.
* for testing
* get set id from storage directly (should use runtime to handler change).
* move test to init
* correct auth key
* fix iteration
* Correct proof content
* actually update block number.
* actually check last justif against its header
* justification relation to new authorities through header hash check is
needed here. This assumes the hash from header is calculated.
* Few changes
* Connected up cheme's branch
* Clean up
* Move things around a bit so that adding the grandpa warp sync request response protocol happens in the node code
* Nits
* Changes to comments
* Cheme changes
* Remove todos and test compile.
* Rename _authority_ related proof function to _warp_sync_ .
* Update client/grandpa-warp-sync/src/lib.rs
quick fix
* Put the warp sync request response protocol behind a feature flag because we dont' need it on a light client.
* Update client/grandpa-warp-sync/src/lib.rs
Quick fix
* Update Cargo.lock
* Adding test, comment on limitation related to 'delay', this could
be implemented but with a cost.
* Set between a delay override last fragment.
* Check for pending authority set change at start.
* adjust index
* custom cache is not a good idea.
* Use a simple cache instead.
* restore broken indentation
* Address crate rename
* Merge conflict badly resolved, sorry
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* update to latest master
* updates on docs, license, meta
* hide ssrs behind feature flag
* implement remaining functions on the server
* sign server line length fix
* fix tests
* fixup in-memory-keystore
* adding failsafe
* skipping ecdsa test for now
* remote keystore param
* remote sign urls made available
* integrating keystore remotes features
* don't forget the dependency
* remove old cruft
* reset local keystore
* applying suggestions
* Switch to single remote, minor grumbles
* minor grumbles, docs
* Remove `RpcMetrics` weirdness
The metrics was returning an error when prometheus was not given. This
was a really weird setup, especially when compared to all other metrics
that just do nothing if there is no registry.
* Fix browser build
* Export app-crypto specific keystore functions
* Also add back the insert function
* Switch KeystoreContainer to an enum
* Only export the bare minimal for LocalKeystore and fix service compile
* fix: should return Arc
* Add docs stating that functions only available in local keystore
* Remove insert and generate functions
* fix: generate function should be available in test
* Add keypair function to trait
* Revert "Add keypair function to trait"
This reverts commit ad921b09ca73d3c09298e3a51b562ef8e0067781.
* Add note for local_keystore function in service
* Provide WASM overwrite functionality in LocalCallExecutor
- add a new module `wasm_overwrite.rs` in client
- scrapes given folder for runtimes
- add two new CLI Options `wasm-overwrite` and `wasm_overwrite_path`
* formatting
* Make comment clearer
remove sc-runtime-test from dev-dependencies
* comments
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix spaces, remove call into backend for 'heap_pages' in 'try_replace'
* Error if path is not a directory, Comments,
Doc Comment for WasmOverwrite
* make WasmOverwrite Option<>
* Change to one CLI argument for overwrites
- move getting runtime version into LocalCallExecutor
* change unwrap() to expect()
* comment
* Remove `check_overwrites`
* Encapsulate checking for overwrites in LocalCallExecutor
* move duplicate code into function
* Update client/cli/src/params/import_params.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* comma
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* cache hash in WasmBlob
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/client.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* move getting overwrite into its own function
* fix error when directory is not a directory
* Error on duplicate WASM runtimes
* better comment, grammar
* docs
* Revert StateBackend back to _
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/call_executor.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Add two tests, fix doc comments
Add a test for the runtime_version method of WasmOverwrite
Add a test for check_overwrite method of LocalCallExecutor
* remove redundant `Return` from expect msg
* Update client/cli/src/params/import_params.rs
Co-authored-by: David <dvdplm@gmail.com>
* Update client/service/src/client/call_executor.rs
Co-authored-by: David <dvdplm@gmail.com>
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: David <dvdplm@gmail.com>
* Update client/service/src/config.rs
Co-authored-by: David <dvdplm@gmail.com>
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: David <dvdplm@gmail.com>
* Add Module Documentation, match on '.wasm' extension
* Add test for scraping WASM blob
* fix expect
* remove creating another block in LocalCallExecutor test
* remove unused import
* add tests for duplicates and scraping wasm
* make tests a bit nicer
* add test for ignoring non-.wasm files
* check error message in test
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove println
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* make tests prettier
* Update client/service/src/client/wasm_overwrite.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* comment for seemingly random client
* locally-built -> custom
* remove unused import
* fix comment
* rename all references to overwrite with override
* fix cli flag in module documentation
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
* Asyncify sign_with
* Asyncify generate/get keys
* Complete BareCryptoStore asyncification
* Cleanup
* Rebase
* Add Proxy
* Inject keystore proxy into extensions
* Implement some methods
* Await on send
* Cleanup
* Send result over the oneshot channel sender
* Process one future at a time
* Fix cargo stuff
* Asyncify sr25519_vrf_sign
* Cherry-pick and fix changes
* Introduce SyncCryptoStore
* SQUASH ME WITH THE first commit
* Implement into SyncCryptoStore
* Implement BareCryptoStore for KeystoreProxyAdapter
* authority-discovery
* AURA
* BABE
* finality-grandpa
* offchain-workers
* benchmarking-cli
* sp_io
* test-utils
* application-crypto
* Extensions and RPC
* Client Service
* bin
* Update cargo.lock
* Implement BareCryptoStore on proxy directly
* Simplify proxy setup
* Fix authority-discover
* Pass async keystore to authority-discovery
* Fix tests
* Use async keystore in authority-discovery
* Rename BareCryptoStore to CryptoStore
* WIP
* Remote mutable borrow in CryptoStore trait
* Implement Keystore with backends
* Remove Proxy implementation
* Fix service builder and keystore user-crates
* Fix tests
* Rework authority-discovery after refactoring
* futures::select!
* Fix multiple mut borrows in authority-discovery
* Merge fixes
* Require sync
* Restore Cargo.lock
* PR feedback - round 1
* Remove Keystore and use LocalKeystore directly
Also renamed KeystoreParams to KeystoreContainer
* Join
* Remove sync requirement
* Fix keystore tests
* Fix tests
* client/authority-discovery: Remove event stream dynamic dispatching
With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.
Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.
* Make it compile
* Fix submit_transaction
* Fix block_on issue
* Use await in async context
* Fix manual seal keystore
* Fix authoring_blocks test
* fix aura authoring_blocks
* Try to fix tests for auth-discovery
* client/authority-discovery: Fix lookup_throttling test
* client/authority-discovery: Fix triggers_dht_get_query test
* Fix epoch_authorship_works
* client/authority-discovery: Remove timing assumption in unit test
* client/authority-discovery: Revert changes to termination test
* PR feedback
* Remove deadcode and mark test code
* Fix test_sync
* Use the correct keyring type
* Return when from_service stream is closed
* Convert SyncCryptoStore to a trait
* Fix line width
* Fix line width - take 2
* Remove unused import
* Fix keystore instantiation
* PR feedback
* Remove KeystoreContainer
* Revert "Remove KeystoreContainer"
This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.
* Take a ref of keystore
* Move keystore to dev-dependencies
* Address some PR feedback
* Missed one
* Pass keystore reference - take 2
* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX
Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:
1. No additional mental overhead of a `SyncCryptoStorePtr`.
2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.
Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.
* Remove SyncCryptoStorePtr
* Remove KeystoreContainer & SyncCryptoStorePtr
* PR feedback
* *: Use CryptoStorePtr whereever possible
* *: Define SyncCryptoStore as a pure extension trait of CryptoStore
* Follow up to SyncCryptoStore extension trait
* Adjust docs for SyncCryptoStore as Ben suggested
* Cleanup unnecessary requirements
* sp-keystore
* Use async_std::task::block_on in keystore
* Fix block_on std requirement
* Update primitives/keystore/src/lib.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Fix wasm build
* Remove unused var
* Fix wasm compilation - take 2
* Revert async-std in keystore
* Fix indent
* Fix version and copyright
* Cleanup feature = "std"
* Auth Discovery: Ignore if from_service is cloed
* Max's suggestion
* Revert async-std usage for block_on
* Address PR feedback
* Fix example offchain worker build
* Address PR feedback
* Update Cargo.lock
* Move unused methods to test helper functions
* Restore accidentally deleted cargo.lock files
* Fix unused imports
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* init_logger: switch from log-based to tracing-based and add compatibility layer
* Move tracing profiling subscriber related config realization
* sp-tracing: change profiling to be a layer instead of a subscriber
* Enable profiling layer in cli
* Change all test env_logger init to sp_tracing::try_init_simple
* Remove all local env_logger dependency
* Add missing tracing-subscriber dependency
* frame-sudo: fix tests
* frame-support: fix tests
* Fix frame/pallet and executor tests
* Fix the remaining tests
* Use subscriber's try_init as recommended by @davidbarsky
* Be explict that the tracing-log feature is needed
* Set subscriber writer to stderr
* Shorter line width
* Update cargo lock tracing version
* Fix sc_tracing crate compile
* Fix sc_authority_discovery crate test
* unremove default-features
* Leave enabled to default true
* Warn if global default cannot be set
* Fix unused import
* Remove unused PROXY_TARGET
* Change all reference from rc5 to rc6
* Change all reference of rc2 to rc6
* Fix styling
* Fix typo
* make logger init error'ing
* re-fixing the test issue
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* service builder: fix todo about jsonrpc Option workaround
* grandpa-rpc: only share executor instead of sub manager
* grandpa-rpc: fix compilation
* grandpa-rpc: rename to subscription_executor
* node/cli: remove another unused jsonrpc dependency
* grandpa: apply style fixes from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* add offchain worker api to set reserved nodes.
* new offchain api to get node public key.
* node public key from converter
* refactor set reserved nodes ocw api.
* new ndoe authorization pallet
* remove unnecessary clone and more.
* more
* tests for node authorization pallet
* remove dependency
* fix build
* more tests.
* refactor
* Update primitives/core/src/offchain/testing.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update frame/node-authorization/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update frame/node-authorization/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update frame/node-authorization/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* format code
* expose NetworkService
* remove NetworkStateInfo in offchain
* replace NodePublicKey with PeerId.
* set max length of peer id.
* clear more
* use BTreeSet for set of peers.
* decode opaque peer id.
* extract NetworkProvider for client offchain.
* use OpaquePeerId in node authorization pallet.
* fix test
* better documentation
* fix test
* doc
* more fix
* Update primitives/core/src/offchain/mod.rs
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update client/offchain/src/api.rs
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* derive serialize and deserialize
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Refactor sc-network/service metrics.
1. Aggregate sc-network metrics into a submodule, introducing
two more sourced metrics to avoid duplicate atomics.
2. Decouple periodic sc-service network metrics from other
metrics, so that they can be updated independently.
* Update client/service/src/metrics.rs
* Update client/service/src/metrics.rs
Arkadiy Paronyan