* Disable log reloading by default
This disables the log reloading that was enabled by default. The problem
is that the log reload implementation of `tracing` is using a lock to
make the layer replaceable. This lock needs to be locked every time we
need to check if a particular target is enabled (assuming the log level
is high enough). This kills the performance when for example
`sometarget=trace` logging is enabled.
* 🤦
* Remove unused parameter
* Fix test
* Fix
* Clean up sc-network
- Avoid using clone() for the Copy type `PeerId`.
- Use `find_map` for `filter_map` and `next`.
- Use `Self`.
* More on Copy types
* Cargo +nightly fmt --all
* More ..
* fmt
* Revert vec![default_notif_handshake_message]
* sp-utils => sc-utils
* cargo fmt
* These files are now in the client so should be licensed as GPL3
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Move alloc primitive (not used in /pallets)
* Move to alternative location as not shared
* moved crates to different dir
* ren sp_chain_spec to sc_chain_spec_primatives
* merged sc-chain-spec and moved allocation up one.
* no no_std
* nudge
* Bump CI
* Add filter reload handle
* add RPC, move logging module from cli to tracing
* remove dup fn
* working example
* Update client/rpc-api/src/system/mod.rs
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Prefer "set" to "reload"
* Re-enable the commented out features of the logger
* Remove duplicate code
* cleanup
* unneeded lvar
* Bump to latest patch release
* Add new CLI option to disable log filter reloading,
Move profiling CLI options to SharedParams
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Applied suggestions from reviews
* Fix calls to init_logger()
* Handle errors when parsing logging directives
* Deny `system_setLogFilter` RPC by default
* One more time
* Don't ignore parse errors for log directives set via CLI or RPC
* Improve docs
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/cli/src/config.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* fix merge errors
* include default directives with system_setLogFilter RPC,
implement system_rawSetLogFilter RPC to exclude defaults
* docs etc...
* update test
* refactor: rename fn
* Add a test for system_set_log_filter – NOTE: the code should likely change to return an error when bad directives are passed
* Update client/cli/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Address review grumbles
* Add doc note on panicking behaviour
* print all invalid directives before panic
* change RPCs to: addLogFilter and resetLogFilter
* make CLI log directives default
* add comments
* restore previous behaviour to panic when hard-coded directives are invalid
* change/refactor directive parsing
* fix line width
* add test for log filter reloading
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* finish up suggestions from code review
* improve test
* change expect message
* change fn name
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* add docs, remove unused fn
* propagate Err on invalid log directive
* Update tracing-subscriber version
* Improve docs for `disable_log_reloading` CLI param
* WIP implementation: RPC and trace capturing
* WIP
* fix incorrect number of digest items
* return errors
* add From impl for Values, rename structs
* fixes
* implement option to choose targets for traces
* rename fn
* fix EnvFilter and add root span
* fix root span
* add docs, remove unnecessary traits
* fix regression on parent_id introduced in a9c73113a8
* fix line width
* remove unused
* include block hash, parent hash & targets in response
* move types from sp-tracing into sp-rpc
move block and parent hash into root of BlockTrace
* switch from log::trace to tracing::trace in state-machine
* use unsigned integer type to represent Ext::id in traces
* ensure id is unique by implementing Subscriber
tracing_subscriber::FmtSubscriber does not guarantee
unique ids
* indentation
* fix typo
* update types
* add sp_io::storage events
* Change response format
- update types
- record distinct timestamps
- sort spans by first entered
* convert to HexDisplay, refactor
* Sort out fallout from merge
* Update client/rpc-api/src/state/mod.rs
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Exit early unless the node runs with --rpc-methods=Unsafe
* Better error handling
* Use wasm-timer
* revert trace alteration in `state-machine` and remove events in `sp_io::storage`
Resolve in follow-up PR
* Review feedback: less collects
* Without Arcs
* Fix span exit
* typo
* cleanup
* Add a few debug messages to tracing module
* Structure traces state-machine/ext; Dispatchable extrinsics spans not working
* Correctly encode Option storage values
* Remove test field for Put and Get
* Try out some changes to dispatch macro
* Add various log messages in dispatch
* Add span dispatch span to new proc macro
* Remove debug messages in dispatch
* Trivial clean up
* Structure remaining state-machine traces (ChangesRoot*)
* Removed unnesecary tracing targets
* Remove log
* New cargo.lock post merge
* Add logging for wasm_overrides
* remove temp logs
* remove temp logs
* remove unused dep
* remove temp logs
* add logging to wasm_overrides
* add logging to state_tracing
* add logging for spans to substrate (includes timings)
* Skip serializing some event fields; Remove most storage traces
* Bring back all ext.rs traces
* Do not skip bool values in events
* Skip serializing span values
* Serialize span values; remove some trace events in ext
* Remove more trace events
* Delete commented out traces
* Remove all unused traces
* Add event filtering
* Fix typo
* wip - change response types to be more efficient
missing import
type
* Serialize struct fields as camelCase
* Add back in event filtering
* Remove name field from event
* Sort spans by time entered
* Sort spans in ASCending order
* Add storage keys target param to rpc
* Limit payload size; improve hash fields; include storage keys
- cleanup event_key_filter
- better block hash representation
- limit payload size
- cleanup based on andrews comments
* Error when serialized payload is to big
* Import MAX_PAYLOAD from rpc-servers
* Clean up ext.rs
* Misc. cleaning and comments
* Strict ordering span Id; no span sort; adjust for rpc base payload
* Add RPC docs to rpc-api/src/state/mod
* Make params bullet points
* Update primitives/rpc/src/tracing.rs
* Put all tracing logic within trace
* Remove attr.record in new_span
* Add back value record in new_span
* restore result collection in ext
* Revert "Add back value record in new_span"
This reverts commit baf1a735f23e5eef1bf6264adfabb788910fa661.
* 🤦
* more 🤦
* Update docs; Try fix line width issues
* Improve docs
* Improve docs
* Remove default key filters + add key recs to docs
* Try restore old traces
* Add back doc comment
* Clean up newlines in ext.rs
* More new line remova;
l
* Use FxHashMap
* Try use EnvFilter directives for event filtering
* Remove directive, filter events by fields
* Use trace metadata correctly
* Try EnvFilter directive with all default targets
* Revert "Try EnvFilter directive with all default targets"
This reverts commit 4cc6ebc721d207c3b846444174f89d45038525ac.
* Clean up clippy warning
* Incorporate Niklas feedback
* Update trace/log macro calls to have better syntx
* Use Ordering::Relaxed
* Improve patch and filter doc comment
* Clean up `BlockSubscriber::new`
* Try optimize `BlockSubscriber::enabled`
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Use contains_key
* use heuristic for payload size
* Add error tupe for client::tracing::block
* Minor tweaks
* Make a note about `--features with-tracing`
* Add CURL example to RPC docs
* Link to substrate-archibe wasm
* Trivial doc clean up based on David feedback
* Explicit result type name
* Respect line length
* Use the error
* Don't print timings when spans close
* Fix failing sc-rpc-api
* Update sp-tracing inner-line doc
* Update client/tracing/src/block/mod.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/call_executor.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/call_executor.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/tracing/src/block/mod.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/tracing/src/block/mod.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Address some review grumbles
* Update primitives/state-machine/src/ext.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Use result_encoded structure fields in ext.rs
* Use value key for ext put
* Add notes about tracing key names matter
Co-authored-by: Matt <mattrutherford@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Before this pr changing the log directives would not change the max log
level. This means that if the node was started with `info` logging and
some `trace` logging was enabled, this `trace` wouldn't be logged. To
fix this we also need to update the max log level. This max log level is
used by the log macros to early return.
* Add boilerplate for JSON-RPC layer for reserved nodes
* Add more boilerplate for JSON-RPC layer for reserved nodes
* Make JSON-RPC layer for reserved nodes async
* Use more realistic data in reserver_peers tests
* Make JSON-RPC layer for reserved nodes blocking
* Apply tomaka's suggestion to reduce .into_iter() for an iter
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Refactor sc-network/service metrics.
1. Aggregate sc-network metrics into a submodule, introducing
two more sourced metrics to avoid duplicate atomics.
2. Decouple periodic sc-service network metrics from other
metrics, so that they can be updated independently.
* Update client/service/src/metrics.rs
* Update client/service/src/metrics.rs
* Update to libp2p-0.23.
Thereby incorporate bandwidth measurement along the
lines previously done by libp2p itself.
* Tweak dependencies for wasm32 compilation.
For wasm32 we need to enable unstable features to
make `task::Builder::local` available.
* Simplify dependencies.
* Simplify.
Leave the calculation of bytes sent/received per second
to the outer layers of the code, subject to their own
individual update intervals.
* Cleanup
* Re-add lost dev dependency.
* Avoid division by zero.
* Remove redundant metric.
* Enable sending of noise legacy handshakes.
* Add comment about monotonic gauge.
* CI
* Add new RPC method to get the chain type
This adds a new RPC method to get the chain type of the running chain.
The chain type needs to be specified in the chain spec. This should make
it easier for tools/UI to display extra information without needing to
rely on parsing the chain name.
* Update client/rpc-api/src/system/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Primitive crate
* Feedback
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:
- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)
refs #4679
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
* Split the Roles bitfield in three
* Forgot to include some changes
* Fix cli test
* More test fixes
* Oh God, merging master broke other tests
* Didn't run the doctests
* Address review
* I'm trying to fix the build blindly because it's taking a good hour to compile on my machine
* Address some review
* Also update the peerset's API to make sense
* Fix peerset tests
* Fix browser node
* client: distinguish between local and network authority
Co-authored-by: André Silva <andre.beat@gmail.com>
* RPC forwarders for adding reserved peers
* implement service side of reserved-peer RPCs
* Clean up unnecessary format! invocation
Co-Authored-By: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* add some tests for the new RPC methods
* remove redundant `data` field
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Adding script for rename, could be applicable for nodes on top of it, too
* add stderr and gitlab ci features
* apply script
* fix now minor details in expected stderr
* Update the Cargo.lock
* fix name: sc-transaction -> sc-tracing
* fix rename in script, too
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Squirrel