This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:
- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)
refs #4679
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
The signal handler will be reset to the default system one. This means
the program will be exited instead of panicking. This is required to
support piping in the console.
* switching to released wasmtime
* update depdencies in general
* Update fdlimit
* Update parity common deps
* Also update schnorrkel
* update kvdb-rocksdb
* update further dependents
* also update primitive types
* update cargo.lock
* update merlin
* Bumping evm version
* Version bump
* Split generate_changelog.sh into separate script
Can be run in the format `generate_changelog.sh $previous_version $version`.
* remove early exit from publish_draft_release.sh
* adding changelog
* ci: change last_github_release to also find pre-releases
Co-authored-by: Martin Pugh <pugh@s3kr.it>
This patch renames the crate for the following two reasons:
1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".
2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.
* setting first batch of descriptions
* fix what I just broke
* next batch
* and pallets, too
* last batch
* set cargo.lock
* keep'em dev-deps
* bump version to alpha.2
* setting versions to development pre-release
fixing version in dependencies
* unset already released wasm-builder
* do not publish test crates
* adding licenses
* setting homepage metadata
* set repository url
* Refactor rebase master prometheus_v0.3
* Milestone1: Final Version of v0.3
* no-std or warm compatibility issues, grapana-data -source code reference and correction,applicable
* Cargo.lock paritytech/master rebase
* prometheus networking.rs del, grafana-data-source networking.rs pub edit and note
* chore: reflect various feedback
* Spaces to tabs.
* Replace grafana and tidy
* Add generics
* Add photo back
* Re-fix spaces in primitives/consensus/babe/src/inherents.rs
* Refactor rebase master prometheus_v0.3
* Milestone1: Final Version of v0.3
* no-std or warm compatibility issues, grapana-data -source code reference and correction,applicable
* prometheus networking.rs del, grafana-data-source networking.rs pub edit and note
* chore: reflect various feedback
* Replace grafana and tidy
* Add generics
* Add photo back
* Re-fix spaces in primitives/consensus/babe/src/inherents.rs
* chore: revert this file back to paritytech/master inherents.rs.
* Add newline at EOF
* Tidy
* Use local registry
* fix typo
Co-Authored-By: Max Inden <mail@max-inden.de>
* chore: Apply review feedback
* endpoint -> exporter
* fix readme
* Remove lazy_static, use ServiceMetrics struct instead
* Switch to using GaugeVecs
* chore: without nightly , edit README
* block_height -> block_height_number
* Switch to a ready_transactions_number gauge
* Update utils/prometheus/src/lib.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* no-prometheus flag add
* /metrics url Input check
* remove prometheus in Tracing
* remove prometheus in Tracing
* chore: master code rebase edit
* gitlab-check-web-wasm edit code
* From:from and cargo.lock update
* with_prometheus_registry add background_tasks
* utils/prometheus/src/lib.rs: Restructure #[cfg] for wasm without hyper
Given that Hyper is not compatible with WASM targets it needs to be
excluded from WASM builds. Instead of introducing #[cfg] lines
throughout the crate, this patch splits the crate into two: known_os and
unknown_os (WASM).
* utils/prometheus/src/lib.rs: Feature gate known_os module
* client/cli/src/lib.rs: Re-add newline at end of file
Co-authored-by: JeseonLEE <zeroday26@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Hyungsuk Kang <hskang9@gmail.com>
* Make WASM browser thing compile
* Fix
* updated exit-future (github repo)
* Switch to broadcast crate
* Migrate client/cli
* Switch exit-future to modernize branch
* Small changes
* Switch to cargo version and fix fg tests
* fix basic-authorship
* Fix crash on grafana macro
* Fix grafana macro
* Switch node python version
* Disable record_metrics_slice in grafana macro on wasm
* Update client/grafana-data-source/src/lib.rs
* Revert "Update client/grafana-data-source/src/lib.rs"
This reverts commit 888009a8e0b7051bd4bfbbfdb0448bcf2e2aae93.
* Add wasm support for state machine
* Switch to my own libp2p version
* Revert "Switch to my own libp2p version"
This reverts commit ce613871b59264b3165b45c37943e6560240daa7.
* Revert "Add wasm support for state machine"
This reverts commit de7eaa0694d9534fc3b164621737968e9a6a7c5f.
* Add sc-browser
* Squash
* remove sc-browser
* Fix keystore on wasm
* stubs for removed functions to make env compatible with old runtimes
* Add test (that doesn't work)
* Fix build scripts
* Revert basic-authorship due to no panics
* Revert cli/informant
* Revert consensus
* revert offchain
* Update utils/browser/Cargo.toml
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* export console functions
* Add new chainspec
* Fix ws in chain spec
* revert chainspec
* Fix chainspec
* Use an Option<PathBuf> in keystore instead of cfg flags
* Remove crud
* Only use wasm-timer for instant and systemtime
* Remove telemetry changes
* Assuming this is ok
* Add a KeystoreConfig
* Add stubs back in
* Update libp2p
* Revert "Add stubs back in"
This reverts commit 4690cf1882aa0f99f7f00a58c4080c8aa9b77c36.
* Remove commented js again
* Bump kvdb-web version
* Fix cli
* Switch branch on futures-timer
* Fix tests
* Remove sc-client test build in check-web-wasm because there isn't a good way to build futures-timer with wasm-bindgen support in the build
* Remove more things ^^
* Switch branch on futures-timer back
* Put DB io stats behind a cfg flag
* Fix things
* Don't timeout transports on wasm
* Update branch of futures-timer and fix bad merge
* Spawn informant
* Fix network test
* Fix delay resets
* Changes
* Fix tests
* use wasm_timer for transaction pool
* Fixes
* Switch futures-timer to crates
* Only diagnose futures on native
* Fix sc-network-test tests
* Select log level in js
* Fix syncing ;^)
* Allow disabling colours in the informant
* Use OutputFormat enum for informant
* MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master
* Update futures-diagnose
* Revert "MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master"
This reverts commit baa4ffc94fd968b6660a2c17ba8113e06af15548.
* Pass whole chain spec in start_client
* Get Instant::now to work in transaction pool again
* Informant dep reordering
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).
This touches a few important things:
- the startup of the async task with tokei:
This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
- increased the version of structopt and tokei
- no more use of structopt internal's API
- less use of generics
Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want
Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
* reset a couple of versions down from v0.2.0
* Unstablise browser-utils (very new) and grafana-data-source (going to be replaced with prometheus sometime)
* unmark a bunch more client crates for stabilization
* fix versions in Cargo.tomls
* Downgrade network-test
* Set frame-metadata version to `10.0.0`
* Hide and documented storage generator as internal api
* Downgrade `sp-externalities`
* Downgrade `sc-cli`
* Downgrade sc-executor et al to 0.8.0
* Downgrade sp-state-machine to 0.8.0
* Downgrade pallet-contracts et al to 0.8.0
* Downgrade sp-sandbox to 0.8.0
* downgrade pallet-evm to 0.8.0
* Downgrade pallet-staking to 0.8.0
2.0 should be implementation of lazy payout alongside all the fixes of
current module
* Downgrade node-transaction-factory to 0.8.0
* frame pallets are 2.0.0
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Wei Tang <accounts@that.world>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
* Switch service to futures03
* Fix tests
* Fix service test and cli
* Re-add Executor trait to SpawnTaskHandle
* Fix node-service
* Update babe
* Fix browser node
* Update aura
* Revert back to tokio-executor to fix runtime panic
* Add todo item
* Fix service tests again
* Timeout test futures
* Fix tests
* nits
* Fix service test
* Remove zstd patch
* Re-add futures01 to aura and babe tests as a dev-dep
* Change failing test to tee
* Fix node
* Upgrade tokio
* fix society
* Start switching grandpa to stable futures
* Revert "Start switching grandpa to stable futures"
This reverts commit 9c1976346237637effc07c13f7d0403daf5e71cf.
* Fix utils
* Revert substrate service test
* Revert gitlab
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* Fix cli for structopt 0.3.7 and pin to that version
This is just some hotfix to make everything compile. In the future it
will require another pr to not depend on internals of StructOpt, but
that will probably also require some additions to StructOpt itself. To
not break the code again with another StructOpt, this also pins the
StructOpt version.
* Fix benches
* Fix for fix
* clean up cargo.toml syntax
* bumping versions to 2.0
* bump networking to 0.8
* move consensus down to 0.8
* bump consensus pallets to 0.8.0, too
* Upping babe and aura pallets
* add remaining, missing version definitions
* missed some
* Adding script for rename, could be applicable for nodes on top of it, too
* add stderr and gitlab ci features
* apply script
* fix now minor details in expected stderr
* Update the Cargo.lock
* fix name: sc-transaction -> sc-tracing
* fix rename in script, too
* Move transaction pool to primitives
* move backend, errors into primitives
* remove unused client depencies
* Move rpc-api into primitives
* Move peerset back to client
* Move rpc/api back to client, move palette/support/rpc into utils
* move support-rpc into subfolder
* move system-rpc into utils
* move transaction-pool and -graph back into client
* fix broken imports
* Clean up test primitives
* Make support test utils independent of frame
* remove unnecessary node dependencies from service
* Reactivate dependency script:
- only enforce the now achieved status quo will remain
- allow for primitives to depend on /client for now without failing
- more discriptive error message so people understand, what it wants
- minor fix to differentiative between ../client and /client (which may be a subfolder)
- don't allow this to fail anylonger.
* fix doc comment
* 'Should not' rather than 'must not'.
* Revert unwanted dependency changes
* fix faulty import
* fixup derive_more version
* fix wrong import path
* Implement instrumentation for performance profiling
* Add profiling to runtime functions declared in `decl_module` inc.
`on_initialize` and `on_finalize`
* Remove changes
* add docs, tidy up
* fix versions
* fix copyright date
* switch to hashmap and instant
* update example
* update example
* implement receiver for logger and make default
* fix comment
* use `if_std!` macro
* remove whitespace
* fix whitespace
* fix nits
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Nikolay Volf