* Refactor rebase master prometheus_v0.3
* Milestone1: Final Version of v0.3
* no-std or warm compatibility issues, grapana-data -source code reference and correction,applicable
* Cargo.lock paritytech/master rebase
* prometheus networking.rs del, grafana-data-source networking.rs pub edit and note
* chore: reflect various feedback
* Spaces to tabs.
* Replace grafana and tidy
* Add generics
* Add photo back
* Re-fix spaces in primitives/consensus/babe/src/inherents.rs
* Refactor rebase master prometheus_v0.3
* Milestone1: Final Version of v0.3
* no-std or warm compatibility issues, grapana-data -source code reference and correction,applicable
* prometheus networking.rs del, grafana-data-source networking.rs pub edit and note
* chore: reflect various feedback
* Replace grafana and tidy
* Add generics
* Add photo back
* Re-fix spaces in primitives/consensus/babe/src/inherents.rs
* chore: revert this file back to paritytech/master inherents.rs.
* Add newline at EOF
* Tidy
* Use local registry
* fix typo
Co-Authored-By: Max Inden <mail@max-inden.de>
* chore: Apply review feedback
* endpoint -> exporter
* fix readme
* Remove lazy_static, use ServiceMetrics struct instead
* Switch to using GaugeVecs
* chore: without nightly , edit README
* block_height -> block_height_number
* Switch to a ready_transactions_number gauge
* Update utils/prometheus/src/lib.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* no-prometheus flag add
* /metrics url Input check
* remove prometheus in Tracing
* remove prometheus in Tracing
* chore: master code rebase edit
* gitlab-check-web-wasm edit code
* From:from and cargo.lock update
* with_prometheus_registry add background_tasks
* utils/prometheus/src/lib.rs: Restructure #[cfg] for wasm without hyper
Given that Hyper is not compatible with WASM targets it needs to be
excluded from WASM builds. Instead of introducing #[cfg] lines
throughout the crate, this patch splits the crate into two: known_os and
unknown_os (WASM).
* utils/prometheus/src/lib.rs: Feature gate known_os module
* client/cli/src/lib.rs: Re-add newline at end of file
Co-authored-by: JeseonLEE <zeroday26@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Hyungsuk Kang <hskang9@gmail.com>
This prs cleans up some of the frame benchmarking stuff:
- Move CLI into `frame-benchmarking-cli`. No frame related CLI should
exists in the default Substrate CLI.
- Move all traits and types related to frame benchmarking into the
`frame-benchmarking` trait. Frame types should be isolated in Frame.
* Initial inspect.
* WiP
* Add parsing tests.
* Finalize CLI.
* Update to latest substrate.
* Remove unused imports.
* Support ImportParams as well, to get the right pruning setting.
* Mention in docs that hash is no 0x.
* Move bytes above extrinsics.
* Switch to fill helper from sc_cli.
* Remove overwrite.
* Fix error.
* Fix error message.
* Remove extra allow.
* init_config
libp2p-0.16 allows configuring yamux through libp2p-yamux, so the
direct dependency is no longer needed.
While at it we also update to the latest versions of yamux and
nohash-hasher, though the code changes do not depend on it.
* service: Don't depend on tokio-executor
Seems to be a leftover dependency that's not used anymore.
* offchain: Upgrade hyper to 0.13, which uses tokio 0.2
* offchain: Adapt HTTP tests to Tokio 0.2
* network: Don't transitively include tokio 0.2 in WASM
1) We don't specifically depend on Tokio codec impls
2) Conflating features in Cargo means that enabling Tokio runtime
in the native environment will also do so in WASM, where it's
obviously not implemented and causes a compilation error.
* grafana-data-source: Pull hyper/tokio only in native environment
* babe_epochAuthorship
remove test-helpers from sp-keyring, bump spec_version, impl_version
* bump Cargo.lock
* add BabeRPC to node-rpc
* rename to BabeApi, remove err_derive
* pass &ServiceBuilder to with_rpc_extensions callback
* sc-consensus-babe-rpc
* Update client/consensus/babe/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Better docs, code style chanegs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* new line at the end of Cargo.toml
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
* Migrate wasmtime backend to wasmtime-api
* Port to a newer version of wasmtime
* Update to the latest changes.
* Rejig the sandbox module a bit
* Materialze
* Fixes.
* executor wasm_runtime fix
* Refactor everything
* More refactoring
* Even more refactorings
* More cleaning.
* Update to the latest wasmtime
* Reformat
* Renames
* Refactoring and comments.
* Docs
* Rename FunctionExecutor to host.
* Imrpove docs.
* fmt
* Remove panic
* Assert the number of arguments are equal between wasmtime and hostfunc.
* Comment a possible panic if there is no corresponding value variant.
* Check signature of the entrypoint.
* Use git version of wasmtime
* Refine and doc the sandbox code.
* Comment RefCells.
* Update wasmtime to the latest-ish master.
This may solve a problem with segfaults.
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Use full SHA1 hash of wasmtime commit.
* Add a panic message.
* Add some documentation
* Update wasmtime version to include SIGSEGV fix
* Update to crates.io version of wasmtime
* Make it work.
* Move the creation of memory into `InstanceWrapper::new`
* Make `InstanceWrapper` !Send & !Sync
* Avoid using `take_mut`
* Update client/executor/wasmtime/Cargo.toml
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Limit maximum size of memory.
* Rename `init_state` to `with_initialized_state`
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* full block import benchmark
* try rocksdb cache
* add profiling helper
* use random keyring instead of zero caching
* update docs
* add more io stats
* remove last sentence
* add ci job to see
* Update primitives/keyring/src/sr25519.rs
Co-Authored-By: Marcio Diaz <marcio.diaz@gmail.com>
* switch to 100tx-block
* remove ci script
Co-authored-by: Marcio Diaz <marcio@parity.io>
* network: Use "one shot" protocol handler.
Add two new `NetworkBehaviour`s, one handling remote block requests
and another one to handle light client requests (both local and from
remote). The change is motivated by the desire to use multiple
substreams of a single connection for different protocols. To achieve
this, libp2p's `OneShotHandler` is used as a protocol handler in each
behaviour. It will open a fresh substream for the duration of the
request and close it afterwards. For block requests, we currently only
handle incoming requests from remote and tests are missing. For light
client handling we support incoming requests from remote and also
ported a substantial amount of functionality over from
`light_dispatch.rs` (including several tests). However the result lacks
in at least two aspects:
(1) We require external updates w.r.t. the best block per peer and
currently nothing updates this information.
(2) We carry a lot of peer-related state around.
Both aspects could be simplified by externalising peer selection and
just requiring a specific peer ID where the request should be sent to.
We still have to maintain some peer related state due to the way
libp2p's swarm and network behaviour work (e.g. we must make sure to
always issue `NetworkBehaviourAction::SendEvent`s to peers we are
connected to, otherwise the actions die a silent death.
Another change implemented here is the use of protocol buffers as the
encoding for network messages. Certain individual fields of messages
are still SCALE encoded. There has been some discussion about this
in another PR (https://github.com/paritytech/substrate/pull/3452), so
far without resolution.
* Uncomment `Behaviour::light_client_request`.
* Add license headers.
* Add command-line flag to enable yamux flow control.
We never enabled proper flow-control for yamux streams which may cause
stream buffers to exceed their configured limit when the stream
producer outpaces the stream consumer. By switching the window update
mode to on-read, producers will only receive more sending credit when
all data has been consumed from the stream buffer. Using this option
creates backpressure on producers. However depending on the protocol
there is a risk of deadlock, if both endpoints concurrently attempt to
send more data than they have credit for and neither side reads before
finishing their writes. To facilitate proper testing, this PR adds a
command-line flag `use-yamux-flow-control`.
* Replace comment with generic message.
* Add a sub command to generate a node key file in the format required by a substrate node
* Update lock file
* Apply suggestions from code review
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Updates as per code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Make WASM browser thing compile
* Fix
* updated exit-future (github repo)
* Switch to broadcast crate
* Migrate client/cli
* Switch exit-future to modernize branch
* Small changes
* Switch to cargo version and fix fg tests
* fix basic-authorship
* Fix crash on grafana macro
* Fix grafana macro
* Switch node python version
* Disable record_metrics_slice in grafana macro on wasm
* Update client/grafana-data-source/src/lib.rs
* Revert "Update client/grafana-data-source/src/lib.rs"
This reverts commit 888009a8e0b7051bd4bfbbfdb0448bcf2e2aae93.
* Add wasm support for state machine
* Switch to my own libp2p version
* Revert "Switch to my own libp2p version"
This reverts commit ce613871b59264b3165b45c37943e6560240daa7.
* Revert "Add wasm support for state machine"
This reverts commit de7eaa0694d9534fc3b164621737968e9a6a7c5f.
* Add sc-browser
* Squash
* remove sc-browser
* Fix keystore on wasm
* stubs for removed functions to make env compatible with old runtimes
* Add test (that doesn't work)
* Fix build scripts
* Revert basic-authorship due to no panics
* Revert cli/informant
* Revert consensus
* revert offchain
* Update utils/browser/Cargo.toml
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* export console functions
* Add new chainspec
* Fix ws in chain spec
* revert chainspec
* Fix chainspec
* Use an Option<PathBuf> in keystore instead of cfg flags
* Remove crud
* Only use wasm-timer for instant and systemtime
* Remove telemetry changes
* Assuming this is ok
* Add a KeystoreConfig
* Add stubs back in
* Update libp2p
* Revert "Add stubs back in"
This reverts commit 4690cf1882aa0f99f7f00a58c4080c8aa9b77c36.
* Remove commented js again
* Bump kvdb-web version
* Fix cli
* Switch branch on futures-timer
* Fix tests
* Remove sc-client test build in check-web-wasm because there isn't a good way to build futures-timer with wasm-bindgen support in the build
* Remove more things ^^
* Switch branch on futures-timer back
* Put DB io stats behind a cfg flag
* Fix things
* Don't timeout transports on wasm
* Update branch of futures-timer and fix bad merge
* Spawn informant
* Fix network test
* Fix delay resets
* Changes
* Fix tests
* use wasm_timer for transaction pool
* Fixes
* Switch futures-timer to crates
* Only diagnose futures on native
* Fix sc-network-test tests
* Select log level in js
* Fix syncing ;^)
* Allow disabling colours in the informant
* Use OutputFormat enum for informant
* MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master
* Update futures-diagnose
* Revert "MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master"
This reverts commit baa4ffc94fd968b6660a2c17ba8113e06af15548.
* Pass whole chain spec in start_client
* Get Instant::now to work in transaction pool again
* Informant dep reordering
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
* babe: pass epoch data via intermediates
* Switch to use Box<dyn Any> for intermediates
* Set intermediate.epoch to be Option
* Fix proposer should put out an empty intermediate
* Remove unnecessary encode/decode
* Add EpochData to block_import_params in slot worker
* Fix aura compile
* Fix integration test
* Init epoch changes module
* Initial integration of new epoch changes module for BABE
* Fix all initial compile errors
* rename: digest -> digests
* Fix babe tests
* Bump impl_version
* Fix more test issues
* Remove test flag for tree
It unfortunately won't work for multiple crates.
* Update cargo lock
* Fix duplicate parking_lot version
* Add missing license header
These are a few changes I missed during the refactoring.
1. Initialization issue and boilerplate
Most importantly: part of the `Configuration` initialization was done in `sc_cli::init`. This means the user can not benefit from this initialization boilerplate if they have multiple `Configuration` since `sc_cli::init` can only be called once.
2. Boilerplate for `VersionInfo` and `Configuration`
I'm also answering to the critic of @bkchr on the initialization using version: https://github.com/paritytech/substrate/pull/4692/files/bea809d4c14a2ede953227ac885e3b3f9771c548#r372047238 This will allow initializing a `Configuration` and provide the version by default.
3. Loading the `chain_spec` explicitly
In the past it was done automatically but in some cases we want to delay this. I moved the code to `Configuration.load_spec()` so it can be called later on. `chain_spec` can also be written directly to the `Configuration` without using this `load_spec` helper.
4. [deleted]
5. Fixing issue that prevents the user to override the port
In the refactoring I introduced a bug by mistake that could potentially prevent the CLI user to override the ports if defaults where provided for these ports (only on cumulus).
6. Change task_executor from Box to Arc
This is useful for cumulus where we have 2 nodes with 2 separate Configuration that need to spawn tasks to the same runtime.
7. Renamed TasksExecutorRequired to TaskExecutor
For consistency.
This is related to https://github.com/paritytech/cumulus/issues/24
This is the continuation (and hopefully the end of) #4692
* Restructure node-template so it is clear that node, runtime, and pallets are separated
* Separating to mock and tests
* restructuring runtime to top-level
* updated release script
* updated Cargo.lock
* Initially scoping out of the problem
* Remove need for exiry in balance locks.
* Remove expiry from locks.
* Remove supefluous balance test
* Amalgamate pieces of balance module
* Split out vesting
* Fix tests
* Fixes for vesting.
* Docs.
* Weight docs.
* Refactor things in terms of set_balances.
* Switch out ED to be free + reserved.
* Remove on_free_balance_zero and some docs.
* Build fixes
* Update frame/vesting/src/lib.rs
Co-Authored-By: Xiliang Chen <xlchen1291@gmail.com>
* Update frame/vesting/src/lib.rs
Co-Authored-By: Xiliang Chen <xlchen1291@gmail.com>
* Migration
* Remove superfluous code.
* Test fixes
* Fix some tests
* Fix repatriate reserve
* Fixes
* Add test for migration
* Final cleanups
* Fix
* Indentation.
* Undo unneeded referencing
* Bump runtime version
* Fixes
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
* client/finality-grandpa: Add regression test observer polling network
Ensure `Future` implementation of `ObserverWork` is polling its
`NetworkBridge`. Regression test for bug introduced in d9837d7dd and
fixed in 504b4e89e.
When polled, `NetworkBridge` forwards reputation change requests from
the `GossipValidator` to the underlying `dyn Network`. This test
triggers a reputation change by calling `GossipValidator::validate` with
an invalid gossip message. After polling the `ObserverWork` which should
poll the `NetworkBridge`, the reputation change should be forwarded to
the test network.
* Nits
Co-authored-by: Max Inden <mail@max-inden.de>
* Expose information about the extrinsic in the metadata
This pr exposes some information about the extrinsic used in the runtime
via metadata. The following information are exposed:
- Version of the extrinsic
- List of all signed extensions used by the extrinsic.
* Increment `spec_version`
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).
This touches a few important things:
- the startup of the async task with tokei:
This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
- increased the version of structopt and tokei
- no more use of structopt internal's API
- less use of generics
Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want
Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
Gavin Wood