Commit Graph

389 Commits

Author SHA1 Message Date
Seun Lanlege 3193b0d4aa removes use of sc_client::Client from node-transaction-factory (#5158)
* removes use of sc_client::Client from node-transaction-factory

* move sc-block-builder to [dev-dependencies] in substrate-test-runtime
2020-03-12 19:19:38 +01:00
Seun Lanlege 9898fc7e81 removes use of sc_client::Client from sc_network (#5147)
* removes use of sc_client::Client from sc_network

* rename BlockProvider to BlockBackend

* fix broken test
2020-03-12 14:02:04 +01:00
Stanislav Tkach 6e1b4cd560 Update sysinfo version to 0.11 (#5216) 2020-03-12 14:00:07 +01:00
Arkadiy Paronyan dc0bf210fb ChainSpec trait (#5185)
* ChainSpec trait

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added docs

* Fixed build

* Fixed build

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-12 00:00:10 +01:00
Max Inden d2345e8d5c client/authority-discovery: Instrument code with Prometheus (#5195)
* client/authority-discovery: Instrument code with Prometheus

Introduce Prometheus metrics into the authority discovery module
enabling one to observe:

- authority_discovery_times_published_total

- authority_discovery_amount_external_addresses_last_published

- authority_discovery_times_requested_total

- authority_discovery_dht_event_received

* client/authority-discovery/src/lib.rs: Reword metric help texts

Co-Authored-By: Ashley <ashley.ruglys@gmail.com>

* client/authority-discovery/src/lib.rs: Reword metric help text

Co-Authored-By: Ashley <ashley.ruglys@gmail.com>

* client/authority-discovery/tests: Fix struct instantiation + basic test

Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2020-03-11 23:59:33 +01:00
Stanislav Tkach 93ee3104e7 Use CLI to configure max instances cache (#5177)
* Use CLI to configure max instances cache

* Fix tests

* Move default value into CLI

* Use SmallVec

* Apply review comments

* Get rid of `SmallVec`

Co-authored-by: Bastian Köcher <git@kchr.de>
2020-03-11 09:43:37 +01:00
Pierre Krieger f21680209f Minor fixes for Android compilation (#5209)
* Minor fixes for Android

* Update parity-multiaddr

* Bump in service Cargo.toml too
2020-03-10 20:54:58 +01:00
Bastian Köcher 1cfcf5cbfe Don't include :code by default in storage proofs (#5179)
* Don't include `:code` by default in storage proofs (#5060)

* Adds test to verify that the runtime currently is always contained in
the proof

* Start passing the runtime wasm code from the outside

* Fix compilation

* More build fixes

* Make the test work as expected now :)

* Last fixes

* Fixes benchmarks

* Review feedback

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Review feedback

* Fix compilation

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>

* Fix compilation and change the way `RuntimeCode` works

* Fix tests

* Switch to `Cow`

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
2020-03-10 11:13:20 +01:00
Max Inden 0cc3b96076 client/service/src/builder.rs: Add build_info metric (#5192)
* client/service/src/builder.rs: Add build_info metric

Add static Prometheus metric exposing the chain name, the version and
the commit.

* client/service/src/builder.rs: Move node_role to static metrics

The Prometheus metrics `node_role` is static and thus there is no need
to keep a reference of it within `ServiceMetrics`. This follows the
example of the `build_info` metric.

* client/service/src/builder.rs: Adjust indentation
2020-03-10 11:12:32 +01:00
Igor Matuszewski 3d73dd1c3b Make sc-offchain tests more resilient on macOS (#5191)
* offchain: Simplify bits of http code

* offchain: Sort dev-dependencies

* deps: Bump parity-multiaddr to 0.7.3

Fixes build failure when using:
rustc 1.43.0-nightly (96bb8b31c 2020-03-05).

* offchain: Raise FD limit for HTTP tests

* offchain: Reword the comment on increasing the test fd limit
2020-03-10 10:45:10 +01:00
André Silva cfd6824e57 grandpa: improve handling of global gossip messages (#5133)
* grandpa: only gossip commits to peers on the same set

* grandpa: track commits uniquely by round and set

* grandpa: fix communication test

* grandpa: add tests for commit gossip handling

* grandpa: add missing docs
2020-03-09 16:46:03 -07:00
Max Inden e8f3c6a686 client/network-gossip: Merge GossipEngine and GossipEngineInner (#5042)
* client/network-gossip: Merge GossipEngine and GossipEngineInner

Given that GossipEngine and GossipEngineInner are not shared between
threads anyone (public interface + background tasks), neither depends on
being Send or Sync. Thus one can merge the two as done in this patch.
One only needs to wrap an `Arc<Mutex<>>` around the whole structure when
the owner (e.g. finality-grandpa) needs to share the gossip engine
between threads.

* client/finality-grandpa: Wrap GossipEngine in Arc Mutex & lock it on use

GossipEngine in itself has no need to be Send and Sync, given that it
does not rely on separately spawned background tasks anymore. Given that
finality-grandpa shares the `NetworkBridge` potentially between threads
its components need to be clonable, thus this patch wraps `GossipEngine`
in an `Arc<Mutex<>>`.
2020-03-09 16:31:29 +01:00
Pierre Krieger 9c6d53f436 Fix NotificationStreamClosed reported when it shouldn't (#5160)
* Fix NotificationStreamClosed reported when it shouldn't

* Fix test

* Add test

* Update client/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Oops, fix test

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-06 21:24:51 +01:00
Nikolay Volf 4aac39c667 Fix possible corner case in revalidation (#5172)
* remove all from members

* add test
2020-03-06 21:24:39 +01:00
Ashley 7177d2fb2b Fix browser node (#5166) 2020-03-06 18:34:53 +01:00
André Silva 9eaa972c21 client: fix notification sinks leak during initial sync (#5161)
* client: fix notification sinks leak during initial sync

* client: add test for notification sink cleanup

* Make it compile

* Further cleanup

* client: fix test for notification sinks cleanup

Co-authored-by: Bastian Köcher <git@kchr.de>
2020-03-06 18:32:50 +01:00
Nikolay Volf 5e1667c406 Add state/database caches to prometheus (#5165)
* add state/database caches to prometheus

* also state-db memory

* use suggestions from review
2020-03-06 18:32:33 +01:00
André Silva 3a792a55cd client: don't log on 0 gossip messages (#5173) 2020-03-06 18:31:35 +01:00
Bastian Köcher c244b1d036 Revert "Build block without checking signatures (#4916)" (#5159)
* Revert "Build block without checking signatures (#4916)"

This reverts commit e50f610907.

* Some further clean ups
2020-03-06 15:27:59 +01:00
Gavin Wood bbaf96e047 Revert quote fixed version and upgrade failure (#5152)
* Revert "use fixed quote (#5135)"

This reverts commit 8bf855b09d.

* Upgrade failure version

* Update frame/staking/reward-curve/Cargo.toml

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Ahh I'm dumb

Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-03-05 22:14:55 +01:00
Gavin Wood d957220435 Add lots of networking metrics for Prometheus (#5126)
* Add some metrics

* Address concerns
2020-03-05 21:38:41 +01:00
Gavin Wood a8441ff968 Remove Backend::destroy_state (#5068)
* Remove `Backend::destroy_state`

This removes the `destroy_state` function of `Backend` and instead moves
the functionality into the `Drop` implementation of the state. This
makes it much easier to work with the state, as the user no longer needs
to call` destroy_state` manually. However, it requires that we switch
from `RwLock` to `ReentrantMutex` as while importing a block we maybe
need to lock again in `drop`.

* Bring back the `RwLock` and some other clean ups

* Fix compilation
2020-03-05 17:01:52 +01:00
Gavin Wood 26fc97f635 Remove substrate-ui.parity.io from CORS whitelist (#5142)
The only up-to-date deployment of PolkadotJS apps is currently on https://polkadot.js.org/apps/.
https://substrate-ui.parity.io is only useful as deliberately kept outdated version of Apps, to be used for projects still on Substrate 1.0
It cannot be used with the up-to-date chains running the Substrate 2.0 codebase, and probably won't be maintained much longer.
2020-03-05 17:01:36 +01:00
Gavin Wood 868d8b8252 Add more prometheus metrics to network::Protocol. (#5145) 2020-03-05 17:01:24 +01:00
Gavin Wood f5dc69b404 Don't remove invalid transactions when skipping. (#5121)
* Don't remove invalid transactions when skipping.

* Use a special kind of extrinsic instead of arbitrary limit.

* Fix txpool tests.

* Attempt to create more blocks.

* Bump lock

Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-03-05 17:00:31 +01:00
Benjamin Kampmann 99ae5342eb removes use of sc_client::Client from sc-rpc (#5063)
* removes use of sc_client::Client from sc-rpc

* remove Client impl from sc-finality-benches

* remove client impl from sc-finality-grandpa

* read_proof accepts iterator

* remove generic Executor param from ExecutorProvider

* fix long ass line

* code style changes

* merge with master

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
2020-03-05 16:41:10 +01:00
Benjamin Kampmann d3208aa7bc Better wasm instance cache (#5109)
* Wasm instance cache

* Reduce slot locking

* Fixed test

* Dispose of instance in case of error

* Fixed benches

* Style, comments, some renames

* Replaced upgradable lock with mutex

* Bump dependencies

* Re-export CallInWasm

* Update client/executor/src/wasm_runtime.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/native_executor.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/native_executor.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/wasm_runtime.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/wasmtime/src/runtime.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/wasm_runtime.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/wasm_runtime.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/wasm_runtime.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Indents

* Whitespace

* Formatting

* Added issue link

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-05 14:02:04 +01:00
Gavin Wood 906e08ee2a Pass Prometheus Registry into Client (#5120)
* Add a few metrics to Client

* Improve PrometheusConfig

* Fix client docs
2020-03-05 13:40:33 +01:00
Gavin Wood ab58a08a53 remove on_exit from grandpa (#5098) 2020-03-05 13:38:16 +01:00
Benjamin Kampmann 0f2606ccaf Use 128mb for db cache default (#5134) 2020-03-05 11:25:28 +01:00
Arkadiy Paronyan 9a1b3b53f2 Revert "Don't include :code by default in storage proofs (#5060)" (#5136)
This reverts commit 6ee39261c8.
2020-03-05 10:55:05 +01:00
Gavin Wood 8bf855b09d use fixed quote (#5135) 2020-03-05 10:29:38 +01:00
Gavin Wood 5bda7c5072 Task manager for background/async tasks in service (#5092)
* Reorganize tasks into task manager

* move to separate file and improve api

* address api issues

* fix spawning inside closures

* decouple executor

* tasks_setup -> tasks_builder

* remove drops

* add deprecatiion comment

* add pub(super)

* fix identation
2020-03-05 09:29:12 +01:00
Benjamin Kampmann 5a33228ea9 Reduce usage of Blake2Hasher (#5132)
This reduces the usage of `Blake2Hasher` in the code base and replaces
it with `BlakeTwo256`. The most important change is the removal of the
custom extern function for `Blake2Hasher`. The runtime `Hash` trait is
now also simplified and directly requires that the implementing type
implements `Hashable`.
2020-03-05 08:51:03 +01:00
Benjamin Kampmann 6ee39261c8 Don't include :code by default in storage proofs (#5060)
* Adds test to verify that the runtime currently is always contained in
the proof

* Start passing the runtime wasm code from the outside

* Fix compilation

* More build fixes

* Make the test work as expected now :)

* Last fixes

* Fixes benchmarks

* Review feedback

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Review feedback

* Fix compilation

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
2020-03-04 20:26:16 +01:00
Benjamin Kampmann 67837c6233 Forget peerset manager nodes after 5mn (#5108)
* Forget peerstore nodes after 5mn

* Bump to one hour

* Also bump on rediscover
2020-03-04 20:24:17 +01:00
Benjamin Kampmann 6cf944cf21 Drop db-cache default from 1gig to 32mb (#5128) 2020-03-04 17:22:02 +01:00
Shawn Tabrizi 3d6329a22d A Pallet for Benchmarking Common Runtime Operations (#4902)
* Benchmark pallet

* Add a bunch more benchmarks

* do nothing test

* new benchmarks

* Clean up extra tests

* Encode and Decode Vec<T::AccountId>

* Starting to migrate benchmarks to macro

* Use macro

* Remove call and storage

* Update Cargo.toml

* Add storage recalc benchmark

* Add support for custom functions in benchmark! macro

* Reset DB for storage recalc

* Feedback from review

* Add more comments

* Remove benchmark pallet from node

* Fix cargo files

* Fix comments

* Change `crate` to `super`

* missed one

* Use results of benchmark encode/decode

* Pass generic to extra functions

* reset macro to master

* Update lib.rs

* Update to use standard syntax
2020-03-04 17:21:42 +01:00
Jakub Pánik e366b5bd2c Added subscribe_all_heads RPC function (#4979)
* Added subscribe_all_heads RPC function

* Update client/rpc/src/chain/tests.rs

Fixed style ( spacing )

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-03-04 14:19:38 +01:00
Bastian Köcher a1b53280b4 Update to SCALE 1.2.0 (#5113)
This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.
2020-03-03 14:40:02 +01:00
Ashley 883ddfc897 Add more metrics to prometheus (#5034)
* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate
2020-03-03 11:36:58 +01:00
Bastian Köcher 4f852d6922 Expose state-db memory info (#5110)
This exposes memory statistics from the state-db.
2020-03-03 10:24:26 +01:00
Bastian Köcher d3244b728a Make sure we remove a peer on disconnect in gossip (#5104)
* Make sure we remove peers on disconnect in gossip state machine

* Clear up the code

* Add a comment
2020-03-02 18:20:04 +01:00
Pierre Krieger 9ab9134ea5 Remove request ID from the new protocol (#5049) 2020-03-02 13:43:29 +01:00
Pierre Krieger b19ea34773 Update to libp2p 0.16.2 (#5088) 2020-03-02 11:00:56 +01:00
pscott d7e4aa41b9 Make export blocks default to json on stdout (#5090)
* Make export blocks default to json on stdout

* Multiline instead of single line to stay under 100 cols

* Change --json flag to --binary, defaulting to json
2020-03-02 10:59:39 +01:00
Ashley 9a925faf7d Remove more instances of futures01 (#4633)
* Start removing last few instances of futures01

* Use to_poll on wasm

* Revert "Use to_poll on wasm"

This reverts commit 1c61728f10d520df5f9b28c415a0db68e478b9c7.

* Fix fg test

* Upgrade network test futures

* Update offchain hyper version

* Update service test

* bump tokio to 0.2.10

* Removed some unneeded tokios

* fixes

* fix run_until_all_full

* Make service test debuggable

* Update client/offchain/src/api/http.rs

Co-Authored-By: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>

* Add service_test to test-int output

* nitpicking

* Finally fix test

* Give up and revert client/serviec/test

* Revert gitlab ci too

Co-authored-by: Demi Obenour <demi@parity.io>
2020-02-28 17:02:33 +01:00
Benjamin Kampmann 25d1b7878a prepping for Alpha.3 (#5080)
* Bump to alpha.3

* update gitlab-ci
2020-02-27 22:06:08 +01:00
Max Inden 9d49d917f6 *: Rename prometheus-exporter crate to substrate-prometheus-end… (#5076)
This patch renames the crate for the following two reasons:

1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".

2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.
2020-02-27 17:48:26 +01:00
Seun Lanlege f26f703ad4 removes use of sc_client::Client from sc_basic_authorship (#5050)
* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path
2020-02-27 17:12:00 +01:00