Commit Graph

786 Commits

Author SHA1 Message Date
Amar Singh 0d2a794d48 done (#6673) 2020-07-19 16:12:04 +02:00
André Silva ae38a806ed grandpa: report equivocations with unsigned extrinsics (#6656)
* grandpa: use unsigned extrinsics for equivocation reporting

* grandpa: allow signed reports as well

* grandpa: change runtime api for submitting unsigned extrinsics

* grandpa: fix tests

* grandpa: add test for unsigned validation

* grandpa: add benchmark for equivocation proof checking

* offences: fix grandpa benchmark

* grandpa: add proper weight for equivocation reporting extrinsic

* grandpa: fix weight unit
2020-07-17 12:32:20 +02:00
Wei Tang 8ae1aa4c28 Switch to bounded mpsc for txpool import notification stream (#6640)
* Switch to bounded mpsc for txpool import notification stream

* Update client/transaction-pool/graph/src/validated_pool.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-07-17 12:31:47 +02:00
Max Inden 6995e5e5e1 client/service/metrics: Rework metric HELP texts (#6647) 2020-07-16 23:07:32 +00:00
Cecile Tonglet 4ac2078992 The telemetry should not be initialized if no telemetry has been provided (#6666)
* Initial commit

Forked at: e10598af1f
Parent branch: origin/master

* Fix: do not initialize telemetry if telemetry is not set

* Update client/telemetry/src/lib.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update client/telemetry/src/lib.rs

* Update client/telemetry/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-07-16 09:46:40 +00:00
Andronik Ordian e10598af1f sc-offchain: upgrade hyper-rustls to 0.18 (#6663)
* sc-offchain: upgrade hyper-rustls to 0.18

* Cargo.lock: revert rand downgrade due to rust-lang/cargo/issues/5702
2020-07-15 16:41:19 +02:00
Pierre Krieger 4720f0fdda Move the legacy protocol handshake to the legacy substream (#5938)
* Move the legacy protocol handshake to the legacy substream

* Fix tests

* Remove line that wasn't supposed to be committed

* Remove hack

* Rework how it's done

* Some little changes

* update_chain wasn't doing its thing

* Fix service tests not calling update_chain

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* [WIP]

* Revert "[WIP]"

This reverts commit 2b892e6a7637c0b1297e6ecdbb919321c9098ff5.

* Update client/network/src/protocol.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix received message not being handshake

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

Co-authored-by: Max Inden <mail@max-inden.de>
2020-07-15 09:29:10 +00:00
Pierre Krieger 4613dc2af4 Remove legacy Clogged event (#6652) 2020-07-14 15:43:08 +00:00
Bastian Köcher 74655edebc Check that serde is present in sc-chain-spec-derive (#6636)
We should check that `serde` is present when using
`sc-chains-spec-derive`, because we require this for the generated
struct. This also switches the generated `derive` instruction to import
`Serialize`/`Deserialize` by path to not require the user to import
these derive macros.
2020-07-13 10:54:16 +02:00
Pierre Krieger 4f883ee9f8 Remove warning about networking taking a lot of time (#6630)
* Remove warning about networking taking a lot of time

* Fix warnings
2020-07-10 20:39:36 +02:00
Bastian Köcher 64114267b2 Revert "Add log rotation (#6564)" (#6627)
This reverts commit 6eb2eb81c5.
2020-07-10 10:43:41 +00:00
Pierre Krieger 150ef0a40a Deduplicate some dependencies (#6628) 2020-07-10 09:28:10 +00:00
Amar Singh bf4e5d4d2c add ios to cfg gates in client/service (#6622)
* done

* sysinfo should not be not android

* add back sysinfo dep to satisfy web wasm ci
2020-07-09 21:01:49 +02:00
Bastian Köcher 5bb834cc6f Make init_logging more easily usable (#6620)
Instead of requiring the `LogRotationOpt`, it now requires an
`Option<LogRotationOpt>`. This makes it much more easy to use the
interface when someone isn't interested on the `LogRotationOpt`'s
2020-07-09 14:47:57 +00:00
Ashley 234e7d0c3d Simplify a few chain components creation APIs related to the service (#6611)
* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-07-09 15:43:04 +02:00
Cecile Tonglet a4427f3635 Clean exit when no space left on device (#6339)
Fixes #6305
2020-07-09 14:58:29 +02:00
Pierre Krieger ef8572f10d Add an authority_discovery_known_authorities_count metric (#6614) 2020-07-09 12:32:55 +00:00
David Craven 4d209543b9 Fix build when with-kvdb-rocksdb is disabled. (#6546) 2020-07-08 22:46:53 -04:00
Bastian Köcher 94cddee160 Improve transaction submission (#6599)
* Improve transaction submission

Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Make sure we don't submit the same transaction twice from the network concurrently

* Remove added listener call

* Feedback

* Ignore banned on resubmit
2020-07-08 17:42:42 +02:00
Max Inden faa72caf91 *: Update to libp2p v0.21.1 (#6559)
* *Cargo.toml: Update versions

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Adjust to one_shot.rs changes

* client/network/discovery: Log address list on trace level

* client/network/discovery: Ignore RoutablePeer and PendingRoutablePeer

* Commit Cargo.lock

* Finish update

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-07-08 14:00:30 +00:00
Pierre Krieger 8ca05879e8 Send Status message on all newly-opened legacy substreams (#6593)
* Send Status message on all newly-opened legacy substreams

* Fix tests
2020-07-08 11:44:51 +00:00
pscott 6eb2eb81c5 Add log rotation (#6564)
* Use flexi_logger; Add log rotation

* Add default rotation; Add FlexiLogger error

* Fix compilation error

* Remove logging to stdout if it's not a tty

* Fix formatting

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove needless debug statement

* Default to unlimited size for log rotation

* Add more comments about log-age option

* Remove unused variable

* Fix typo in comment

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-08 10:11:09 +00:00
Max Inden 62f306d972 client/network: Rename DebugInfoBehaviour to PeerInfoBehaviour (#6556)
Information retrieved via `DebugInfoBehaviour` is not only used for
debugging purposes, e.g. disconnecting from nodes not responding to
pings, using external addresses retrieved via indentify, ...

In order for the name to reflect the usage of the module this commit
renames it.
2020-07-08 09:08:47 +00:00
Bastian Köcher ce0b55ff09 Upgrade kvdb-*, trie-db and memory-db (#6584)
* Upgrade `kvdb-*`, `trie-db` and `memory-db`

The updates of `trie-db` and `memory-db` are important, as they fix the
non-deterministic build of Polkadot/Substrate.

* Change `trie-db` version

* Update test-utils/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update primitives/trie/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update `Cargo.lock` and `trie-bench`

* Fix UI tests

* Switch to fixed version of memory-db

Co-authored-by: Andronik Ordian <write@reusable.software>
2020-07-08 10:42:56 +02:00
Pierre Krieger 36cf6e7dc4 Use async/await in build_network_future (#6533)
* Use async/await in build_network_future

* Address concerns

* Fix test
2020-07-07 10:32:30 +00:00
André Silva a9c21b8b84 babe: report equivocations (#6362)
* slots: create primitives crate for consensus slots

* offences: add method to check if an offence is unknown

* babe: initial equivocation reporting implementation

* babe: organize imports

* babe: working equivocation reporting

* babe: add slot number to equivocation proof

* session: move duplicate traits to session primitives

* babe: move equivocation stuff to its own file

* offences: fix test

* session: don't have primitives depend on frame_support

* babe: use opaque type for key owner proof

* babe: cleanup client equivocation reporting

* babe: cleanup equivocation code in pallet

* babe: allow sending signed equivocation reports

* node: fix compilation

* fix test compilation

* babe: return bool on check_equivocation_proof

* babe: add test for equivocation reporting

* babe: add more tests

* babe: add test for validate unsigned

* babe: take slot number in generate_key_ownership_proof API

* babe: add benchmark for equivocation proof checking

* session: add benchmark for membership proof checking

* offences: fix babe benchmark

* babe: add weights based on benchmark results

* babe: adjust weights after benchmarking on reference hardware

* babe: reorder checks in check_and_report_equivocation
2020-07-04 12:18:13 +02:00
André Silva fb2fa25331 primitives: use generic Header in testing runtime (#6561)
* primitives: use generic Header in testing runtime

* frame: remove unused imports

* Remove warning

Co-authored-by: Bastian Köcher <git@kchr.de>
2020-07-03 17:07:46 +00:00
Max Inden 635310c8d4 client/network: Remove unused Result returned by NetworkWorker (#6552) 2020-07-02 15:18:08 +02:00
Bastian Köcher 4f7f312be5 Move create_inherents into the block-builder (#6553)
* Move `create_inherents` into the block-builder

This moves the `create_inherents` call into the block-builder. This has
the advantage that `create_inherents` will be able to reuse the same
context that will be used when applying the extrinsics and we also save
one call to `on_initialize`. To make sure that `create_inherents` does
not modify any state, we execute it in a transaction that is
rolled-back after doing the runtime call.

* Feedback and build fix

* Update primitives/runtime/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Update client/block-builder/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2020-07-02 15:17:14 +02:00
pscott 540ae1c161 Update SubstrateCli to return String (#6550)
* Update SubstrateCli to return String

* Add default implementation for executable_name()

* Use display instead of PathBuf

* Get file_name in default impl of executable_name

* Remove String::from and use .into()

* Use default impl for executable_name()

* Use .as_str() and remove useless .to_string()

* Update only sp-io when running companion build

* Remove unneeded update of sp-io in CI

Co-authored-by: Cecile Tonglet <cecile@parity.io>
2020-07-02 13:34:56 +02:00
Cecile Tonglet 7b0028d75a Drop the tokio runtime before the task_manager (#6548)
* Initial commit

Forked at: ece0364170
Parent branch: origin/master

* Drop the tokio runtime before the task_manager

The tokio runtime must be dropped before the task_manager. Otherwise the objects the task_manager
keep alive are dropped before the tasks are finished.
2020-07-02 13:02:08 +02:00
Ashley 5f751e4472 Remove ServiceBuilderCommand and implement the chain ops as standalone functions instead. (#6543)
* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* Make task_manager in run_until_exit and make subcommands async

* Change the async_run fn to return a future and task manager

* async_run should take a result fn

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix spaces in export_raw_state

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-07-02 12:57:56 +02:00
cheme 424d5c722d Restrict Protected to some heap types. (#6471)
* Restrict `Protected` to some heap types.

* Comment abut Protected usage.

* Remove Protected from crypto, use secrecy crate for existing uses.

* use a parse function

* fix error convert

* Rename and move secretY string function.

* std result
2020-07-01 09:59:07 +00:00
David d73de3bed7 Fix mocking multiple http calls in the same function call (#6510)
* Fix mocking multiple http calls in the same function call

Fixes an issue where a function call would perform more than one http request and wait for each to complete before proceeding. The `RequestId` comes from the length of the `requests` collection in the `OffchainState` and if a request is completed before the next one starts it will be removed and the "next expected" will be off by one. This PR tries to fix that by using a request counter that tracks how many requests have been performed so that we can `remove()` items from the `expected_requests` at the right index.

I suspect that this is a sub-optimal soluton and perhaps requests and their mocks should live side by side in the same collection, e.g. in a tuple of `(PendingRequest, Option<ExpectedRequest>)`.

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Panic on overflow

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Use a Deque and push/pop expected requests

* fix test

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-07-01 10:22:47 +02:00
André Silva 4cf1b4fa7b grandpa: minor cleanups in communication module (#6371)
* grandpa: replace Result<(), ()> with Option<()>

* grandpa: replace &Option<T> with Option<&T>

* grandpa: cleanup local id and keystore usages

* grandpa: return bool on check_message_signature

* grandpa: fix erroneous log message on startup

* grandpa: fix test
2020-06-30 18:59:36 +01:00
Bastian Köcher 53be6ec510 Support synching of blocks that are not new_best (#6508)
* Start

* Remove debug println

* Add tests
2020-06-30 14:44:52 +00:00
Ashley b832e35c5e Remove the service, replacing it with a struct of individual chain components (#6352)
* WIP

* Making progress

* Almost ready

* Get service tests compiling

* Fix node screenshot

* Line widths

* Fix node cli tests

* Fix node cli warning

* ChainComponents -> ServiceComponents, fix tests

* make spawn_handle public

* Remove spawnnamed impl for taskmanager

* Move the keep alive stuff to the task manager

* Move the telemetry, base path, rpc keep_alive to the service builder

* Make the task manager keep alive an internal detail

* Rewrite the browser start_client future

* Remove run_node etc

* Revert my personal changes to browser-demo/build.sh

* use |config|

* Add a runtime_version function to SubstrateCli

* Reexport role and runtime version from sc cli

* Update Cargo.lock

* runtime_version -> native_runtime_version

* Pass chain spec to native_runtime_version for polkadot

* Fix line widths

* Traitify ServiceComponents Client
2020-06-30 11:00:42 +01:00
Bastian Köcher 493d5d8591 Fix tx-pool returning the same transaction multiple times (#6535)
* Fix tx-pool returning the same transaction multiple times

This fixes a bug that lead to returning the same transaction multiple
times when iterating the `ready` iterator. Internally the transaction
was kept in the `best` list and could be duplicated in that list be
re-inserting it again. This `best` list is using a `TransactionRef`
which internally uses a `insertion_id`. This `insertion_id` could lead
to the same transaction being inserted multiple times into the `best`
list.

* Update client/transaction-pool/src/testing/pool.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-06-30 11:02:46 +02:00
Toralf Wittner 3de2a88075 Update to libp2p v0.20.1 (#6465)
* Update to libp2p-0.20.0

* Update to `libp2p-0.20.1`.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-06-30 08:02:51 +00:00
Pierre Krieger 505540a2a0 Increase the limit for the maximum size of the telemetry name (#6523)
* Increase the limit for the maximum size of the telemetry name

* Fix test
2020-06-26 14:48:11 +00:00
Pierre Krieger 39c3d1c026 Fix an extra semi-colon yielding a wrong error (#6520)
* Fix an extra semi-colon yielding a wrong error

* Update client/cli/src/commands/run_cmd.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-26 08:05:24 +00:00
Benjamin Kampmann f6ebd5f87d Releasing rc4 – Rhinoceros (#6515)
Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-06-25 23:18:43 +02:00
Maciej Hirsz b17ccf24a8 Include genesis hash in system.connected (#6498)
* feat: Include genesis hash in system.connected message for telemetry

* chore: Spread call arguments into multiline

* chore: Removed commented code
2020-06-25 11:30:01 +02:00
Shawn Tabrizi 935ee6f545 Add DB Read/Write Tracking to Benchmarking Pipeline (#6386)
* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @kianenigma review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* Update client/db/src/bench.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-06-24 21:03:55 +02:00
André Silva a7b31bb8d2 client: fix print of slot duration on startup (#6495) 2020-06-24 17:42:27 +02:00
Cecile Tonglet 4bf044eac6 Ensure the listen addresses are consistent with the transport (#6436)
* Initial commit

Forked at: 82bdf1a891
No parent branch.

* Ensure the listen addresses are consistent with the transport

* Update client/network/src/error.rs

* Update client/network/src/service.rs

* Better implementation

* Fix bad previous impl

* add boot_nodes

* reserved nodes

* test boot nodes

* reserved nodes tests

* add public_addresses and make specific error type

* Update client/network/src/error.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-06-23 17:26:00 +02:00
Max Inden bd79b3debc client/network/service: Add primary dimension to connection metrics (#6472)
* client/network/service: Add primary dimension to connection metrics

Two nodes can be interconnected via one or more connections. The first
of those connections is called the primary connection.

This commit adds another dimension to the
`sub_libp2p_connections_{closed,opened}_total` metrics to differentiate
primary and non-primary connections being opened / closed.

By intuition more than one connection between two nodes is rare.
Tracking the fact whether a connection is primary or not will help prove
or disprove this intuition.

* .maintain/monitoring: Ensure to sum over all connections_closed variants

* client/network/service: Rename is_primary to is_first

* client/network/service: Split by metric name with two additional metrics

* Revert ".maintain/monitoring: Ensure to sum over all connections_closed variants"

This reverts commit 2d2f93e414440b9fc9e8f7fae6fe48bd95af6b8f.

* client/network/service: Remove labels from distinct metrics
2020-06-23 17:25:42 +02:00
Ashley d976f712b1 Fix the browser node and ensure it doesn't colour the informant output (#6457)
* Fix browser informant

* Fix documentation

* Add an informant_output_format function to the cli config

* Wrap informant output format in an option

* Revert batch verifier

* Remove wasm-timer from primitives io cargo lock

* Drop informant_output_format function

* derive debug for output format
2020-06-23 16:50:33 +02:00
Cecile Tonglet 63793c8b97 impl Debug for sc_service::Configuration (#6400)
* Initial commit

Forked at: 252416d385
No parent branch.

* Make sc_service::Configuration derive Debug

* Replace task_executor fn's input by proper TaskExecutor type (cleaner)

* impl From<Fn> for TaskExecutor

* Update client/cli/src/runner.rs

* Add some doc, examples and tests

* Replace Deref by fn spawn as suggested

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-23 12:47:13 +02:00
pscott ceb0fa6358 Optimize offchain worker api by re-using http-client (#6454)
* Fix typo in offchain's docs

* Use Self keyword in AsyncApi::new()

* Move httpclient to be part of OffchainWorkers to optimize block import

* Fix compilation errors for tests

* Add wrapper struct for HyperClient

* Use lazy_static share SharedClient amongst OffchainWorkers. Remove the need to raise the fd limit

* Revert "Use lazy_static share SharedClient amongst OffchainWorkers. Remove the need to raise the fd limit"

This reverts commit 7af97498a2383b5d7405e27823db8fd97245da41.

* Add lazy_static for tests
2020-06-23 12:09:47 +02:00