Commit Graph

549 Commits

Author SHA1 Message Date
Joshy Orndorff a08bf89662 Support Justification and FinalityProof import in pow import queue. (#5781) 2020-04-25 10:57:06 +00:00
André Silva 3793fbf9cc babe: remove error fallback from threshold calculation (#5764)
* babe: remove error fallbacks from threshold calculation

* babe: fix indent

* babe: fix domain

* babe: fix compilation

* babe: improve expect

* babe: add one more note about p's limit
2020-04-24 17:56:41 -04:00
Max Inden 324e51fbbf client/authority-discovery: Reduce log level replaced by metrics
Instead of logging value-found-event-handling failures or value-put
failures on error level, log them on debug level only additionally
recording them via Prometheus.

Motivation is that both events can happen in "normal" operations and
thus clutter the logs.
2020-04-24 17:40:48 +02:00
Wei Tang 969720c2ad babe: secondary blocks with VRF (#5501)
* babe: secondary blocks with VRF

* Fix node runtime compile

* Fix test-utils runtime interface

* Fix babe tests

* typo: v == 2

* babe: support online configuration upgrades

* Fix rpc tests

* Fix runtime version tests

* Switch to use NextConfigDescriptor instead of changing runtime interface

* Fix tests

* epoch-changes: map function that allows converting with different epoch types

* Add migration script for the epoch config change

* Fix docs for PrimaryAndSecondaryVRFSlots

* Add docs of `SecondaryVRF` in babe crate

* babe-primitives: Secondary -> SecondaryPlain

* babe-client: Secondary -> SecondaryPlain

* Fix migration tests

* test-utils-runtime: Secondary -> SecondaryPlain

* Fix missing name change in test-utils-runtime

* Fix migration: Epoch should be EpochV0

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* Fix new epochChanges version

* Fix babe-primitives naming changes

* Fix merge issues in babe-client

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-04-24 16:03:03 +01:00
Bernhard Schuster 72ee7d5797 feat/ocw/bookkeeping (#5200)
Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-04-24 16:46:19 +02:00
Pierre Krieger f66168505b Fix the cfg guards in service/metrics.rs (#5770) 2020-04-24 16:16:54 +02:00
Wei Tang 770cc24c47 babe: support online configuration upgrades (#5514)
* babe: support online configuration upgrades

* Switch to use NextConfigDescriptor instead of changing runtime interface

* Fix tests

* epoch-changes: map function that allows converting with different epoch types

* Add migration script for the epoch config change

* Fix migration tests

* Fix migration: Epoch should be EpochV0

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* Fix new epochChanges version

* Fix unused imports

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-04-24 14:59:14 +01:00
Pierre Krieger 6a7e86e677 Use new block requests protocol (#5760)
* Use new block requests protocol

* Tweak comment
2020-04-24 13:48:22 +02:00
Chevdor e731817e24 Fix error message when providing an incorrect peer-id (#5724)
* Fix error message when providing an incorrect peer-id

* Fix error message to cover all cases

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-04-23 22:20:24 +02:00
Arkadiy Paronyan 59f13e6e0d Refactored in-mem backed to use the actual trie (#5730) 2020-04-23 19:00:36 +02:00
Bastian Köcher 46677555ac Allow missing functions when checking the new runtime's version (#5741) 2020-04-23 14:54:52 +02:00
Svyatoslav Nikolsky 8689a97a0d make GrandpaJustification::decode_and_verify_finalizes public (#5746) 2020-04-23 12:07:08 +01:00
Benjamin Kampmann 624e95b1af Fix leak in stream notifications (#5739) 2020-04-23 12:01:09 +02:00
Max Inden e84b06a8c0 client/network-gossip/state_machine: Reduce indentation level (#5714)
* client/network-gossip/src/state_machine: Add unit test for on_incoming

Add two unit tests to ensure `on_incoming` is ingoring discarded
messages and reports and ignores messages from unknown peers.

* client/network-gossip/state_machine: Reduce indentation level

* client/network-gossip/bridge: Remove unnecessary into_iter

* client/network-gossip/state_machine: Report success after register check

* client/network-gossip/state_machine: Error not report on unregistered

`peers` contains all the peers we're connected to. If we receive a
message from a peer not in this list, that means there's an internal
problem within the local client. It's not the fault of the peer in
question.

With the above in mind instead of reducing the reputation of such peer,
log an error.
2020-04-23 09:58:43 +02:00
Toralf Wittner 421ef498f4 network: Only insert global addresses into the DHT. (#5735)
* network: Only insert global addresses into the DHT.

Currently every address reported via libp2p-identify is inserted into
the DHT which thus contains a multitude of unreachable addresses such
as from 127.0.0.0/8 or 10.0.0.0/8.

Issue #5099 suggested a dedicated service over UDP to gauge the
reachability of an address, which would however incur extra I/O costs
and be of limited use.

As an alternative and simpler tactic, this PR only allows global IP
addresses to be inserted into the DHT unless an explicit command-line
flag `--allow-non-global-addresses-in-dht` is given or a node is
started with `--dev`. This opt-in behaviour is meant to allow
site-local networks to still make use of a DHT.

* Enable non-global in more test setups.

* Replace command-line option with different name.

* Another test fix.
2020-04-23 09:52:20 +02:00
André Silva 96b7cec1ce slots: fix slot lenience methods (#5742)
* slots: extract slot lenience from babe and aura

* slots: add tests for slot lenience

* slots: fix comment in test
2020-04-23 09:42:53 +02:00
Pierre Krieger c424ce94a3 Enable yamux flow control by default (#5734)
* Enable yamux flow control by default

* Tweak description

* Apply suggestions from code review
2020-04-22 14:31:05 +02:00
Pierre Krieger a7f578d63a Add a protocol that answers finality proofs (#5718)
* Add a protocol that answers finality proofs

* Fix documentation

* Use Toggle
2020-04-22 10:58:26 +02:00
Pierre Krieger 3ad9f65f11 Send a status message on block announce handshake (#5726)
* Send a status message on block announce handshake

* Make sure to send the handshake to all handlers
2020-04-22 10:20:52 +02:00
Pierre Krieger 25c3ab2c1e Fix warning reported in out_events when Registry is None (#5716) 2020-04-22 00:36:12 +02:00
pscott f5d7b5bc92 Remove TaskManagerBuilder (#5725)
* Remove TaskManagerBuilder

* Clean up use declaration fo SpawnTaskHandle

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-04-22 00:35:01 +02:00
Max Inden 798de8337b client/authority-discovery: Add metric exposing priority group size (#5707)
Expose the amount of addresses that are passed from the authority
discovery module down to the network peer set manager as a priority
group in order to connect to the corresponding nodes.

```
authority_discovery_priority_group_size 10
```
2020-04-21 10:54:37 +02:00
Pierre Krieger f2578bdc3b Pass an encoded Roles as the notifications protocols handshakes (#5665) 2020-04-21 10:47:37 +02:00
Max Inden 5db336c743 client/network-gossip: Move sink IO outside of state_machine (#5669)
* client/network-gossip: Move sink IO outside of state_machine

`ConsensusGossip` is supposed to be a deterministic state machine.
`GossipEngine` wrapping `ConsensusGossip` should handle IO operations.

This commit moves the `message_sink` IO operations to `GossipEngine`.
More specifically on incoming messages a `GossipEngine` calls
`ConsensusGossip::on_incoming` to validate and register the messages.
`ConsensusGossip` returns the valid messages which are then forwarded by
`GossipEngine` to the upper layer via the `message_sinks`.

* client/network-gossip: Adjust and extend tests

* Update client/network-gossip/src/bridge.rs

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
2020-04-20 16:40:44 -04:00
Yuanchao Sun 52d0173ec3 Add RPC function state_getProof, resolves #1110 (#5649)
* Add RPC function state_getProof, resolves #1110

* Apply suggestions from code review

* Update client/rpc/src/state/state_full.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock

* Make block hash optional

* Wrap StorageProof as Bytes

* Add struct ReadProof

* Fix typo

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-20 21:09:55 +02:00
cheme 4ffcf98d8d Child trie api changes BREAKING (#4857)
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
2020-04-20 15:21:22 +02:00
Bastian Köcher 1d1caed335 Improve tracing (#5698)
* Improve tracing implementation

* Enable tracing in runtime interfaces

* Switch to `TRACE` level
2020-04-20 14:37:27 +02:00
Pierre Krieger ca1c60c2cf Move around definitions in sc_network (#5701) 2020-04-20 14:35:17 +02:00
Igor Matuszewski 4b1f7d187f Allow to expose a subset of unsafe RPCs (#5233)
* sc-cli: Use type-safe constructors for RPC/Prometheus interfaces

* service: Simplify rpc handler creation

Could probably be further simplifies once [this][commit] lands.

[commit]: https://github.com/paritytech/jsonrpc/commit/20485387ed06a48f1a70bf4d609a7cde6cf0accf

* service: Streamline some HTTP & WS server start logic

* client: Introduce a simple RPC policy mechanism

* rpc/system: Check unsafe RPCs

* rpc/offchain: Check unsafe RPCs

* rpc/author: Check unsafe RPCs
2020-04-20 11:03:58 +02:00
Pierre Krieger cff487dd22 Refactor misleading log in discovery (#5679)
* Refactor misleading log

* Forgot to git add the compilation fix
2020-04-18 14:04:23 +02:00
Pierre Krieger bc6ebabf23 Force libp2p 0.18.1 (#5689) 2020-04-17 23:00:44 +02:00
Pierre Krieger f544907bd2 Revert listening on IPv6 by default (#5687) 2020-04-17 20:34:23 +02:00
Ashley 95dc400bb8 Make network_config_path an Option (#5661)
* Make network_config_path an Option

* Fix network tests

* Use None as the network config path

* Fix cli

* Don't make PathBuf an Option in a cli context
2020-04-17 14:48:45 +02:00
Gavin Wood fd2cb9ca83 Transaction versioning in the RuntimeVersion (#5582)
* Add transaction_version

* Semantic versioning for runtimes

* Move new field to bottom

* Versioning

* Runtime versioning stuff.

* Fix test

* Adds tests and fixes bugs

* Bump runtime

Co-authored-by: Bastian Köcher <git@kchr.de>
2020-04-17 12:10:31 +02:00
Nikolay Volf fea626ca84 Add prometheus registry to transaction pool, with couple of initial metrics (#5657)
* make new contructor

* add metrics to txpool

* fix review

* fix doc comment

* change to counters

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* use dedicated wrapper

Co-authored-by: Max Inden <mail@max-inden.de>
2020-04-17 11:02:45 +02:00
Bastian Köcher c5c9b56e94 Listen on ipv6 by default as well (#5677)
Instead of just listening on ipv4 by default, this enables listening on
ipv6 as well. While doing this, it also fixes a bug which lead to always
listen on all sockets, even if we specify listen addresses explicitly.
2020-04-17 09:53:52 +02:00
Pierre Krieger 0fd5643e84 Adjustments to Kademlia-related metrics (#5660)
* Turn kbuckets_num_nodes into a GaugeVec

* random_kademlia_queries -> kademlia_random_queries

* kademalia_random_queries_total now a CounterVec

* Add metrics about records store
2020-04-17 09:11:47 +02:00
Tomasz Drwięga bc0b8fbddf Add performance tracing to validate_transaction (#5671)
* Validate transaction timeouts

* Add timing

* Add tracing to transaction validation.

* Fix docs.

Co-authored-by: NikVolf <nikvolf@gmail.com>
2020-04-17 08:42:50 +02:00
Nikolay Volf 372f8b2c7e Batch signature verification (#5023)
* create parallel tasks extension

* make type system happy

* basic externalities

* test for dynamic extensions

* batching test

* remove premature verify_batch

* shnschnorrkel batch

* alter test

* shnschnorrkel test

* executive batching

* some docs

* also multi/any signatgures

* error propagation

* styling

* make verification extension optional

* experimental ed25519 parallelization

* some merge fallout

* utilize task executor

* merge fallout

* utilize task executor more

* another merge fallout

* feature-gate sp-io

* arrange toml

* fix no-std

* sr25519 batching and refactoring

* add docs

* fix name

* add newline

* fix block import test

* long sr25519 test

* blocking instead of parking

* move everything in crypto

* return batch_verify to check :)

* use condvars

* use multi-threaded executor for benches

* don't call via host interface

* try no spawning

* add true

* cleanup

* straighten batching

* remove signature check from this test (?)

* remove now pointless test

* remove another now useless test

* fix warnings

* Revert "remove another now useless test"

This reverts commit bbdec24bb67ed4373072daef7c863e1a8825bd8b.

* rethink the sp-io-part

* Revert "remove now pointless test"

This reverts commit 4d553066322e65782264caa6053d4cd5538df977.

* fix wording

* add  wording

* add todo and fix

* return check and fix

* add logging in sp-io

* Update primitives/io/src/batch_verifier.rs

Co-Authored-By: cheme <emericchevalier.pro@gmail.com>

* address review and use std condvar

* account for early exit

* address reivew

* address review

* more suggestions

* add docs for batch verification

* remove unused

* more review suggestions

* move to sp-runtime

* add expects

* remove blocks

* use entry

* Update primitives/io/src/batch_verifier.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/externalities/src/extensions.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* update overlooked note

* remove stupid return

* Update primitives/io/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/io/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* fix wording

* bump spec_version

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-16 21:40:04 +02:00
Nikolay Volf 36243068bd update parity-util-mem (#5668) 2020-04-16 21:38:04 +02:00
Max Inden d742e88e79 client/authority-discovery: Allow to be run by sentry node (#5568)
* client/authority-discovery: Allow to be run by sentry node

When run as a sentry node, the authority discovery module does not
publish any addresses to the dht, but still discovers validators and
sentry nodes of validators.

* client/authority-discovery/src/lib: Wrap lines at 100 characters

* client/authority-discovery: Remove TODO and unused import

* client/authority-discovery: Pass role to new unit tests

* client/authority-discovery: Apply suggestions

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* bin/node/cli/src/service: Use expressions instead of statements

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-04-16 18:11:26 +01:00
Pierre Krieger 95ee37d242 Add alternative RPC methods to system_networkState (#5643)
* Add alternatives to system_networkState

* Fix tests

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-04-16 16:36:14 +01:00
Max Inden 6bd93825ca client/authority-discovery: Rework error handling (#5631)
* client/authority-discovery: Rework error handling

Instead of `handle_dht_events` returning a `Result<(), Error>`, return
a `Poll<Error>` where `Poll::Pending` signals that there are no more
events to handle and `Poll::Ready(Error)` signals that a fatal error
occured. Non fatal errors are handled within `handle_dht_events`
directly, thus looping in `poll` is not necessary anymore.

* client/authority-discovery: Return () instead of error on termiantion

* Update client/authority-discovery/src/lib.rs

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-04-16 16:09:13 +01:00
Pierre Krieger 239d0998ea Several tweaks to networking Prometheus metrics (#5636) 2020-04-16 15:18:35 +02:00
Bastian Köcher 240b304b4c Upgrade sysinfo to 0.13.3 (#5659)
This release includes a fix for an out of bound access that was reported
by one of the validators.
2020-04-16 12:59:31 +02:00
Nikolay Volf 84991146e3 don't use default features to drop some dependencies (#5658) 2020-04-16 12:08:16 +02:00
Toralf Wittner 4db45a85de Use a Kademlia instance per ProtocolId. (#5045) 2020-04-16 10:43:40 +02:00
Max Inden 1c7525fd4b client/authority-discovery: Increase request counter per request (#5632) 2020-04-16 10:42:08 +02:00
Pierre Krieger efde6056f6 Temporarily increase notifications buffer size (#5644)
* Temporarily increase notifications buffer size

* Add a 511.0 bucket
2020-04-15 23:27:33 +02:00
Pierre Krieger 1c7a3fe7d5 Remove the tokio thread name (#5650) 2020-04-15 21:49:53 +02:00