Commit Graph

265 Commits

Author SHA1 Message Date
Pierre Krieger 36a8c720f2 Add a warning if users pass --sentry or --sentry-nodes (#6779)
* Add a warning if users pass --sentry or --sentry-nodes

* Apply suggestions from code review

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix text

Co-authored-by: parity-processbot <>
Co-authored-by: Max Inden <mail@max-inden.de>
2020-08-12 09:58:01 +00:00
Nikolay Volf c347300e3e Move to upstream wasmtime, refactor globals snapshot (#6759)
* refactor globals snapshot

* ignore test

* update pwasm-utils ref

* line width

* add doc comment for internal struct

* add explanation for iteration

* Demote rustdoc to a comment

* use 0.14

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2020-08-11 15:05:31 +00:00
Arkadiy Paronyan 58ebf50839 Block packet size limit (#6398)
* Block packet size limit

* Update client/network/src/protocol.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Add block response limit

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-08-11 16:12:30 +02:00
Pierre Krieger ee35dc9415 Fix legacy substream fallback not working (#6826)
* Fix legacy substream fallback not working

* Make it nicer
2020-08-05 16:20:09 +00:00
Pierre Krieger 840781cddb Fix warning being printed by authority-discovery (#6820) 2020-08-05 09:16:19 +00:00
Pierre Krieger e449330e27 Add details to legacy requests (#6747) 2020-08-03 14:55:32 +00:00
Max Inden 0553dabe32 client/network: Fix wrong metric help text (#6794)
The `sub_libp2p_kademlia_query_duration` metric only has the dimension
`type` not `protocol`.
2020-08-03 09:08:06 +00:00
Max Inden 1365eef4c1 client/network: Expose DHT query duration to Prometheus (#6784)
Expose duration of DHT put and get request as a Prometheus histogram.
2020-08-03 08:30:06 +00:00
Pierre Krieger a1786a92ec Don't close inbound notifications substreams immediately (#6781)
* Don't close inbound notifications substreams immediately

* Fix not closing in return to node A closing
2020-07-31 14:42:53 +00:00
Pierre Krieger 5060324b91 Ignore flaky test (#6767) 2020-07-30 14:58:30 +02:00
Max Inden dc33e64643 client/network: Add peers to DHT only if protocols match (#6549)
* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Add peers to DHT only if protocols match

With https://github.com/libp2p/rust-libp2p/pull/1628 rust-libp2p allows
manually controlling which peers are inserted into the routing table.
Instead of adding each peer to the routing table automatically, insert
them only if they support the local nodes protocol id (e.g. `dot`)
retrieved via the `identify` behaviour.

For now this works around
https://github.com/libp2p/rust-libp2p/issues/1611. In the future one
might add more requirements. For example one might try to exclude
light-clients.

* Cargo.toml: Remove crates.io patch for libp2p

* client/network/src/behaviour: Adjust to PeerInfo name change

* client/network/src/discovery: Rework Kademlia event matching

* client/network/discovery: Add trace on adding peer to DHT

* client/network/discovery: Retrieve protocol name from kad behaviour

* client/network/discovery: Fix formatting

* client/network: Change DiscoveryBehaviour::add_self_reported signature

* client/network: Document manual insertion strategy

* client/network/discovery: Remove TODO for ignoring DHT address

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-07-29 14:44:15 +02:00
Gavin Wood 6bfbb7c6f1 Cleanup our sort usage (#6754) 2020-07-29 14:00:51 +02:00
Pierre Krieger 1ab7719314 Add a back-pressure-friendly alternative to NetworkService::write_notifications 🎉 (#6692)
* Add NetworkService::send_notifications

* Doc

* Doc

* API adjustment

* Address concerns

* Make it compile

* Start implementation

* Progress in the implementation

* Change implementation strategy again

* More work before weekend

* Finish changes

* Minor doc fix

* Revert some minor changes

* Apply suggestions from code review

* GroupError -> NotifsHandlerError

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* state_transition_waker -> close_waker

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Finish renames in service.rs

* More renames

* More review suggestsions applied

* More review addressing

* Final change

* 512 -> 2048

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2020-07-29 11:23:19 +00:00
Max Inden e674d64a72 client/network: Adjust wording (#6755)
Rename `NetworkWorker::from_worker` to `NetworkWorker::from_service` as
it is a channel from the `NetworkService` to the `NetworkWorker`.
2020-07-29 11:18:40 +00:00
Bastian Köcher e7d8040af8 Update parity-scale-codec to prepare for breaking rustc release (#6746)
This updates parity-scale-codec{-derive} to prepare for a rustc release
that would otherwise break the derive implementation:
https://github.com/rust-lang/rust/pull/73084
2020-07-28 20:26:01 +00:00
Bastian Köcher 9310f15ac2 Name all the tasks! (#6726)
* Remove any implementation of `Spawn` or `Executor` from our task executors

* Fix compilation

* Rename `SpawnBlockingExecutor`

* Update primitives/core/src/traits.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-26 12:56:17 +00:00
Wei Tang ac8de557d6 Better handling of stable-only build (#6569)
* Better handling of stable-only build

* Fix node template build

* Fix wasm builder node-template version mismatch

* Fix load_spec error

* Add , in parameter

* Add descrptive panic messages in tests

* Add descriptive tests in node/executor benches

* Fix missing compact_code_unwrap

* Add missing wasm_binary_unwrap function for executor integration test

* Only define import_sp_io in no_std

* Small Cargo.toml styling fix

* Bump wasm-builder to 2.0.0

* Fix all `with_wasm_builder_from_crates` version in Substrate

* Use `with_wasm_builder_from_crates` for node-template

Co-authored-by: Gavin Wood <gavin@parity.io>
2020-07-22 21:40:11 +02:00
pscott 046fda914a Improve overall performance (#6699)
* Improve overall performance

* Clean up code

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-21 14:46:49 +02:00
Pierre Krieger ab82eb1c98 Add sync_legacy_requests_received metric (#6698) 2020-07-21 11:59:49 +00:00
Benjamin Kampmann 629a801b6a Fixup cargo unleash (#6689)
* Bumping to rc5

* Revert "Bumping to rc5"

This reverts commit 5a63a8230e5a12db1b4ce4ac62e4967c52291da7.

* Bumping for rc5, correctly this time
2020-07-21 10:26:47 +00:00
Roman Borschel 025c501766 Update libp2p to 0.22. (#6690)
* Update libp2p to 0.22.

Primarily to begin the libp2p-noise upgrade process.

See https://github.com/libp2p/rust-libp2p/issues/1631.

* CI
2020-07-21 07:44:41 +00:00
Pierre Krieger 4720f0fdda Move the legacy protocol handshake to the legacy substream (#5938)
* Move the legacy protocol handshake to the legacy substream

* Fix tests

* Remove line that wasn't supposed to be committed

* Remove hack

* Rework how it's done

* Some little changes

* update_chain wasn't doing its thing

* Fix service tests not calling update_chain

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* [WIP]

* Revert "[WIP]"

This reverts commit 2b892e6a7637c0b1297e6ecdbb919321c9098ff5.

* Update client/network/src/protocol.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix received message not being handshake

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

Co-authored-by: Max Inden <mail@max-inden.de>
2020-07-15 09:29:10 +00:00
Pierre Krieger 4613dc2af4 Remove legacy Clogged event (#6652) 2020-07-14 15:43:08 +00:00
Pierre Krieger 150ef0a40a Deduplicate some dependencies (#6628) 2020-07-10 09:28:10 +00:00
Bastian Köcher 94cddee160 Improve transaction submission (#6599)
* Improve transaction submission

Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Make sure we don't submit the same transaction twice from the network concurrently

* Remove added listener call

* Feedback

* Ignore banned on resubmit
2020-07-08 17:42:42 +02:00
Max Inden faa72caf91 *: Update to libp2p v0.21.1 (#6559)
* *Cargo.toml: Update versions

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Adjust to one_shot.rs changes

* client/network/discovery: Log address list on trace level

* client/network/discovery: Ignore RoutablePeer and PendingRoutablePeer

* Commit Cargo.lock

* Finish update

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-07-08 14:00:30 +00:00
Pierre Krieger 8ca05879e8 Send Status message on all newly-opened legacy substreams (#6593)
* Send Status message on all newly-opened legacy substreams

* Fix tests
2020-07-08 11:44:51 +00:00
Max Inden 62f306d972 client/network: Rename DebugInfoBehaviour to PeerInfoBehaviour (#6556)
Information retrieved via `DebugInfoBehaviour` is not only used for
debugging purposes, e.g. disconnecting from nodes not responding to
pings, using external addresses retrieved via indentify, ...

In order for the name to reflect the usage of the module this commit
renames it.
2020-07-08 09:08:47 +00:00
Max Inden 635310c8d4 client/network: Remove unused Result returned by NetworkWorker (#6552) 2020-07-02 15:18:08 +02:00
Bastian Köcher 53be6ec510 Support synching of blocks that are not new_best (#6508)
* Start

* Remove debug println

* Add tests
2020-06-30 14:44:52 +00:00
Toralf Wittner 3de2a88075 Update to libp2p v0.20.1 (#6465)
* Update to libp2p-0.20.0

* Update to `libp2p-0.20.1`.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-06-30 08:02:51 +00:00
Benjamin Kampmann f6ebd5f87d Releasing rc4 – Rhinoceros (#6515)
Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-06-25 23:18:43 +02:00
Cecile Tonglet 4bf044eac6 Ensure the listen addresses are consistent with the transport (#6436)
* Initial commit

Forked at: 82bdf1a891
No parent branch.

* Ensure the listen addresses are consistent with the transport

* Update client/network/src/error.rs

* Update client/network/src/service.rs

* Better implementation

* Fix bad previous impl

* add boot_nodes

* reserved nodes

* test boot nodes

* reserved nodes tests

* add public_addresses and make specific error type

* Update client/network/src/error.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-06-23 17:26:00 +02:00
Max Inden bd79b3debc client/network/service: Add primary dimension to connection metrics (#6472)
* client/network/service: Add primary dimension to connection metrics

Two nodes can be interconnected via one or more connections. The first
of those connections is called the primary connection.

This commit adds another dimension to the
`sub_libp2p_connections_{closed,opened}_total` metrics to differentiate
primary and non-primary connections being opened / closed.

By intuition more than one connection between two nodes is rare.
Tracking the fact whether a connection is primary or not will help prove
or disprove this intuition.

* .maintain/monitoring: Ensure to sum over all connections_closed variants

* client/network/service: Rename is_primary to is_first

* client/network/service: Split by metric name with two additional metrics

* Revert ".maintain/monitoring: Ensure to sum over all connections_closed variants"

This reverts commit 2d2f93e414440b9fc9e8f7fae6fe48bd95af6b8f.

* client/network/service: Remove labels from distinct metrics
2020-06-23 17:25:42 +02:00
Roman Borschel 6aa8965f33 Avoid panic on dropping a sc_network::service::out_events::Receiver. (#6458)
* Avoid panic on dropping a `Receiver`.

* CI
2020-06-23 10:51:35 +02:00
André Silva d9776d8233 network: remove unused variable (#6460) 2020-06-22 20:41:37 +02:00
Nikolay Volf 36d2eefdc9 change everything to transaction (#6440) 2020-06-21 12:34:19 +02:00
Arkadiy Paronyan 7b34438178 Validate encoding of extrinsics passed to runtime (#6442)
* Validate encoding of extrinsics passed to runtime

* Bump codec version explicitly
2020-06-19 19:27:16 +00:00
Nikolay Volf 6e85555e0b add network propagated metrics (#6438) 2020-06-19 17:40:39 +03:00
Bastian Köcher 7858828d98 Update sync chain info on own block import (#6424)
Before we only updated the chain info of sync when we have imported
something using the import queue. However, if you import your own
blocks, this is not done using the import queue and so sync is not
updated. If we don't do this, it can lead to sync switching to "major
sync" mode because sync is not informed about new blocks. This
especially happens on Cumulus, where a collator is selected multiple
times to include its block into the relay chain and thus, sync switches
to major sync mode while the node is still building blocks.
2020-06-19 16:00:58 +02:00
Pierre Krieger dd7830d481 Remove --legacy-network-protocol CLI flag (#6411) 2020-06-19 00:23:58 +02:00
Pierre Krieger 527cc35afb Increase network buffer sizes even more (#6080) 2020-06-18 11:55:45 +00:00
arkpar 850ee8b43a Revert "Block packet size limit"
This reverts commit 08e0747d2b.
2020-06-18 10:43:03 +02:00
arkpar 08e0747d2b Block packet size limit 2020-06-18 10:39:54 +02:00
Pierre Krieger fec43fe6d7 Remove penalty on duplicate Status message (#6377) 2020-06-18 09:34:53 +02:00
Svyatoslav Nikolsky 194b3dfb17 fix BlockAttributes encoding (#6281) 2020-06-16 22:51:45 +02:00
Pierre Krieger 0787b189fa Use /dns/ instead of /dns4/ (#6369) 2020-06-16 22:49:44 +02:00
Roman Borschel 731e7d77c7 Find the alive incoming entry on disconnect. (#6320)
When a peer in `Incoming` state disconnects, the "alive" entry
in the `incoming` list for that peer must be updated (set to `false`).
Currently the entry that is updated may be an earlier entry for the
same peer that is already no longer alive. This can happen if a
peer repeatedly connects (incoming) and disconnects between invocations to
`poll()` of the behaviour.
2020-06-11 11:55:55 +02:00
Roman Borschel 273f31b7aa Avoid self-lookups in Authority Discovery (#6317)
* Ensure authority discovery avoids self-lookups.

Thereby additionally guard the `NetworkService` against
adding the local peer to the PSM or registering a
"known address" for the local peer.

* Clarify comments.

* See if returning errors is ok.
2020-06-10 18:50:37 +02:00
Denis Pisarev f9c0c6a719 New CI image (#6223)
* fix (ci): hotfix Docker release

* change (ci): moving to the tested CI image with a proper name

* change (ci): rename substrate-ci-linux

* Reduce the lots_of_incoming_peers_works test load (#6314)

* change (ci): moving to the tested CI image with a proper name

* change (ci): rename substrate-ci-linux

* Reduce the lots_of_incoming_peers_works test load (#6314)

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-06-10 13:12:21 +02:00