Commit Graph

780 Commits

Author SHA1 Message Date
Pierre Krieger d5b308f13c Fix networking debug_asserts (#8200)
* Fix networking debug_asserts

* Fix comment
2021-02-25 13:51:29 +01:00
Pierre Krieger a3d749f3c5 Add some debug_asserts for #8171 (#8181) 2021-02-24 17:13:34 +01:00
Pierre Krieger ba659f9440 Bump lru dependency (#8182) 2021-02-23 12:52:59 +01:00
Roman Borschel fbd3148bba Update to libp2p-0.35.1 (#8141) 2021-02-22 09:52:22 +01:00
Pierre Krieger aa95196be3 Extract transactions handling from protocol.rs (#8110)
* Extract transactions handling from protocol.rs

* Oops, boolean

* Do this better

* Update client/network/src/transactions.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* [WIP] Fix handshake

* Finish handshake change

* Bugfix

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2021-02-18 17:04:23 +01:00
Pierre Krieger f6273f7914 Remove all code related to sentry nodes (#8079)
* Remove all code related to sentry nodes

* More fixing
2021-02-15 08:45:03 +00:00
Benjamin Kampmann 6a964c5df0 Releasing 3.0 (#8098)
* bumping version for next release
* add changelog
* add guide
2021-02-10 19:23:18 +01:00
Benjamin Kampmann 2d31af3eb6 sc-network: switch on default features for libp2p on non-wasm-builds (#8088) 2021-02-10 08:42:37 +00:00
Pierre Krieger 1eeafdd2b3 Remove backwards-compatibility networking hack (#8068)
* Remove backwards-compatibility networking hack

* Fix compilation

* Try fix
2021-02-08 13:44:34 +00:00
Benjamin Kampmann 8e36d87ca8 Update dependencies ahead of next release (#8015)
Updates dependencies:
parity-db 0.2.2
paste
prometheus 0.11
cfg-if 1.0
strum 0.20
env_logger 0.8
pin-project
prost
nix
platforms
quickcheck 1.0
2021-02-04 19:17:42 +01:00
Robert Klotzner 0ec82c3484 Export IfDisconnected in public module. (#8034) 2021-02-03 09:39:39 +01:00
Pierre Krieger 11b00681d2 Improve log line (#8032)
Co-authored-by: parity-processbot <>
2021-02-03 08:24:23 +00:00
Arkadiy Paronyan e905c9a92d Storage chains: serve transactions over IPFS/bitswap (#7963)
* IPFS server for transactions

* Style

* Indent

* Log message

* CLI option

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Style

* Style

* Minor fixes

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-02-03 07:14:23 +00:00
Robert Klotzner 3628998d3c Add a send_request function to NetworkService (#8008)
* Add a `send_request` to `NetworkService`.

This function delivers responses via a provided sender and also allows
for sending requests to currently not connected peers.

* Document caveats of send_request better.

* Fix compilation in certain cases.

* Update docs + introduce IfDisconnected enum

for more readable function calls.

* Doc fix.

* Rename send_request to detached_request.

* Whitespace fix - arrrgh

* Update client/network/src/service.rs

spaces/tabs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update client/network/src/request_responses.rs

Documentation fix

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Update client/network/src/service.rs

Typo.

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Update client/network/src/service.rs

Better docs.

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Update client/network/src/service.rs

Typo.

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Update client/network/src/service.rs

Doc improvements.

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Remove error in logs on dialing a peer.

This is now valid behaviour.

* Rename detached_request to start_request.

As suggested by @romanb.

* Fix merged master.

* Fix too long lines.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2021-02-02 20:52:12 +01:00
Pierre Krieger eeff8f3204 Disable Kademlia random walk when --reserved-nodes is passed (#7999)
* Disable Kademlia random walk when --reserved-nodes is passed

* Update client/network/src/discovery.rs

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
2021-02-02 13:52:09 +00:00
Max Inden 3006100977 client/network: Use request response for light client requests (#7895)
* client/network: Re-enable light_client_handler.rs unit tests

* client/network: Add scaffolding for light client using req-resp

* client/network: Make it compile

* client/network: Rename OutEvent SendRequest

* client/network: Restructure light client request client and handler

* client/network: Rename light client request client to sender

* client/network: Remove light client prepare_request

* client/network/src/light: Rework configuration

* client/network: Formatting

* client/network/light: Remove RequestId

* client/network/light: Make request functions methods

* client/network/light: Refactor request wrapping

* client/network/light: Fix warnings

* client/network/light: Serialize request in method

* client/network/light: Make returning response a method

* client/network/light: Depend on request response to timeout requests

* client/network: Fix test compilation

* client/network/light: Re-enable connection test

* client/network/light: Re-enable timeout test

* client/network/light: Re-enable incorrect_response test

* client/network/light: Re-enable wrong_response_type test

* client/network/light: Re-enable retry_count_failures test

* client/network/light: Re-enable issue_request tests

* client/network/light: Re-enable send_receive tests

* client/network/light: Deduplicate test logic

* client/network/light: Remove unused imports

* client/network/light: Handle request failure

* client/network/light: Move generate_protocol_config

* client/network: Fix test compilation

* client/network: Rename light client request client to sender

* client/network: Handle too-many-requests error

* client/network: Update outdated comments

* client/network/light: Choose any peer if none has best block defined

* .maintain: Replace sentry-node with local-docker-test-network

Sentry nodes are deprecated. Thus there is no need for
`.maintain/sentry-node` to spin up a sentry node test environment.
Instead this commit rewrites the setup to contain two full-connected
validators and one light client.

With the steps below one can now spin up a local test network with
two validators, one light-client, Prometheus and Grafana.

- cargo build --release
- sudo docker-compose -f .maintain/local-docker-test-network/docker-compose.yml up

* client/network/light: Handle oneshot cancellation

* client/network/light: Do not reduce retry count on missing peer

* client/network/request-response: Assert in debug request id to be unique

* client/network/light: Choose same limit as block request protocol

* client/network: Report reputation changes via response

Allow request response protocol handlers to issue reputation changes, by
sending them back along with the response payload.

* client/network: Remove resolved TODOs
2021-02-01 15:59:47 +00:00
Bastian Köcher d40f9c166f Sync: Fix issue of not freeing a block announcement slot (#8006)
* Sync: Fix issue of not freeing a block announcement slot

There was a bug that when the block announcement validation returned an
error, the slot reserved for this validation wasn't freed. This could
lead to a situation where we rejected any block announcement from such a
peer for that the block announcement returned an error multiple times.

* Better logging

* Fuck I'm dumb

* 🤦
2021-01-29 16:29:25 +00:00
Guillaume Thiolliere f48296e9ad Update parity-scale-codec to 2.0 (#7994)
* update cargo.toml

* use 2.0 in mmmr
2021-01-29 12:22:45 +00:00
Pierre Krieger dc70c889ce Introduce sc_peerset::DropReason (#7996)
* Introduce sc_peerset::DropReason

* Fix peerset tests
2021-01-29 09:33:27 +00:00
Pierre Krieger 12fe61a965 Increase maximum size of transaction notifications (#7993) 2021-01-27 10:41:48 +00:00
Max Inden b692954c6e client/network: Report reputation changes via response (#7958)
* client/network: Report reputation changes via response

When handling a request by a remote peer in a request response handler,
one might want to in- or de-crease the reputation of the peer. E.g. one
might want to decrease the reputation slightly for each request, given
that it forces the local node to do work, or one might want to issue a
larger reputation change due to a malformed request by the remote peer.

Instead of having to pass a peerset handle to each request response
handler, this commit suggests to allow handlers to isssue reputation
changes via the provided `pending_response` `oneshot` channel.

A reputation change issued by a request response handler via the
`pending_response` channel is received by the
`RequestResponsesBehaviour` which passes the reputation change up as an
event to eventually be send to a peerset via a peerset handle.

* client/network/req-resp: Use Vec::new instead of None::<Vec<_>>

* client/network: Rename Response to OutgoingResponse

Given that a request-response request is not called `Request` but
`InomingRequest`, rename a request-response response to
`OutgoingResponse`.

* client/finality-grandpa-warp: Send empty rep change via response
2021-01-26 10:45:41 +00:00
Max Inden 62d551a063 client/network/req-resp: Prevent request id collision (#7957)
* client/network/req-resp: Add unit test for request id collision

* client/network/req-resp: Prevent request id collision

`RequestId` is a monotonically increasing integer, starting at
`1`. A `RequestId` is unique for a single `RequestResponse`
behaviour, but not across multiple `RequestResponse` behaviours. Thus
when handling `RequestId` in the context of multiple
`RequestResponse` behaviours, one needs to couple the protocol name
with the `RequestId` to get a unique request identifier.

This commit ensures that pending requests (`pending_requests`) and
pending responses (`pending_response_arrival_time`) are tracked both by
their protocol name and `RequestId`.

* client/network/req-resp: Remove unused import

* client/network/req-resp: Introduce ProtocolRequestId struct

* client/network/req-resp: Update test doc comment

Treat `RequestId` as an opaque type.

* client/network/req-resp: Improve expect proof
2021-01-25 13:09:45 +00:00
Cecile Tonglet 970cc25cef Telemetry per node (#7463) 2021-01-20 12:28:56 +01:00
Pierre Krieger 9d6b9684d9 Disable Nagle algorithm (#7932)
* Disable Nagle algorithm

* Oops, didn't compile
2021-01-20 10:14:24 +00:00
Bastian Köcher 450b96c50d Sync: Propagate block announcement data (#7903)
* Sync: Propagate block announcement data

This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.

* Review comment

* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked

* Also use out_peers
2021-01-19 17:01:11 +01:00
Pierre Krieger 8e04515912 Add explicit limits to notifications sizes and adjust yamux buffer size (#7925)
* Add explicit limits to notifications sizes and adjust yamux buffer size

* Docfix

* Tests

* Document these 10 bytes
2021-01-19 11:00:37 +00:00
Pierre Krieger e39ffb5a3c Address review comments of #7916 (#7917) 2021-01-18 16:52:25 +01:00
Pierre Krieger c58a2d9a74 Allow configuring Yamux window size (#7916) 2021-01-18 11:10:47 +01:00
Pierre Krieger 7db15cfb75 Fix not restoring non-poisoned state (#7906) 2021-01-15 12:47:32 +00:00
Pierre Krieger 782fe4a3fb Fix bad debug_assert (#7904) 2021-01-15 11:55:21 +01:00
Max Inden 3927b649cb *: Update to libp2p v0.34.0 (#7888)
* *: Update to libp2p v0.34.0

* client/network: Update bytes, unsigned-varint and asynchronous-codec

* client: Update to prost v0.7
2021-01-15 09:55:42 +00:00
Pierre Krieger 4678da3efb Clean-up pass in network/src/protocol.rs (#7889)
* Remove statistics system

* Remove ContextData struct

* Remove next_request_id

* Some TryFrom nit-picking

* Use constants for peer sets
2021-01-13 10:27:49 +00:00
Guillaume Thiolliere 014e6f03c6 bumpd minor version (#7873) 2021-01-12 09:25:47 +01:00
Igor Matuszewski 028f3d2674 Merge 2.0.1 backport branch into mainline master (#7842)
* Backport paritytech/substrate#7381

* Bring back genesis storage build in aura/timestamp

To not change spec version, see
https://github.com/paritytech/substrate/pull/7686#discussion_r540032743

* Backport paritytech/substrate#7238

* Backport paritytech/substrate#7395

* Bump impl_version

* Fix UI tests and bump trybuild dep

See https://github.com/rust-lang/rust/pull/73996

Backports:
https://github.com/paritytech/substrate/pull/7764
https://github.com/paritytech/substrate/pull/7656

* Partially backport paritytech/substrate#7838

* Release frame-support with a dep compilation fix

* Bump patch level for remaining crates

This is done because at the time of writing cargo-unleash does not fully
support partial workspace publishing and mixes both local and crates.io
versions of the packages, leading to errors in the release check workflow.

* Backport paritytech/substrate#7854

...to fix compilation error when using futures-* v0.3.9.

* Adding Changelog  entry for patch release

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
2021-01-11 11:24:29 +01:00
Max Inden ffb5410ba1 client/network: Re-enable light_client_handler.rs unit tests (#7853) 2021-01-08 17:22:55 +01:00
Pierre Krieger a4032b1dfb Update to futures 0.3.9 (#7854) 2021-01-08 15:47:33 +00:00
Pierre Krieger 779c4f8616 Rework priority groups, take 2 (#7700)
* Rework priority groups

* Broken tests fix

* Fix warning causing CI to fail

* [Hack] Try restore backwards-compatibility

* Fix peerset bug

* Doc fixes and clean up

* Error on state mismatch

* Try debug CI

* CI debugging

* [CI debug] Can I please see this line

* Revert "[CI debug] Can I please see this line"

This reverts commit 4b7cf7c1511f579cd818b21d46bd11642dfac5cb.

* Revert "CI debugging"

This reverts commit 9011f1f564b860386dc7dd6ffa9fc34ea7107623.

* Fix error! which isn't actually an error

* Fix Ok() returned when actually Err()

* Tweaks and fixes

* Fix build

* Peerset bugfix

* [Debug] Try outbound GrandPa slots

* Another bugfix

* Revert "[Debug] Try outbound GrandPa slots"

This reverts commit d175b9208c088faad77d9f0ce36ff6f48bd92dd3.

* [Debug] Try outbound GrandPa slots

* Apply suggestions from code review

Co-authored-by: Max Inden <mail@max-inden.de>

* Use consts for hardcoded peersets

* Revert "Try debug CI"

This reverts commit 62c4ad5e79c03d561c714a008022ecac463a597e.

* Renames

* Line widths

* Add doc

Co-authored-by: Max Inden <mail@max-inden.de>
2021-01-07 13:52:39 +00:00
Andronik Ordian 2b48e09b24 upgrade a few dependencies (#7831)
* upgrade a few dependencies

* make it compile at the expense of duplicate deps

* fix web-wasm and a warning

* introduce activate-wasm-bindgen-features crate

* Revert "introduce activate-wasm-bindgen-features crate"

This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11.

* add getrandom feature to sc-consensus-aura
2021-01-06 16:12:17 +00:00
Max Inden 3f629f743b client/network: Use request response for block requests (#7478)
* client/network: Add scaffolding for finality req to use req resp
	#sc

* client/network/src/finality_requests: Remove

* client/network/src/behaviour: Pass request id down to sync

* client/network: Use request response for block requests

* client/network: Move handler logic into *_*_handler.rs

* client/network: Track ongoing finality requests in protocol.rs

* client/network: Remove commented out finalization initialization

* client/network: Add docs for request handlers

* client/network/finality_request_handler: Log errors

* client/network/block_request_handler: Log errors

* client/network: Format

* client/network: Handle block request failure

* protocols/network: Fix tests

* client/network/src/behaviour: Handle request sending errors

* client/network: Move response handling into custom method

* client/network/protocol: Handle block response errors

* client/network/protocol: Remove tracking of obsolete requests

* client/network/protocol: Remove block request start time tracking

This will be handled generically via request-responses.

* client/network/protocol: Refactor on_*_request_started

* client/network: Pass protocol config instead of protocol name

* client/network: Pass protocol config in tests

* client/network/config: Document request response configs

* client/network/src/_request_handler: Document protocol config gen

* client/network/src/protocol: Document Peer request values

* client/network: Rework request response to always use oneshot

* client/network: Unified metric reporting for all request protocols

* client/network: Move protobuf parsing into protocol.rs

* client/network/src/protocol: Return pending events after poll

* client/network: Improve error handling and documentation

* client/network/behaviour: Remove outdated error types

* Update client/network/src/block_request_handler.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* Update client/network/src/finality_request_handler.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* client/network/protocol: Reduce reputation on timeout

* client/network/protocol: Refine reputation changes

* client/network/block_request_handler: Set and explain queue length

* client/service: Deny block requests when light client

* client/service: Fix role matching

* client: Enforce line width

* client/network/request_responses: Fix unit tests

* client/network: Expose time to build response via metrics

* client/network/request_responses: Fix early connection closed error

* client/network/protocol: Fix line length

* client/network/protocol: Disconnect on most request failures

* client/network/protocol: Disconnect peer when oneshot is canceled

* client/network/protocol: Disconnect peer even when connection closed

* client/network/protocol: Remove debugging log line

* client/network/request_response: Use Clone::clone for error

* client/network/request_response: Remove outdated comment

With libp2p v0.33.0 libp2p-request-response properly sends inbound
failures on connections being closed.

Co-authored-by: Addie Wagenknecht <addie@nortd.com>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2021-01-05 18:20:54 +00:00
Max Inden f0b99dd2f2 *: Update to libp2p v0.33.0 (#7759)
* *: Update to libp2p v0.33.0

* client/network: Consistently track request arrival time

With https://github.com/libp2p/rust-libp2p/pull/1886/ one is guaranteed
to receive either a `ResponseSent` or a `InboundFailure` event for each
received inbound request via `RequestResponseEvent::Message`. Given this
guarantee there is no need to track arrival times in a best-effort
manner and thus there is no need to use a LRU cache for arrival times.

* client/offchain: Adjust to PeerId API changes
2021-01-04 11:54:51 +00:00
Bastian Köcher e3e651f72c Happy new year (#7814)
* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL
2021-01-04 09:03:13 +00:00
Bastian Köcher 104747daa0 Remove unnecessary parameter from sync (#7761) 2020-12-18 15:02:08 +01:00
Bastian Köcher d3c7a99d3c Update common block in sync after importing blocks of a peer, please read UPDATE (#7733)
* Update common block in sync after importing blocks of a peer

This updates the sync code to update the common block of a peer, after
we have imported blocks from this peer. This fixes a bug for when we are
connected to one or more nodes that are doing a full sync as our node.
Nodes in full sync will not announce new blocks, as we don't send import
notifications on full sync. The problem as now that we were connected to
some peer that reported some low number as its best and we tried to sync
these blocks. But, as we did not update the common block of this peer,
we would sync these blocks over and over again. Being captured in some
time warp.
The solution to this problem is that we increase the common number as we
import blocks from this peer.

* Test

* Test name..

* Fix test

* Cleanup some code and write some new regression test

* Implement the ancestor search

* Check that the common number is smaller than the last finalized block

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Change the way we build the status messages

* Start some new test...

* Finish test

* Rename test

* Update client/network/src/protocol.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-12-18 12:04:17 +01:00
honeywest 1ecc3d5ed8 delete client in Cargo in client (#7752) 2020-12-17 17:12:27 +00:00
dependabot[bot] 688227ef48 Bump zeroize from 1.1.1 to 1.2.0 (#7731)
Bumps [zeroize](https://github.com/iqlusioninc/crates) from 1.1.1 to 1.2.0.
- [Release notes](https://github.com/iqlusioninc/crates/releases)
- [Commits](https://github.com/iqlusioninc/crates/compare/zeroize/v1.1.1...zeroize/v1.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-12-15 20:01:05 +01:00
Pierre Krieger ae4f9b14ae Fix wrong order of values in log line (#7728) 2020-12-14 18:37:19 +00:00
Bastian Köcher 059746183d Default block announce validation should reject additional data (#7726)
If we are using the default block announce validator and receive
additional data alongside the block announcement, we should disconnect
this peer.
2020-12-14 18:47:08 +01:00
Guillaume Thiolliere 8814d793c7 Fix docs (#7710)
* fix docs

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
2020-12-10 14:38:56 +01:00
Max Inden e4e120e9b3 *: Update to libp2p v0.32.2 (#7708)
Version v0.32.2 pins async-tls to the semver compliant v0.11.0 instead
of the yanked async-tls v0.10.2.
2020-12-10 09:22:40 +00:00
Max Inden 0a2636d20c *: Update to libp2p v0.32.0 (#7696)
* *: Update to libp2p v0.32.0

* Cargo.lock: Update async-tls to 0.10.2

* client/network/request_response: Adjust to new request response events

* client/network/request_response.rs: Clean up silently failing responses

* client/network/discovery: Lazily instantiate mdns

* client/network/discovery: Exclude MdnsWrapper for target_os unknown

* client/network/discovery: Fix indentation

* client/network/request-response: Use LruCache to track pending resp time

* client/network/request_responses: Fix early connection closed error

* client/network/request-response: Replace debug_assert with debug

* client/network/request-response: Fix typo

* client/network/request-response: Don't emit event on send_response fail

* client/network/request-response: Revert waker.wake_by_ref()

* client/network/request-resp: Make duration in InboundRequest optional

* client/network/req-resp: Don't emit two events for busy builder

When a response builder is busy incoming requests are dropped.
Previously this was reported both via a `ResponseFailure::Busy` and a
`ReponseFailure::Network(InboundFailure::Omisssion)` event.

With this commit the former is removed, leaving only the latter in
place.
2020-12-09 21:58:22 +00:00