* Clean up sc-network
- Avoid using clone() for the Copy type `PeerId`.
- Use `find_map` for `filter_map` and `next`.
- Use `Self`.
* More on Copy types
* Cargo +nightly fmt --all
* More ..
* fmt
* Revert vec![default_notif_handshake_message]
* Split native executor stuff from wasm executor stuff
* Remove `native_runtime_version` in places
* Fix warning
* Fix test warning
* Remove redundant NativeRuntimeInfo trait
* Add a warning for use_native
* Run cargo fmt
* Revert "Add a warning for use_native"
This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.
* Make choosing an executor (native/wasm) an explicit part of service construction
* Add Cargo.lock
* Rename Executor to ExecutorDispatch
* Update bin/node/executor/src/lib.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Fix tests
* Fix minor node-executor error
* Fix node cli command thing
Co-authored-by: Squirrel <gilescope@gmail.com>
* RUSTSEC-2021-0076 bump libsecp256k1
libsecp256k1 allows overflowing signatures
https://rustsec.org/advisories/RUSTSEC-2021-0076
Changes were made to conform to libsecp256k1 version differences.
Closes#9356
* parse_standard_slice() -> parse_overflowing_slice()
* Added v2 host function for ecdsa_verify
* Add feature tag over helpers
* Added ecdsa_verify v2 to test runner
* PR feedback
- Spaces -> tabs
- renamed two helper functions
* Fixed imports after rebasing
* Bump rest of libsecp256k1 (and libp2p)
libp2p also uses libsecp256k1 so it is required to be bumped too, along
with all the version difference changes.
* Add version2 for ecdsa pubkey recovery
* libp2p rebase master fixes
* Fix test panic when non Behaviour event is returned
* Update bin/node/browser-testing/Cargo.toml
* Update primitives/core/src/ecdsa.rs
* Update primitives/core/src/ecdsa.rs
* Update Cargo.lock
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.
Updated code related to deprecated warning on tracing-subscriber `scope()`
( See https://github.com/tokio-rs/tracing/issues/1429 )
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* mark template and utils as non-publish
* switch to development version for testing
* activate unleash check
* maybe if I disable all rules...
* Fix isolated compilation of `max-encoded-len-derive` with `syn`
error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
--> src/lib.rs:88:29
|
88 | .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
| --------- ^^ ----------------------------------- _
| |
| syn::Path
error: aborting due to previous error
For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`
* WIP: bump changes crates since v3 tag to next breaking
cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0
FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`
* Update lockfile
* WIP: Bump sp-transaction-pool as well
* WIP: Bump sp-offchain as well
* WIP: Bump frame-system-rpc-runtime-api as well
* WIP: Bump sp-authority-discovery as well
* Manually deactivate dev-deps before `cargo unleash check`
Otherwise we run into `Cycle detected` error.
* Bump sp-consensus-slots
* Add missing Cargo.lock change
* Bump sp-consensus-vrf as well
* Bump sp-keyring as well
* Bump sp-consensus-pow as well
* Try to speed up the `unleash-check` job
Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help
* fixup: Ensure the temp target dir exists for unleash check
* Bump pallet-transaction-payment-rpc-runtime-api as well
Needed for Polkadot
* Bump pallet-transaction-payment-rpc as well
Needed for Polkadot
* Try updating crates after patching in the Polkadot CI job
* Use another approach to update patched Substrate crates
* Try to update all sp-core versions in Polkadot CI job
* Simplify sp-core version checking
* Apply another shellcheck lint
* Just do the simplest thing I guess
* Welp don't do --offline then
* Clean up `unleash-check` job triggers
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Fix a note in unleash-check cache step
* Add a note about temporary optimization in cargo-unleash
* Pin a newer version of cargo-unleash
Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Allow fallback names for protocols
* Apply suggestions from code review
Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
* Fix some issues
* Fix compilation after merging master
Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
* Make grandpa work
* Introduce `SharedData`
* Add test and fix bugs
* Switch to `SharedData`
* Make grandpa tests working
* More Babe work
* Make it async
* Fix fix
* Use `async_trait` in sc-consensus-slots
This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.
* Make grandpa tests compile
* More Babe tests work
* Fix network test
* Start fixing service test
* Finish service-test
* Fix sc-consensus-aura
* Fix fix fix
* More fixes
* Make everything compile *yeah*
* Fix build when we have Rust 1.51
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix warning
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* primitives/runtime: initial changes on supporting multiple Justifications
* primitives/runtime: make Justifications strongly typed
* Encode/decode Justifications
* primitives/runtime: add Justification type
* backend: apply_finality and finalize_block takes a single Justification
* manual-seal: create engine id and let rpc take encoded justification
* backend: skeleton functions for appending justifications
* backend: initial implementation append_justification
Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k
* backend: guard against duplicate consensus engine id
* client/db: add check for block finality
* client/api: add append_justification to in_mem db
* client/light: add no-op append_justification
* network: fix decode call for Justification
* network: only send a single Justification in BlockData
* network: minor comment update
* protocol: update field names to distinguish single justification
* client: further field renames to plural
* client: update function names to plural justifications
* client/db: upgrade existing database for new format
* network: remove dependency on grandpa crate
* db: fix check for finalized block
* grandpa: check for multiple grandpa justifications hwne importing
* backend: update Finalizer trait to take multiple Justifications
* db: remove debugging statements in migration code
* manual-seal: update note about engine id
* db: fix check for finalized block
* client: update variable name to reflect it is now plural
* grandpa: fix incorrect empty Justications in test
* primitives: make Justifications opaque to avoid being empty
* network: fix detecting empty Justification
* runtime: doc strings for Justifications functions
* runtime: add into_justifications
* primitives: check for duplicates in when adding to Justifications
* network/test: use real grandpa engine id in test
* client: fix reviewer comments
* primitives: rename Justifications::push to append
* backend: revert changes to Finalizer trait
* backend: revert mark_finalized
* backend: revert changes to finalize_block
* backend: revert finalized_blocks
* db: add a quick early return for performance
* client: minor reviewer comments
* service/test: use local ConsensusEngineId
* network: add link to issue for sending multiple Justifications
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* network: tweaks to review suggestions
* network: revert change to BlockData for backwards compatibility
* Apply suggestion from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* primitives: update doc comment for Justifications
* client/db/upgrade: avoid grandpa crate dependency
* consensus: revert to single Justification for import_justification
* primitives: improve justifications docs
* style cleanups
* use and_then
* client: rename JUSTIFICATIONS db column
* network: revert to using FRNK in network-test
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
We introduced banning of peers who spam us with the same request (more
than 2 times). However, we missed that it is completely legal to send
the same request multiple times as long as we did not provide any
answer. An example for that is the justification request. This request
is send multiple times until we could fetch the justification from one
of our peers. So, the solution to this problem is to tag requests as
fulfilled and to start counting these fulfilled requests. If the number
is higher than what we allow, the peer should be banned.
* Decrease the peer reputation on invalid block requests
This pr changes the block request handler to decrease the reputation of
peers when they send the same request multiple times or they send us an
invalid block request.
* Review feedback
* Change log target
* Remove unused code
* client/network: Re-enable light_client_handler.rs unit tests
* client/network: Add scaffolding for light client using req-resp
* client/network: Make it compile
* client/network: Rename OutEvent SendRequest
* client/network: Restructure light client request client and handler
* client/network: Rename light client request client to sender
* client/network: Remove light client prepare_request
* client/network/src/light: Rework configuration
* client/network: Formatting
* client/network/light: Remove RequestId
* client/network/light: Make request functions methods
* client/network/light: Refactor request wrapping
* client/network/light: Fix warnings
* client/network/light: Serialize request in method
* client/network/light: Make returning response a method
* client/network/light: Depend on request response to timeout requests
* client/network: Fix test compilation
* client/network/light: Re-enable connection test
* client/network/light: Re-enable timeout test
* client/network/light: Re-enable incorrect_response test
* client/network/light: Re-enable wrong_response_type test
* client/network/light: Re-enable retry_count_failures test
* client/network/light: Re-enable issue_request tests
* client/network/light: Re-enable send_receive tests
* client/network/light: Deduplicate test logic
* client/network/light: Remove unused imports
* client/network/light: Handle request failure
* client/network/light: Move generate_protocol_config
* client/network: Fix test compilation
* client/network: Rename light client request client to sender
* client/network: Handle too-many-requests error
* client/network: Update outdated comments
* client/network/light: Choose any peer if none has best block defined
* .maintain: Replace sentry-node with local-docker-test-network
Sentry nodes are deprecated. Thus there is no need for
`.maintain/sentry-node` to spin up a sentry node test environment.
Instead this commit rewrites the setup to contain two full-connected
validators and one light client.
With the steps below one can now spin up a local test network with
two validators, one light-client, Prometheus and Grafana.
- cargo build --release
- sudo docker-compose -f .maintain/local-docker-test-network/docker-compose.yml up
* client/network/light: Handle oneshot cancellation
* client/network/light: Do not reduce retry count on missing peer
* client/network/request-response: Assert in debug request id to be unique
* client/network/light: Choose same limit as block request protocol
* client/network: Report reputation changes via response
Allow request response protocol handlers to issue reputation changes, by
sending them back along with the response payload.
* client/network: Remove resolved TODOs
* Sync: Fix issue of not freeing a block announcement slot
There was a bug that when the block announcement validation returned an
error, the slot reserved for this validation wasn't freed. This could
lead to a situation where we rejected any block announcement from such a
peer for that the block announcement returned an error multiple times.
* Better logging
* Fuck I'm dumb
* 🤦
* Sync: Propagate block announcement data
This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.
* Review comment
* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked
* Also use out_peers
* Rework priority groups
* Broken tests fix
* Fix warning causing CI to fail
* [Hack] Try restore backwards-compatibility
* Fix peerset bug
* Doc fixes and clean up
* Error on state mismatch
* Try debug CI
* CI debugging
* [CI debug] Can I please see this line
* Revert "[CI debug] Can I please see this line"
This reverts commit 4b7cf7c1511f579cd818b21d46bd11642dfac5cb.
* Revert "CI debugging"
This reverts commit 9011f1f564b860386dc7dd6ffa9fc34ea7107623.
* Fix error! which isn't actually an error
* Fix Ok() returned when actually Err()
* Tweaks and fixes
* Fix build
* Peerset bugfix
* [Debug] Try outbound GrandPa slots
* Another bugfix
* Revert "[Debug] Try outbound GrandPa slots"
This reverts commit d175b9208c088faad77d9f0ce36ff6f48bd92dd3.
* [Debug] Try outbound GrandPa slots
* Apply suggestions from code review
Co-authored-by: Max Inden <mail@max-inden.de>
* Use consts for hardcoded peersets
* Revert "Try debug CI"
This reverts commit 62c4ad5e79c03d561c714a008022ecac463a597e.
* Renames
* Line widths
* Add doc
Co-authored-by: Max Inden <mail@max-inden.de>
* upgrade a few dependencies
* make it compile at the expense of duplicate deps
* fix web-wasm and a warning
* introduce activate-wasm-bindgen-features crate
* Revert "introduce activate-wasm-bindgen-features crate"
This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11.
* add getrandom feature to sc-consensus-aura
* *: Update to libp2p v0.33.0
* client/network: Consistently track request arrival time
With https://github.com/libp2p/rust-libp2p/pull/1886/ one is guaranteed
to receive either a `ResponseSent` or a `InboundFailure` event for each
received inbound request via `RequestResponseEvent::Message`. Given this
guarantee there is no need to track arrival times in a best-effort
manner and thus there is no need to use a LRU cache for arrival times.
* client/offchain: Adjust to PeerId API changes
* Update common block in sync after importing blocks of a peer
This updates the sync code to update the common block of a peer, after
we have imported blocks from this peer. This fixes a bug for when we are
connected to one or more nodes that are doing a full sync as our node.
Nodes in full sync will not announce new blocks, as we don't send import
notifications on full sync. The problem as now that we were connected to
some peer that reported some low number as its best and we tried to sync
these blocks. But, as we did not update the common block of this peer,
we would sync these blocks over and over again. Being captured in some
time warp.
The solution to this problem is that we increase the common number as we
import blocks from this peer.
* Test
* Test name..
* Fix test
* Cleanup some code and write some new regression test
* Implement the ancestor search
* Check that the common number is smaller than the last finalized block
* Update client/network/src/protocol/sync.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/network/src/protocol/sync.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/network/src/protocol/sync.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Change the way we build the status messages
* Start some new test...
* Finish test
* Rename test
* Update client/network/src/protocol.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* *: Update to libp2p v0.32.0
* Cargo.lock: Update async-tls to 0.10.2
* client/network/request_response: Adjust to new request response events
* client/network/request_response.rs: Clean up silently failing responses
* client/network/discovery: Lazily instantiate mdns
* client/network/discovery: Exclude MdnsWrapper for target_os unknown
* client/network/discovery: Fix indentation
* client/network/request-response: Use LruCache to track pending resp time
* client/network/request_responses: Fix early connection closed error
* client/network/request-response: Replace debug_assert with debug
* client/network/request-response: Fix typo
* client/network/request-response: Don't emit event on send_response fail
* client/network/request-response: Revert waker.wake_by_ref()
* client/network/request-resp: Make duration in InboundRequest optional
* client/network/req-resp: Don't emit two events for busy builder
When a response builder is busy incoming requests are dropped.
Previously this was reported both via a `ResponseFailure::Busy` and a
`ReponseFailure::Network(InboundFailure::Omisssion)` event.
With this commit the former is removed, leaving only the latter in
place.
* Inform sync explicitly about new best block
Instead of "fishing" the new best block out of the processed blocks, we
now tell sync directly that there is a new best block. It also makes
sure that we update the corresponding sync handshake to the new best
block. This is required for parachains as they first import blocks and
declare the new best block after being made aware of it by the relay chain.
* Adds test
* Make sure async stuff had time to run
* Remove necessity to pass ConsensusEngineId when registering notifications protocol
* Line width
* Fix tests protocol name
* Other renames
* Doc update
* Change issue in TODO
Dan Shields