Commit Graph

2043 Commits

Author SHA1 Message Date
Bastian Köcher 685b43c3fc try-runtime: Use configured wasm execution method (#13694)
* try-runtime: Use configured wasm execution method

* Fix compilation
2023-03-24 17:50:16 +00:00
Davide Galassi 40e1704e1c Keystore overhaul (final) (#13683)
* Introduce keystore specialized sign methods

* Get rid of 'AppKey::UntypedGeneric' associated type.

Untyped generics are accessible using associated types 'Generic' associated type.
I.e. <T as AppKey>::Public::Generic

* Get rid of 'CryptoTypePublicPair'

* Trivial fix

* Small refactory of local keystore implementations

* Remove 'crypto_id' method from 'Public'

* Trivial rename of 'AppKey' to 'AppCrypto'

* Remove unused import

* Improve docs

* Better signature related errors for authority-discovery

* Apply review suggestion

* Apply review suggestions

Co-authored-by: Koute <koute@users.noreply.github.com>

* Authority discoverty signing error revisited

* Signing error revisited for babe and aura as well

* Further cleanup

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-03-24 13:46:02 +00:00
Alexandru Vasile 30122da07a chain_head/tests: Check finalized block event before new block (#13680)
* chain_head/tests: Mock client for custom block notification

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chain_head/tests: Check finalized block event before new block

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/rpc-spec-v2/src/chain_head/test_utils.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/rpc-spec-v2/src/chain_head/test_utils.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/rpc-spec-v2/src/chain_head/test_utils.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* chain_head/tests: Run import events with 10min timeout

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* chain_head/tests: Add comments about test

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-03-23 15:40:02 +02:00
Sebastian Kunert 7bd3f3d762 Remove unused light-client leftover (#13687)
* Remove unused light-client leftover

* Remove more unused code
2023-03-23 11:35:39 +01:00
Alexandru Vasile c2f32c9cbd fix/chain_head: Ensure correct events for finalized branch (#13632)
* chain_head/follow: Ensure correct events for finalized branch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Reenable tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Do some clean ups and add some more docs

* Fix gramatic

* Update client/rpc-spec-v2/src/chain_head/chain_head_follow.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* rpc/chain_head: Introduce error for absent headers

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-03-22 12:52:10 +02:00
Niklas Adolfsson 3b0a0aa92d refactor: state RPC remove max_rpc_payload_size (#13649)
This limit is not needed anymore as the JSON-RPC servers doesn't terminate
the connection if the RPC max limit is exceeded anymore
2023-03-21 15:25:33 +00:00
Arkadiy Paronyan 87197b094f Bump parity-db (#13656) 2023-03-21 10:45:40 +01:00
Davide Galassi 480396fe06 Keystore overhaul (iter 2) (#13634)
* Remove bloat about remote keystore

* Update docs and remove unused 'KeystoreRef' trait

* Use wherever possible, MemoryKeystore for testing

* Remove unrequired fully qualified method syntax for Keystore
2023-03-20 18:21:26 +00:00
Mira Ressel faaa0c2851 address clippy & rustdoc warnings
The comment was out of date anyway, State::Live no longer takes a
snapshot_path argument.
2023-03-20 16:35:53 +01:00
Oliver Tale-Yazdi 58a7f57594 Ignore flaky test (#13631)
* Ignore flaky test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Re-enable running_the_node_works_and_can_be_interrupted

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Re-enable notifications_back_pressure

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Unused import

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-03-17 16:06:40 +00:00
PG Herveou f5773795e9 Fix typos in pallet-contracts (#13629)
* Fix typos in pallet-contracts
2023-03-17 16:31:51 +01:00
Davide Galassi bc2724f022 Bump grandpa to 0.16.2 (#13622) 2023-03-17 14:21:02 +01:00
Davide Galassi f110941b7f Keystore overhaul (#13615)
* Remove 'supported_keys' 'sign_with_any' and 'sign_with_all' from keystore trait

* Remove the aync keystore

* Renaming:
- SyncCryptoStore -> Keystore
- SyncCryptoStorePtr -> KeystorePtr
- KeyStore -> MemoryKeystore

* Fix authority discovery worker and tests

* Rename 'insert_unknown' to 'insert'

* Remove leftover
2023-03-17 11:24:14 +00:00
Bastian Köcher 3e73b7557e Try to fix flaky temp-base-path-work test (#13505)
* Try to fix flaky `temp-base-path-work` test

The test is most of the time failing when checking if the database path was deleted. The assumption
is that it takes a little bit more time by the OS to actually clean up the temp path under high
load. The pr tries to fix this by checking multiple times if the path was deleted. Besides that it
also ensures that the tests that require the benchmark feature don't fail when compiled without the feature.

* ".git/.scripts/commands/fmt/fmt.sh"

* Capture signals earlier

* Rewrite tests to let them having one big timeout

* Remove unneeded dep

* Update bin/node/cli/tests/common.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Review feedback

* Update bin/node/cli/tests/common.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

---------

Co-authored-by: command-bot <>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
2023-03-16 12:24:20 +01:00
Adrian Catangiu 3708b156d9 sc-consensus-beefy: improve beefy gossip validator (#13606)
* sc-consensus-beefy: improve beefy gossip validator

Old gossip validator was pretty dumb, being very permissive with
incoming votes - only condition it had was to be newer than best
finalized.

New filter conditions:
 - voter rounds are initialized (discarding votes until voter is
   actually active),
 - only votes for current active set id are accepted,
 - only votes for rounds in the current voting session are accepted,
 - only votes for GRANDPA finalized blocks are accepted,
 - when BEEFY voter reaches mandatory round, only votes for said
   mandatory round are accepted.

New validator uses the VoterOracle to easily implement above conditions
and only allow through votes that are immediately useful to the voter.

After every GRANDPA or BEEFY finality, the gossip validator filter is
updated.

* sc-consensus-beefy: remove votes enqueueing

Since gossip validator will simply disallow votes for future rounds,
and only allow votes that the voter can immediately process, there
is no need for the voter to enqueue votes.

It will see these "future" votes later in rebroadcasts, when voter
will also be able to process them. Only at that point does gossip
accept and consume them.

* sc-consensus-beefy: refactor persistent state

Move best-beefy and best-grandpa into VoterOracle instead
of passing them around as params.
VoterOracle ultimately needs to know best-beefy and/or best-grandpa
for most of its functions.

* sc-consensus-beefy: further restrict gossip validator

Assuming mandatory done in current session:
Instead of allowing votes for any round in the current session, only
accept votes for rounds equal or better than best BEEFY finalized.

* sc-consensus-beefy: add a couple of comments

* sc-consensus-beefy: fix tests involving multiple tasks

Finalize blocks one a time in tests where we want gossip to happen
in a certain round. Otherwise, some tasks may be left behind in
terms of gossip round numbers because once "scheduled" a task will
greedily process as much as possible.

This change should be in line with the real-world scenario where
voters run "in parallel" across nodes, the only points of
synchronization being the finality notifications.

* sc-consensus-beefy: address review comments

---------

Signed-off-by: acatangiu <adrian@parity.io>
2023-03-16 11:02:39 +02:00
Alexandru Vasile 1a88833d73 Metadata V15: Expose API to fetch metadata for version (#13287)
* impl_runtime_apis: Generate getters for `metadata_at` functions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime: Implement new `Metadata` runtime trait

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime: Move `metadata_at` functions to construct_runtime macro

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* contruct_runtime: Use `OpaqueMetadata` from hidden imports

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Add tests for the new API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Adjust metdata naming

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose `metadata-v14` feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose metadata only under feature flags

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose v14 metadata by default

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose metadata feature for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Test metadata under different feature flags

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* client/tests: Adjust testing to reflect trait Metadata change

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Add intermediate representation types for metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Convert metadata to V14

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Add API to convert metadata to multiple versions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Expose V14 under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Adjust to metadata IR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: More adjustments

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Guard v14 details under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* CI: Ensure `quick-benchmarks` uses `metadata-v14`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Use `metadata-v14` for benchmarks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust cargo fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* kitchensink-runtime: Add feature flag for `metadata-v14`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support/test: Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support/test: Check crates locally

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Activate metadata-v14 for pallets

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove metadata-v14 feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata_ir: Move `api.rs` to `mod.rs`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Handle latest metadata conversion via IR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Add constant for metadata version 14

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support/test: Fix merge conflict

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame/support/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/src/metadata_ir/mod.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/test/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* frame/metadata: Collect pallet documentation for MetadataIR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Check pallet documentation is propagated to MetadataIR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-03-15 17:49:28 +00:00
cheme ef165cec3e update trie deps (#13601) 2023-03-14 21:03:50 +00:00
Adrian Catangiu 81a02cb7c4 sc-consensus-beefy: fix on-demand async state machine (#13599)
`futures_util::pending!()` macro only yields to the event loop once,
resulting in many calls to the `OnDemandJustificationsEngine::next()`
made by the tokio reactor even though the on-demand-engine is idle.

Replace it with the correct `futures::future::pending::<()>()` function
which forever yields to the event loop, which is what we want when
the engine is idle.

Signed-off-by: Adrian Catangiu <adrian@parity.io>
2023-03-14 16:52:53 +02:00
Aaro Altonen 9ced14e2de Move code from sc-network-common back to sc-network (#13592)
* Move service tests to `client/network/tests`

These tests depend on `sc-network` and `sc-network-sync` so they should
live outside the crate.

* Move some configs from `sc-network-common` to `sc-network`

* Move `NetworkService` traits to `sc-network`

* Move request-responses to `sc-network`

* Remove more stuff

* Remove rest of configs from `sc-network-common` to `sc-network`

* Remove more stuff

* Fix warnings

* Update client/network/src/request_responses.rs

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

* Fix cargo doc

---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>
2023-03-14 12:06:40 +00:00
Bastian Köcher c5aee09d9e sc-slots: Cleanup (#13590)
Calculate remaining proposal duration once and also use proposal time as timeout for creating the
inherent data.
2023-03-13 22:46:53 +01:00
André Silva da7190a113 grandpa: send neighbor packets to light clients on set transition (#13559)
* grandpa: send neighbor packets to light clients on set transition

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove match statement

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-03-09 14:05:04 +00:00
dependabot[bot] 16fc65dc1c Bump wasmtime from 6.0.0 to 6.0.1 (#13566)
Bumps [wasmtime](https://github.com/bytecodealliance/wasmtime) from 6.0.0 to 6.0.1.
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-some-possible-changes.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/compare/v6.0.0...v6.0.1)

---
updated-dependencies:
- dependency-name: wasmtime
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-09 07:08:00 +00:00
André Silva 13b0f24abd consensus: remove caching functionality from block import pipeline (#13551)
* consensus: remove caching functionality from block import pipeline

* client: update docs on Verifier::verify

* node: fix block production benchmark
2023-03-07 11:19:19 +00:00
Alexandru Vasile 48baaa53ad rpc/chainHead: Fix pruned blocks events from forks (#13379)
* rpc/chainhead: Test unpin for noncanonical prunned blocks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tests: Ensure fork is not reported by the Finalized event

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chainhead: Detect pruned forks to ignore from events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tests: Check unpin can be called on pruned hashes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Handle race with memory blocks and notifications

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Add data config for the `follow` future

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Address feedback

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Move best block cache on the data config

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Send new events from the finalized stream

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chian_head: Report all pruned blocks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Move `chainHead_follow` logic on dedicated file

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Delegate follow logic to `chain_head_follow`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Remove subscriptions on drop

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tests: Ignore pruned blocks for a longer fork

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tests: Check all pruned blocks are reported, not just stale heads

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tests: Remove println debug and fix indentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Remove unnecessary trait bounds

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Add debug log for pruned forks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "rpc/chain_head: Add debug log for pruned forks"

This reverts commit 425d6e7a8b60421bcece12add1941fe58524cf52.

* Adjust blockID for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/rpc-spec-v2/src/chain_head/chain_head_follow.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* rpc/chain_head: Rename `ChainHeadFollow` to `ChainHeadFollower`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Remove subscriptions manually

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Improve log messages by adding subID and errors

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Ensure `follow` stops sending events on first error

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Use default constructor

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Add `StartupPoint` structure

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Rename `in_memory_blocks`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Fix comment typo

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Keep unique blocks and remove itertools

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Make sure `bestBlocks` events are generated in order

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Maintain order of reported blocks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Parent of finalized block could be unpinned

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/chain_head: Fix warning

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-03-07 11:56:14 +02:00
Dmitry Markin 3118026576 Make unbounded channels size warning exact (part 2) (#13504) 2023-03-07 10:57:26 +03:00
Liu-Cheng Xu 58d1d9e117 Expose new_full_parts_with_genesis_builder from sc-service (#13539)
I forgot this in https://github.com/paritytech/substrate/pull/12291
2023-03-06 21:39:32 +01:00
Aaro Altonen 1a7f5be07f Extract syncing protocol from sc-network (#12828)
* Move import queue out of `sc-network`

Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.

This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.

* Move stuff to SyncingEngine

* Move `ChainSync` instanation to `SyncingEngine`

Some of the tests have to be rewritten

* Move peer hashmap to `SyncingEngine`

* Let `SyncingEngine` to implement `ChainSyncInterface`

* Introduce `SyncStatusProvider`

* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`

* Implement `SyncEventStream`

Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.

Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.

* Introduce `ChainSyncInterface`

This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.

* Move event stream polling to `SyncingEngine`

Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.

The code needs refactoring.

* Make `SyncingEngine` into an asynchronous runner

This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.

Code needs a lot of refactoring.

* Fix warnings

* Code refactoring

* Use `SyncingService` for BEEFY

* Use `SyncingService` for GRANDPA

* Remove call delegation from `NetworkService`

* Remove `ChainSyncService`

* Remove `ChainSync` service tests

They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.

* Refactor code

* Refactor code

* Update client/finality-grandpa/src/communication/tests.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Fix warnings

* Apply review comments

* Fix docs

* Fix test

* cargo-fmt

* Update client/network/sync/src/engine.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Update client/network/sync/src/engine.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Add missing docs

* Refactor code

---------

Co-authored-by: Anton <anton.kalyaev@gmail.com>
2023-03-06 16:33:38 +00:00
Arkadiy Paronyan ac4a23609c Bump parity-db (#13521) 2023-03-03 17:16:59 +00:00
Bastian Köcher 44abe690a3 network-gossip: Do not report peer on duplicate message if its the first time (#13508)
Two peers can send us the same gossip message. Before this pr we would have reported the second peer
for sending a duplicate message. However, this isn't correct if we haven't seen the message from
this peer yet. So, we should not report them as they can not be aware of our internal state.
2023-03-02 11:06:46 +00:00
Adrian Catangiu a21880b76f sc-consensus-beefy: fix metrics: use correct names (#13494)
Signed-off-by: acatangiu <adrian@parity.io>
2023-03-01 18:04:09 +02:00
André Silva b29584d93a sp-consensus: remove unused error variants (#13495) 2023-03-01 13:04:57 +00:00
Dmitry Markin ad410738e7 Make unbounded channels size warning exact (part 1) (#13490)
* Replace `futures-channel` with `async-channel` in `out_events`

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Also print the backtrace of `send()` call

* Switch from `backtrace` crate to `std::backtrace`

* Remove outdated `backtrace` dependency

* Remove `backtrace` from `Cargo.lock`

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-03-01 09:54:31 +00:00
yjh 20bf3c938e chore: move genesis block builder to chain-spec crate. (#13427)
* chore: move genesis block builder to block builder crate.

* add missing file

* chore: move genesis block builder to sc-chain-spec

* Update client/chain-spec/src/genesis.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update test-utils/runtime/client/src/lib.rs

* fix warnings

* fix warnings

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-02-28 20:50:57 +01:00
Davide Galassi 1ef9c473e7 Move BEEFY code to consensus (#13484)
* Move beefy primitives to consensus dir
* Move beefy gadget to client consensus folder
* Rename beefy crates
2023-02-28 15:56:22 +01:00
Aaro Altonen 033a0d01a4 Adjust the ratio of the in-peers/out-peers (#13477)
Establish fewer outbound connections in an attempt to allow publicly
available nodes to accept more full nodes.

Maintain the overall number of connections node should establish.
2023-02-28 09:52:11 +02:00
Michal Kucharczyk b5e7fbfb58 Service::Error: Storage variant removed (#13481) 2023-02-27 23:10:34 +00:00
Arsenii Lyashenko a1c546e9aa Add Windows support for storage monitor (#13466)
* Add Windows support for storage monitor

* Apply suggested changes
2023-02-27 20:26:38 +00:00
Davide Galassi 40c36c0c8a Move grandpa crates to consensus folder (#13458)
* Move grandpa under consensus dir
* Rename grandpa folder
* Finish grandpa renaming
* Minor tweaks
* Cargo fmt
* Adjust path to chain spec
2023-02-27 17:15:08 +01:00
Koute 91c595bc78 Further storage iterator refactoring (#13445)
* Remove `Backend::apply_to_key_values_while`

* Add `IterArgs::start_at_exclusive`

* Use `start_at_exclusive` in functions which used `Backend::apply_to_key_values_while`

* Remove `Backend::apply_to_keys_while`

* Remove `for_keys_with_prefix`, `for_key_values_with_prefix` and `for_child_keys_with_prefix`

* Remove unnecessary `to_vec` calls

* Fix unused method warning in no_std

* Remove unnecessary import

* Also check proof sizes in the test

* Iterate over both keys and values in `prove_range_read_with_size` and add a test
2023-02-27 10:45:42 +00:00
Anton bb00b262d7 [client/network] Remove unused argument (#13444)
* improve error message

* removed unused argument

* docs: disconnect_peer_inner no longer accepts `ban`

* remove redundant trace message

```
sync: Too many full nodes, rejecting 12D3KooWSQAP2fh4qBkLXBW4mvCtbAiK8sqMnExWHHTZtVAxZ8bQ
sync: 12D3KooWSQAP2fh4qBkLXBW4mvCtbAiK8sqMnExWHHTZtVAxZ8bQ disconnected
```

is enough to understand that we've refused to connect to the given peer

* Revert "removed unused argument"

This reverts commit c87f755b1fd03494fb446b604fe25c2418da7c87.

* ban peer for 10s after disconnect

* do not accept incoming conns if peer was banned

* Revert "do not accept incoming conns if peer was banned"

This reverts commit 7e59d05975765f2547468e9dcfd1361516c41e06.

* Revert "ban peer for 10s after disconnect"

This reverts commit 3859201ced42a5b2d18c0600e29efd20962a7289.

* Revert "Revert "removed unused argument""

This reverts commit f1dc623646dc5a69e1822c35f428e90dffe34d95.

* format code

* Revert "remove redundant trace message"

This reverts commit a87e65f08553dbe69027e9aa4f7ca4779ccaa7f2.
2023-02-27 10:43:15 +00:00
Liu-Cheng Xu 1d5ea509d0 Support the subscription of every imported block (#13372)
* Support the subscription of every import block

Close #13315

* Clean up any closed block import notification sinks

* Apply review suggestions

* Nit

* `every_block_import_notification_sinks` -> `every_import_notification_sinks`

* Apply review suggestions
2023-02-27 09:02:54 +00:00
Bastian Köcher 941288c6d0 wasm-executor: Support growing the memory (#12520)
* As always, start with something :P

* Add support for max_heap_pages

* Add support for wasmtime

* Make it compile

* Fix compilation

* Copy wrongly merged code

* Fix compilation

* Some fixes

* Fix

* Get stuff working

* More work

* More fixes

* ...

* More

* FIXEs

* Switch wasmi to use `RuntimeBlob` like wasmtime

* Removed unused stuff

* Cleanup

* More cleanups

* Introduce `CallContext`

* Fixes

* More fixes

* Add builder for creating the `WasmExecutor`

* Adds some docs

* FMT

* First round of feedback.

* Review feedback round 2

* More fixes

* Fix try-runtime

* Update client/executor/wasmtime/src/instance_wrapper.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/executor/common/src/wasm_runtime.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/executor/common/src/runtime_blob/runtime_blob.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/executor/common/src/wasm_runtime.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/allocator/src/freeing_bump.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/allocator/src/freeing_bump.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Feedback round 3

* FMT

* Review comments

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-02-24 11:43:01 +00:00
Koute 17e055e594 Bump wasmtime to 6.0.0 (#13429)
* Bump `wasmtime` to 6.0.0

* Disable cranelift egraphs
2023-02-23 10:56:11 +00:00
Koute f8e3bdad3d Rework storage iterators (#13284)
* Rework storage iterators

* Make sure storage iteration is also accounted for when benchmarking

* Use `trie-db` from crates.io

* Appease clippy

* Bump `trie-bench` to 0.35.0

* Fix tests' compilation

* Update comment to clarify how `IterArgs::start_at` works

* Add extra tests

* Fix iterators on `Client` so that they behave as before

* Add extra `unwrap`s in tests

* More clippy fixes

* Come on clippy, give me a break already

* Rename `allow_missing` to `stop_on_incomplete_database`

* Add `#[inline]` to `with_recorder_and_cache`

* Use `with_recorder_and_cache` in `with_trie_db`; add doc comment

* Simplify code: use `with_trie_db` in `next_storage_key_from_root`

* Remove `expect`s in the benchmarking CLI

* Add extra doc comments

* Move `RawIter` before `TrieBackendEssence` (no code changes; just cut-paste)

* Remove a TODO in tests

* Update comment for `StorageIterator::was_complete`

* Update `trie-db` to 0.25.1
2023-02-22 07:49:25 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Michal Kucharczyk 5ef88dd398 BlockId removal: BlockBuilderProvider::new_block_at (#13401)
* `BlockId` removal: `BlockBuilderProvider::new_block_at`

It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`

* fmt

* fix

* more fixes
2023-02-21 18:36:00 +00:00
Michal Kucharczyk 7a10154188 BlockId removal: runtime-api refactor (#13255)
* BlockId removal: refactor of runtime API

It changes the arguments of:
- `ApiExt` methods:  `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`

It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* BlockId removal: refactor of runtime API - tests

- tests adjusted to new runtime API,
- some tests migrated from block number to block hash

* benchmarking-cli: BlockId(0) migrated to info().genesis_hash

`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.

* trivial: api.call(BlockId) -> api.call(Hash)

- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,

* CallApiAtParams::at changed to Block::Hash

* missed doc updated

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

* BlockId removal: Benchmark::consumed_weight

Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.

* at_hash renamed

* wrong merge fixed

* beefy worker: merged with master

* beefy: tests: missing block problem fixed

* Apply review suggestion

* fix

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-02-20 22:47:21 +00:00
Adrian Catangiu bd8cb7ba2d client/beefy: drop justification on import if pallet not enabled (#13422)
BEEFY pallet allows setting on-chain BEEFY genesis to some future
block. Disregard any BEEFY justifications attached to imported blocks
that predate configured BEEFY genesis.

Signed-off-by: acatangiu <adrian@parity.io>
2023-02-20 16:58:37 +02:00
Dmitry Markin 8d033b6dfb Use async/await instead of manual polling of NetworkWorker (#13219)
* Convert `NetworkWorker::poll()` into async `next_action()`

* Use `NetworkWorker::next_action` instead of `poll` in `sc-network-test`

* Revert "Use `NetworkWorker::next_action` instead of `poll` in `sc-network-test`"

This reverts commit 4b5d851ec864f78f9d083a18a618fbe117c896d2.

* Fix `sc-network-test` to poll `NetworkWorker::next_action`

* Fix `sc_network::service` tests to poll `NetworkWorker::next_action`

* Fix docs

* kick CI

* Factor out `next_worker_message()` & `next_swarm_event()`

* Error handling: replace `futures::pending!()` with `expect()`

* Simplify stream polling in `select!`

* Replace `NetworkWorker::next_action()` with `run()`

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* minor: comment

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Print debug log when network future is shut down

* Evaluate `NetworkWorker::run()` future once before the loop

* Fix client code to match new `NetworkService` interfaces

* Make clippy happy

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Revert "Apply suggestions from code review"

This reverts commit 9fa646d0ed613e5f8623d3d37d1d59ec0a535850.

* Make `NetworkWorker::run()` consume `self`

* Terminate system RPC future if RPC rx stream has terminated.

* Rewrite with let-else

* Fix comments

* Get `best_seen_block` and call `on_block_finalized` via `ChainSync` instead of `NetworkService`

* rustfmt

* make clippy happy

* Tests: schedule wake if `next_action()` returned true

* minor: comment

* minor: fix `NetworkWorker` rustdoc

* minor: amend the rustdoc

* Fix bug that caused `on_demand_beefy_justification_sync` test to hang

* rustfmt

* Apply review suggestions

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-02-20 12:08:02 +00:00
Bastian Köcher fdd5203add state-db: Print warning when using large pruning window on RocksDb (#13414)
* state-db: Print warning when using large pruning window on RocksDb

This pr changes state-db to print a warning when using a large pruning window and running with a
database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out
of memory errors because this option together with RocksDb etc puts the entire pruning window into
memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central!

* Review comments
2023-02-18 16:59:08 +00:00