* rpc/chainhead: Test unpin for noncanonical prunned blocks
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tests: Ensure fork is not reported by the Finalized event
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chainhead: Detect pruned forks to ignore from events
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tests: Check unpin can be called on pruned hashes
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Fix clippy
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Handle race with memory blocks and notifications
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Add data config for the `follow` future
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Address feedback
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Move best block cache on the data config
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Send new events from the finalized stream
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chian_head: Report all pruned blocks
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Move `chainHead_follow` logic on dedicated file
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Delegate follow logic to `chain_head_follow`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Remove subscriptions on drop
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tests: Ignore pruned blocks for a longer fork
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tests: Check all pruned blocks are reported, not just stale heads
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tests: Remove println debug and fix indentation
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Remove unnecessary trait bounds
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Add debug log for pruned forks
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Revert "rpc/chain_head: Add debug log for pruned forks"
This reverts commit 425d6e7a8b60421bcece12add1941fe58524cf52.
* Adjust blockID for testing
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update client/rpc-spec-v2/src/chain_head/chain_head_follow.rs
Co-authored-by: Davide Galassi <davxy@datawok.net>
* rpc/chain_head: Rename `ChainHeadFollow` to `ChainHeadFollower`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Remove subscriptions manually
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Improve log messages by adding subID and errors
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Ensure `follow` stops sending events on first error
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Use default constructor
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Add `StartupPoint` structure
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Rename `in_memory_blocks`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Fix comment typo
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Keep unique blocks and remove itertools
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Make sure `bestBlocks` events are generated in order
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Maintain order of reported blocks
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Parent of finalized block could be unpinned
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/chain_head: Fix warning
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Move import queue out of `sc-network`
Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.
This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.
* Move stuff to SyncingEngine
* Move `ChainSync` instanation to `SyncingEngine`
Some of the tests have to be rewritten
* Move peer hashmap to `SyncingEngine`
* Let `SyncingEngine` to implement `ChainSyncInterface`
* Introduce `SyncStatusProvider`
* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`
* Implement `SyncEventStream`
Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.
Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.
* Introduce `ChainSyncInterface`
This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.
* Move event stream polling to `SyncingEngine`
Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.
The code needs refactoring.
* Make `SyncingEngine` into an asynchronous runner
This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.
Code needs a lot of refactoring.
* Fix warnings
* Code refactoring
* Use `SyncingService` for BEEFY
* Use `SyncingService` for GRANDPA
* Remove call delegation from `NetworkService`
* Remove `ChainSyncService`
* Remove `ChainSync` service tests
They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.
* Refactor code
* Refactor code
* Update client/finality-grandpa/src/communication/tests.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Fix warnings
* Apply review comments
* Fix docs
* Fix test
* cargo-fmt
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Add missing docs
* Refactor code
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Two peers can send us the same gossip message. Before this pr we would have reported the second peer
for sending a duplicate message. However, this isn't correct if we haven't seen the message from
this peer yet. So, we should not report them as they can not be aware of our internal state.
* Replace `futures-channel` with `async-channel` in `out_events`
* Apply suggestions from code review
Co-authored-by: Koute <koute@users.noreply.github.com>
* Also print the backtrace of `send()` call
* Switch from `backtrace` crate to `std::backtrace`
* Remove outdated `backtrace` dependency
* Remove `backtrace` from `Cargo.lock`
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
Establish fewer outbound connections in an attempt to allow publicly
available nodes to accept more full nodes.
Maintain the overall number of connections node should establish.
* Remove `Backend::apply_to_key_values_while`
* Add `IterArgs::start_at_exclusive`
* Use `start_at_exclusive` in functions which used `Backend::apply_to_key_values_while`
* Remove `Backend::apply_to_keys_while`
* Remove `for_keys_with_prefix`, `for_key_values_with_prefix` and `for_child_keys_with_prefix`
* Remove unnecessary `to_vec` calls
* Fix unused method warning in no_std
* Remove unnecessary import
* Also check proof sizes in the test
* Iterate over both keys and values in `prove_range_read_with_size` and add a test
* improve error message
* removed unused argument
* docs: disconnect_peer_inner no longer accepts `ban`
* remove redundant trace message
```
sync: Too many full nodes, rejecting 12D3KooWSQAP2fh4qBkLXBW4mvCtbAiK8sqMnExWHHTZtVAxZ8bQ
sync: 12D3KooWSQAP2fh4qBkLXBW4mvCtbAiK8sqMnExWHHTZtVAxZ8bQ disconnected
```
is enough to understand that we've refused to connect to the given peer
* Revert "removed unused argument"
This reverts commit c87f755b1fd03494fb446b604fe25c2418da7c87.
* ban peer for 10s after disconnect
* do not accept incoming conns if peer was banned
* Revert "do not accept incoming conns if peer was banned"
This reverts commit 7e59d05975765f2547468e9dcfd1361516c41e06.
* Revert "ban peer for 10s after disconnect"
This reverts commit 3859201ced42a5b2d18c0600e29efd20962a7289.
* Revert "Revert "removed unused argument""
This reverts commit f1dc623646dc5a69e1822c35f428e90dffe34d95.
* format code
* Revert "remove redundant trace message"
This reverts commit a87e65f08553dbe69027e9aa4f7ca4779ccaa7f2.
* Rework storage iterators
* Make sure storage iteration is also accounted for when benchmarking
* Use `trie-db` from crates.io
* Appease clippy
* Bump `trie-bench` to 0.35.0
* Fix tests' compilation
* Update comment to clarify how `IterArgs::start_at` works
* Add extra tests
* Fix iterators on `Client` so that they behave as before
* Add extra `unwrap`s in tests
* More clippy fixes
* Come on clippy, give me a break already
* Rename `allow_missing` to `stop_on_incomplete_database`
* Add `#[inline]` to `with_recorder_and_cache`
* Use `with_recorder_and_cache` in `with_trie_db`; add doc comment
* Simplify code: use `with_trie_db` in `next_storage_key_from_root`
* Remove `expect`s in the benchmarking CLI
* Add extra doc comments
* Move `RawIter` before `TrieBackendEssence` (no code changes; just cut-paste)
* Remove a TODO in tests
* Update comment for `StorageIterator::was_complete`
* Update `trie-db` to 0.25.1
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* `BlockId` removal: `BlockBuilderProvider::new_block_at`
It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`
* fmt
* fix
* more fixes
* BlockId removal: refactor of runtime API
It changes the arguments of:
- `ApiExt` methods: `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`
It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* BlockId removal: refactor of runtime API - tests
- tests adjusted to new runtime API,
- some tests migrated from block number to block hash
* benchmarking-cli: BlockId(0) migrated to info().genesis_hash
`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.
* trivial: api.call(BlockId) -> api.call(Hash)
- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,
* CallApiAtParams::at changed to Block::Hash
* missed doc updated
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* ".git/.scripts/commands/fmt/fmt.sh"
* BlockId removal: Benchmark::consumed_weight
Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.
* at_hash renamed
* wrong merge fixed
* beefy worker: merged with master
* beefy: tests: missing block problem fixed
* Apply review suggestion
* fix
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
* Convert `NetworkWorker::poll()` into async `next_action()`
* Use `NetworkWorker::next_action` instead of `poll` in `sc-network-test`
* Revert "Use `NetworkWorker::next_action` instead of `poll` in `sc-network-test`"
This reverts commit 4b5d851ec864f78f9d083a18a618fbe117c896d2.
* Fix `sc-network-test` to poll `NetworkWorker::next_action`
* Fix `sc_network::service` tests to poll `NetworkWorker::next_action`
* Fix docs
* kick CI
* Factor out `next_worker_message()` & `next_swarm_event()`
* Error handling: replace `futures::pending!()` with `expect()`
* Simplify stream polling in `select!`
* Replace `NetworkWorker::next_action()` with `run()`
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* minor: comment
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* Print debug log when network future is shut down
* Evaluate `NetworkWorker::run()` future once before the loop
* Fix client code to match new `NetworkService` interfaces
* Make clippy happy
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* Revert "Apply suggestions from code review"
This reverts commit 9fa646d0ed613e5f8623d3d37d1d59ec0a535850.
* Make `NetworkWorker::run()` consume `self`
* Terminate system RPC future if RPC rx stream has terminated.
* Rewrite with let-else
* Fix comments
* Get `best_seen_block` and call `on_block_finalized` via `ChainSync` instead of `NetworkService`
* rustfmt
* make clippy happy
* Tests: schedule wake if `next_action()` returned true
* minor: comment
* minor: fix `NetworkWorker` rustdoc
* minor: amend the rustdoc
* Fix bug that caused `on_demand_beefy_justification_sync` test to hang
* rustfmt
* Apply review suggestions
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
* state-db: Print warning when using large pruning window on RocksDb
This pr changes state-db to print a warning when using a large pruning window and running with a
database that isn't supporting ref-counting like RocksDb. This makes the user aware of potential out
of memory errors because this option together with RocksDb etc puts the entire pruning window into
memory. Besides that the pr introduces `LOG_TARGET` for having the target declared central!
* Review comments
* BlockId removal: Client::runtime_version_at
It changes the arguments of `Client::runtime_version_at` from: `BlockId<Block>` to: `Block::Hash`
* Apply suggestions from code review
Co-authored-by: Anton <anton.kalyaev@gmail.com>
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: parity-processbot <>
* Removed assumptions about ancestry from fork tree prune method
* Tests improvement
* Fork tree prune refactory
* Code refactory
* Correctly handle borderline, but legit, case
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Removed duplicated test
---------
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* feat: improve prove_finality api and export it
* fmt
* fix
* improve prove_finality and kept private
* Update client/finality-grandpa/src/finality_proof.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* add `prove_finality_proof` to `FinalityProofProvider`
* fix some and impl Clone for FinalityProofProvider
* improve by suggestions
---------
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* sc-client-db: Fix `PruningMode::ArchiveCanonical`
When running a node with `--state-pruning archive-canonical` it was directly failing on genesis.
There was an issue in the state-db `pin` implementation. It was not checking the state of a block
correctly when running with archive canonical (and also not for every other block after they are canonicalized).
* FMT
For on-demand justifications, peer selection is based on witnessed
gossip votes. This commit changes the condition for selecting a peer
to request justification for `block` from
"last voted on >= `block`" to "peer last voted on strict > `block`".
When allowing `>=` we see nodes continuously spamming unsuccessful
on-demand requests to nodes which are still voting on a block without
having a justification available.
One way to fix the spam would be to add some rate-limiting or backoff
period when requesting justifications.
The other solution (present in this commit) is to simply request
justifications from peers that are voting on future blocks so we know
they're _guaranteed_ to have the wanted mandatory justification
available to send back.
Signed-off-by: acatangiu <adrian@parity.io>
* grandpa: don't error if best block and finality target are inconsistent
* grandpa: add test for best block override
* grandpa: make clippy happy
* grandpa: log selectchain override as debug instead of warn
* Finalization target should be chosed as some ancestor of SelectChain::best_chain
* More test assertions
* Improve docs
* Removed stale docs
* Rename 'target' to 'base' in lookup method
* Fix typo
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Rename 'target_hash' to 'base_hash' in 'SelectChain::finality_target()'
* Apply suggestions from code review
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Docs improvement
* Doc fix
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* Apply more code suggestions
---------
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
* Bump `wasmtime` to 4.0.0 (and a few other deps)
* Use `Error::msg` instead of `anyhow!`
* Bump `wasmtime` to 5.0.0
* Update `Cargo.lock`
* Add `wasmtime` feature to `sp-wasm-interface` dependency
* client/beefy: detect equivocated votes
* client/beefy: make sure to persist state after voting
* client/beefy: drop never-used aux-schema v2 migration
* impl review suggestion
---------
Signed-off-by: Adrian Catangiu <adrian@parity.io>
André Silva