Commit Graph

403 Commits

Author SHA1 Message Date
Anton f2dcd9520c upgrade libp2p to 0.50.0 (#12734)
* upgrade libp2p to 0.50.0

* on_swarm_event and on_connection_handler_event

* replace `Swarm::new` with `Swarm::with_threadpool_executor`

* on_swarm_event and on_connection_handler_event part 2

* on_swarm_event and on_connection_handler_event part 3

* on_swarm_event and on_connection_handler_event part 4

* update libp2p

* libp2p 0.50.0

* rename OutboundQueryCompleted to OutboundQueryProgressed

refs https://github.com/libp2p/rust-libp2p/pull/2712

* remove unused var

* accumulate outbound_query_records until query is finished

* format code

* use p_handler instead of new_handler

https://github.com/paritytech/substrate/pull/12734#discussion_r1027640610

* pass ListenFailure to kademlia

https://github.com/paritytech/substrate/pull/12734#discussion_r1034716664

* use tokio executor in tests

https://github.com/paritytech/substrate/pull/12734#discussion_r1039291776

* use chrono Local::now

instead of deprecated Local::today

* remove unused vars from request_responses tests

* attempt to fix pallet UI tests

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI
2023-01-05 12:03:41 +00:00
Michal Kucharczyk 9c69fc1b32 BlockId removal: refactor: BlockBackend::block|block_status (#13014)
* BlockId removal: refactor: BlockBackend::block|block_status

It changes the arguments of:
-  `BlockBackend::block`
-  `BlockBackend::block_status`

method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* non-obvious reworks

* doc fix

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: parity-processbot <>
2023-01-02 09:42:05 +00:00
Bastian Köcher 0a94112c9d Print "stalled" task on shutdown (#13022)
* Print "stalled" task on shutdown

When the node is shutting down, we give the Tokio runtime 60 seconds to shutdown. If after these 60
seconds there are still running tasks, we now print these tasks. This should help debugging nodes
that have stalled tasks.

This pr introduces a `TaskRegistry` that keeps track of all running tasks. Each task registers and
unregisters itself in this `TaskRegistry`.

* Fix rustdoc

* Update client/service/src/lib.rs
2022-12-28 09:16:52 +01:00
Bastian Köcher 9f5ed21fe9 Make CLI state pruning optional again (#13017)
* Make CLI state pruning optional again

The state pruning setting is stored in the database when it is created. In later runs it is fine to
drop the `--state-pruning` CLI argument as the setting is stored in the database. The state db will
only return an error if the stored state pruning doesn't match the state pruning given via CLI.

Recently we improved the state pruning CLI handling and accidentally made the state pruning value
always present (as we set some default value for the clap). If we could find out if a user has
passed a value or the default value was taken, we could keep the default value in the CLI interface,
but clap isn't supporting this right now. So, we need to go back and make `state_pruning` an
optional with the default written into the docs.

It also adds a test to ensure that we don't break this behavior again.

* More docs
2022-12-26 17:37:09 +01:00
Qinxuan Chen 40c04cf11d *: update rand to v0.8.5 (#12962)
* *: update rand to v0.8.5

* *: remove useless deps from Cargo.toml

* fix pallet-session-benchmarking

* fix pallet-election-provider-support test

* remove useless rand from dev-dependencies
2022-12-19 07:38:51 +01:00
bernardo 88feb686ee improve ocw validator/collator CLI description (#12931)
* improve ocw validator/collator CLI description

* rename WhenValidating to WhenAuthoring

* Update client/cli/src/arg_enums.rs

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
2022-12-16 09:49:36 +01:00
Bastian Köcher 2f2e481d72 Ensure that we inform all tasks to stop before starting the 60 seconds shutdown (#12897)
* Ensure that we inform all tasks to stop before starting the 60 seconds shutdown

The change of waiting in maximum 60 seconds for the node to shutdown actually introduced a bug. We
were actually waiting always 60 seconds as we didn't informed our tasks to shutdown. The solution to
this problem is to drop the task manager as this will then inform all tasks to end. It also adds
tests to ensure that the behaviors work as expected. (This should already have been done in the
first pr! :()

* ".git/.scripts/fmt.sh" 1

Co-authored-by: command-bot <>
2022-12-10 13:28:36 +00:00
Bastian Köcher 487ed143df Don't indefinitely block on shutting down Tokio (#12885)
* Don't indefinitely on shutting down Tokio

Now we wait in maximum 60 seconds before we shutdown the node. Tasks are may be leaked and leading
to some data corruption.

* Drink less :thinking_face:
2022-12-09 10:31:57 +00:00
Alexandru Vasile 8a24e27377 cli: Improve pruning documentation (#12819)
* cli: Improve pruning documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Keep `finalized` notation and remove `canonical` one

* cli: Fix cargo doc

* cli: `PruningModeClap` IR enum

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Convert PruningModeClap into pruning modes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Use `PruningModeClap`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Rename to `DatabasePruningMode`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Implement `FromStr` instead of `clap::ValueEnum`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/cli/src/params/pruning_params.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add option documentation back

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply suggestions from code review

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2022-12-08 20:15:14 +00:00
Dmitry Markin 5eb84f9cc6 Upgrade tokio to 1.22.0 and replace async-std with tokio (#12646)
* Replace deprecated libp2p feature specs with correct ones

* Bump tokio to 1.21.2

* Replace async-std libp2p primitives with tokio ones

* minor: rustfmt

* Fix TestNet to run initialization in the tokio context

* Convert telemetry test from async-std to tokio

* Convert notifications tests from async-std to tokio

* Convert chain sync tests from async-std to tokio

* Ditch async-std completely

* Make executor mandatory

* Bump tokio to 1.22.0

* minor: rustfmt

* Explicitly use tokio runtime in tests

* Move more tests to explicit tokio runtime

* Explicitly set multithreaded runtime in tokio test

* minor: rustfmt

* minor: fix comment

* Replace async-std with tokio in MMR tests
2022-12-05 08:18:46 +00:00
Oliver Tale-Yazdi 7ed2bc7258 Fix rustdoc (#12777)
* Fix table formatting

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix sp-runtime-interface table

Using HTML now since multi-line tables are not a thing and fmt
destroys them.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More rustdoc fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tags

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More fixes...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use Bastis patch

Co-authored-by: Bastian Köcher <git@kchr.de>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more backticks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* change ci image

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-11-25 15:13:38 +01:00
Koute 4214bc527c Remove the wasmtime feature flag (#12684)
* Remove the `wasmtime` feature flag

* rustfmt
2022-11-18 14:21:44 +01:00
Andrew Jones d5a5f3673e Release sp-keyring and pallet-contracts-primitives 7.0.0 (#12716)
* Bump sp-keyring

* Bump pallet-contracts-primitives

* Cargo.lock
2022-11-16 10:07:56 +00:00
Niklas Adolfsson 2b8af8cb1a release sp-core 7.0.0 and sp-runtime 7.0.0 (#12599)
* chore(release): sp-core v7.0.0

* chore(release): sp-runtime v7.0.0

* fix bad merge
2022-11-15 14:54:14 +00:00
Michal Kucharczyk 1ed70004e7 BlockId removal: &Hash to Hash (#12626)
It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
2022-11-07 21:42:16 +00:00
Bastian Köcher 81f123b72e Make --db case insensitive again (#12630)
This was broken in the switch to Clap v4.
2022-11-07 14:01:23 +01:00
dependabot[bot] be6f51895b Bump regex from 1.5.5 to 1.6.0 (#12117)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.6.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parity-processbot <>
2022-11-07 10:17:53 +00:00
Sebastian Kunert 096553cb59 Use ArgAction::Set for enable-offchain-indexing flag (#12521)
* Use ArgAction::Set for enable-offchain-indexing

* Provide default value for `enable-offchain-indexing`
2022-10-18 15:41:19 +00:00
Michal Kucharczyk b16135f602 BlockId removal: refactor: StorageProvider (#12510)
* BlockId removal: refactor: StorageProvider

It changes the arguments of `Backend::StorageProvider` trait from:
block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/api/src/backend.rs

Co-authored-by: Dmitrii Markin <dmitry@markin.tech>

* GrandpaBlockImport::current_set_id reworked

* ExportStateCmd reworked

* trigger CI job

* trigger CI job

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Dmitrii Markin <dmitry@markin.tech>
2022-10-18 12:52:04 +00:00
Pierre Krieger 8671fb7615 Decrease default --out-peers from 25 to 15 (#12434)
Co-authored-by: Bastian Köcher <git@kchr.de>
2022-10-18 10:46:25 +02:00
Sebastian Kunert f687db40f7 Bump clap to 4.0.x and adjust to best practices (#12381)
* Bump clap to 3.2.22

* Replace `from_os_str` with `value_parser`

* Replace `from_str` and `try_from_str` with `value_parser`

* Move possible_values to the new format

* Remove unwanted print

* Add missing match branch

* Update clap to 4.0.9 and make it compile

* Replace deprecated `clap` macro with `command` and `value`

* Move remaining `clap` attributes to `arg`

* Remove no-op value_parsers

* Adjust value_parser for state_version

* Remove "deprecated" feature flag and bump to 4.0.11

* Improve range

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions

* Trigger CI

* Fix unused error warning

* Fix doc errors

* Fix ArgGroup naming conflict

* Change default_value to default_value_t

* Use 1.. instead of 0..

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-10-18 06:52:46 +00:00
Dmitrii Markin bb175f0373 Upgrade libp2p to 0.49.0 (#12256)
* cargo upgrade libp2p

* Get rid of `NetworkBehaviourEventProcess` in handling of `CustomMessageOutcome`

* Get rid of `NetworkBehaviourEventProcess` in handling of `request_responses::Event`

* Get rid of `NetworkBehaviourEventProcess` in handling of `peer_info::PeerInfoEvent`

* Get rid of `NetworkBehaviourEventProcess` in handling of `DiscoveryOut`

* Get rid of `poll()` method in `Bahaviour`

* minor: comments

* Upgrade libp2p to 0.49.0 (unreleased)

* Support multiple Kad protocol names

* Make borrow checker happy

* minor: wording

* Make substrate build with libp2p-0.49.0

* rustfmt

* Get rid of MdnsWrapper

* Resolve deprecation warnings

* Fix documentation

* Apply suggestions from code review: fix typos

Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>

* Apply suggestion: simplify kad protocol name matching

Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
2022-10-17 11:02:37 +00:00
Koute 1b3f211efa Enable the wasmtime-based WASM executor by default (#12486) 2022-10-13 10:31:00 +00:00
Michal Kucharczyk 5e00d361cc unsafe_pruning flag removed (#12385) 2022-09-30 12:06:46 +02:00
Aaro Altonen 4c19c13d05 Move transactions protocol to its own crate (#12264)
* Move transaction protocol to its own crate

* Update Cargo.lock

* Fix binaries

* Update client/network/transactions/src/lib.rs

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

* Update client/service/src/builder.rs

Co-authored-by: Bastian Köcher <info@kchr.de>

* Apply review comments

* Revert one change and apply cargo-fmt

* Remove Transaction from Message

* Add array-bytes

* trigger CI

* Add comment about codec index

Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Bastian Köcher <info@kchr.de>
2022-09-26 12:10:09 +00:00
ZhiYong 11fa9af104 Remove discarded blocks and states from database by default (#11983)
* 1.Add pruning param "canonical" in sc-cli.
2.Make PruningMode's default value to ArchiveCanonical.

* Update tests in sc-state-db.

* Update tests in sc-state-db.

* 1.Add a new value `AllWithNonFinalized` in `enum BlocksPruning` which Corresponds to `blocks_pruning 0` in CLI .
2.Change value `All` to `AllFinalized` in `enum BlocksPruning` and make it to keep full finalized block history.

* Make some corresponding adjustments based on the content in the conversation.

* Update client/db/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review.

* 1.Change `blocks_pruning` to be like `state_pruning` .

* Fmt and add some doc.

* Update client/cli/src/params/pruning_params.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/cli/src/params/pruning_params.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update doc.

* Change `new_test_with_tx_storage` to take `BlocksPruning`.

* Fmt

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-09-26 07:46:59 +00:00
Xavier Lau 86198c5471 Use array-bytes for All Array/Bytes/Hex Operations (#12190)
* Use `array-bytes` for All Array/Bytes/Hex Operations

Signed-off-by: Xavier Lau <xavier@inv.cafe>

* Reorder

* Self Review

* Format

* Fix Tests

* Bump `array-bytes`

* Optimize large test res

Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
2022-09-21 06:12:20 +00:00
Bastian Köcher 907144496e Do not drop the task_manager for benchmarking stuff (#12147)
We can not drop the `task_manager` for benchmarking stuff, because otherwise stuff that may needs
this feature (like background signature verification) will fail. Besides the base path setup is
moved to `SharedParams` directly. Meaning any call to `base_path` will now directly return a tmp
path when `--dev` is given.
2022-09-02 07:27:00 +00:00
dependabot[bot] 75a76d967b Bump serde_json from 1.0.79 to 1.0.85 (#12079)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.79 to 1.0.85.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.79...v1.0.85)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-26 09:32:20 +02:00
Bastian Köcher 73d9ae3284 Introduce trie level cache and remove state cache (#11407)
* trie state cache

* Also cache missing access on read.

* fix comp

* bis

* fix

* use has_lru

* remove local storage cache on size 0.

* No cache.

* local cache only

* trie cache and local cache

* storage cache (with local)

* trie cache no local cache

* Add state access benchmark

* Remove warnings etc

* Add trie cache benchmark

* No extra "clone" required

* Change benchmark to use multiple blocks

* Use patches

* Integrate shitty implementation

* More stuff

* Revert "Merge branch 'master' into trie_state_cache"

This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.

* Improve benchmark

* Adapt to latest changes

* Adapt to changes in trie

* Add a test that uses iterator

* Start fixing it

* Remove obsolete file

* Make it compile

* Start rewriting the trie node cache

* More work on the cache

* More docs and code etc

* Make data cache an optional

* Tests

* Remove debug stuff

* Recorder

* Some docs and a simple test for the recorder

* Compile fixes

* Make it compile

* More fixes

* More fixes

* Fix fix fix

* Make sure cache and recorder work together for basic stuff

* Test that data caching and recording works

* Test `TrieDBMut` with caching

* Try something

* Fixes, fixes, fixes

* Forward the recorder

* Make it compile

* Use recorder in more places

* Switch to new `with_optional_recorder` fn

* Refactor and cleanups

* Move `ProvingBackend` tests

* Simplify

* Move over all functionality to the essence

* Fix compilation

* Implement estimate encoded size for StorageProof

* Start using the `cache` everywhere

* Use the cache everywhere

* Fix compilation

* Fix tests

* Adds `TrieBackendBuilder` and enhances the tests

* Ensure that recorder drain checks that values are found as expected

* Switch over to `TrieBackendBuilder`

* Start fixing the problem with child tries and recording

* Fix recording of child tries

* Make it compile

* Overwrite `storage_hash` in `TrieBackend`

* Add `storage_cache` to  the benchmarks

* Fix `no_std` build

* Speed up cache lookup

* Extend the state access benchmark to also hash a runtime

* Fix build

* Fix compilation

* Rewrite value cache

* Add lru cache

* Ensure that the cache lru works

* Value cache should not be optional

* Add support for keeping the shared node cache in its bounds

* Make the cache configurable

* Check that the cache respects the bounds

* Adds a new test

* Fixes

* Docs and some renamings

* More docs

* Start using the new recorder

* Fix more code

* Take `self` argument

* Remove warnings

* Fix benchmark

* Fix accounting

* Rip off the state cache

* Start fixing fallout after removing the state cache

* Make it compile after trie changes

* Fix test

* Add some logging

* Some docs

* Some fixups and clean ups

* Fix benchmark

* Remove unneeded file

* Use git for patching

* Make CI happy

* Update primitives/trie/Cargo.toml

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update primitives/state-machine/src/trie_backend.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Introduce new `AsTrieBackend` trait

* Make the LocalTrieCache not clonable

* Make it work in no_std and add docs

* Remove duplicate dependency

* Switch to ahash for better performance

* Speedup value cache merge

* Output errors on underflow

* Ensure the internal LRU map doesn't grow too much

* Use const fn to calculate the value cache element size

* Remove cache configuration

* Fix

* Clear the cache in between for more testing

* Try to come up with a failing test case

* Make the test fail

* Fix the child trie recording

* Make everything compile after the changes to trie

* Adapt to latest trie-db changes

* Fix on stable

* Update primitives/trie/src/cache.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Fix wrong merge

* Docs

* Fix warnings

* Cargo.lock

* Bump pin-project

* Fix warnings

* Switch to released crate version

* More fixes

* Make clippy and rustdocs happy

* More clippy

* Print error when using deprecated `--state-cache-size`

* 🤦

* Fixes

* Fix storage_hash linkings

* Update client/rpc/src/dev/mod.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Review feedback

* encode bound

* Rework the shared value cache

Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.

* Improve of the cache by merging the old and new solution

* FMT

* Please stop coming back all the time :crying:

* Update primitives/trie/src/cache/shared_cache.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Fixes

* Make clippy happy

* Ensure we don't deadlock

* Only use one lock to simplify the code

* Do not depend on `Hasher`

* Fix tests

* FMT

* Clippy 🤦

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
2022-08-18 18:59:22 +00:00
Squirrel a68a80fbae declone and close the door (#12035)
* declone and close the door

* cargo fmt

* remove brackets
2022-08-15 19:38:36 +00:00
dependabot[bot] 9c2a2495fe Bump rpassword from 5.0.1 to 7.0.0 (#11826)
* Bump rpassword from 5.0.1 to 7.0.0

Bumps [rpassword](https://github.com/conradkleinespel/rpassword) from 5.0.1 to 7.0.0.
- [Release notes](https://github.com/conradkleinespel/rpassword/releases)
- [Commits](https://github.com/conradkleinespel/rpassword/compare/v5.0.1...v7.0.0)

---
updated-dependencies:
- dependency-name: rpassword
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Use new API

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2022-08-15 07:57:15 +00:00
lucasvanmol 98252cb900 Fix broken links in polkadot --help (#12010)
https://docs.substrate.io/v3/runtime/custom-rpcs/#public-rpcs > https://docs.substrate.io/main-docs/build/custom-rpc/#public-rpcs
2022-08-11 13:13:49 +00:00
Nikos Kontakis 20c49b20a7 Rename --pruning and --keep-blocks to be more similar to one another (#11934)
* rename prunning and keep-blocks flags

* Add aliases in keep-blocks and pruning for backward compatibility

* Rename in code variables from  and  to  and
2022-08-08 09:31:26 +00:00
Sebastian Kunert 9f409dc0b8 Prepare for rust 1.62.1 (#11903)
* Update UI test output for rust 1.62.1

* switch ci to staging image to check that everything works

* fix artifacts node-bench-regression-guard

* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning

* Fix more clippy lints

* Make clippy happy by relying on auto-deref were possible

* Add tracking issue to the comments

* pin ci image

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-07-26 14:37:05 +02:00
Dmitry Markin b7ecd1af85 Cleanup light client leftovers (#11865)
* Remove --light cli option

* Cleanup light client leftovers

* Remove commented-out code and clean-up more light client leftovers

* Fix formatting with `cargo +nightly fmt`

* Remove FIXME regarding db directory structure

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-07-21 08:36:00 +00:00
Roman 1902dc169d Upgrade libp2p to 0.46.1 (#11787)
* Update libp2p to 0.46.0

* Update libp2p to 0.46.1

* Fix telemetry initialization

* Fix tests
2022-07-12 17:00:00 +00:00
Xiliang Chen 47b27c292f CLI flag to configure tx ban duration (#11786)
* add tx-ban-seconds

* fix

* trigger CI

* trigger CI

* remove test print

* Update client/cli/src/params/transaction_pool_params.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2022-07-12 11:19:13 +00:00
Roman 1988c4ac1d Upgrade to libp2p 0.45.1 (#11682)
* Upgrade to libp2p 0.45.1

* Limit max_negotiating_inbound_streams to 512

* Upgrade prost-build to 0.10

* Set max_negotiating_inbound_streams to 2048

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Fix authority discovery protobuf

* Fix comments in authority-discovery schema

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Add a comment about transport initialization

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2022-06-17 13:47:14 +00:00
Nazar Mokrynskyi f9ea8b8d0f Make it possible to disable RocksDB completely (#11537)
* Make it possible to disable RocksDB completely

* Make ParityDB non-optional

* Address review comments
2022-06-14 16:55:56 +00:00
Davide Galassi f2d578b678 Improve inspection and generation of node keys (#11525)
* Improve inspection and generation of node keys

* Lock stdout before write

* Fix typo

* Fix offset

* Remove useless code

* Set inspect-node-key 'network' option as obsolete
2022-06-09 19:55:02 +02:00
Nazar Mokrynskyi 9765f200cb Improve docs on --keep-blocks CLI parameter and related data structures (#11611)
* Improve docs on `--keep-blocks` CLI parameter and related data structures

* Update client/db/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-06-08 10:48:46 +02:00
Sergej Sakac ccbab1beea Fixed pruning docs (#11519) 2022-05-28 11:55:51 +02:00
Bastian Köcher 7ff7457f4b Make --dev listen by default on /ip4/0.0.0.0/tcp/30333 (#11492)
If `--validator` is passed we also listen on this address and as `--dev` is a shortcut for multiple
CLI args, including `--validator`, we should make it consistent.
2022-05-22 09:19:07 +02:00
Niklas Adolfsson c75474f7b8 rpc servers CLI: add --max--subscriptions--per--connection + fix a few bugs (#11461)
* cli: fix RPC CLI nits

* remove needless lines

* cargo fmt

* Update client/service/src/lib.rs

Co-authored-by: James Wilson <james@jsdw.me>

Co-authored-by: James Wilson <james@jsdw.me>
2022-05-21 08:45:57 +00:00
Koute dd854c16e2 Switch to pooling copy-on-write instantiation strategy for WASM (#11232)
* Switch to pooling copy-on-write instantiation strategy for WASM

* Fix benchmark compilation

* Fix `cargo fmt`

* Fix compilation of another benchmark I've missed

* Cleanups according to review comments

* Move `max_memory_size` to `Semantics`

* Set `memory_guaranteed_dense_image_size` to `max_memory_size`

* Rename `wasm_instantiation_strategy` to `wasmtime_instantiation_strategy`

* Update the doc-comments regarding the instantiation strategy

* Extend the integration tests to test every instantiation strategy

* Don't drop the temporary directory until the runtime is dropped in benchmarks

* Don't drop the temporary directory until the runtime is dropped in tests
2022-05-19 07:32:53 +00:00
dependabot[bot] 4a3758d9b8 Bump log from 0.4.16 to 0.4.17 (#11369)
* Bump log from 0.4.16 to 0.4.17

Bumps [log](https://github.com/rust-lang/log) from 0.4.16 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits/0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-05-15 22:21:04 -04:00
dependabot[bot] ebc4c4417a Bump clap from 3.1.6 to 3.1.18 (#11376)
* Bump clap from 3.1.6 to 3.1.17

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.6 to 3.1.17.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.6...v3.1.17)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update utils/frame/try-runtime/cli/Cargo.toml

* Update frame/election-provider-support/solution-type/fuzzer/Cargo.toml

* use 3.1.18

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-05-15 21:37:23 -04:00
David 29c0c6a4a8 jsonrpsee integration (#8783)
* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-05-10 08:52:19 +00:00
Roman Gafiyatullin 729cba9d9e Remove the --unsafe-pruning CLI-argument (step 1) (#10995)
* sc-client-db: utils::open_database(...) — return OpenDbError so that the caller could tell the `OpenDbError::DoesNotExist` clearly

* sc-client-db: utils::open_database(..) — accept the `create: bool` argument

* sc-client-db: pruning — optional argument in the DatabaseSettings

* sc-state-db: Split `Error<E>` into separate `Error<E>` and `StateDbError`

* StateDb::open: choose the pruning-mode depending on the requested and stored values

* sc-state-db: test for different combinations of stored and requested pruning-modes

* CLI-argument: mark the unsafe-pruning as deprecated

* Fix tests

* tests: do not specify --pruning when running the substrate over the existing storage

* fix types for benches

* cargo fmt

* Check whether the pruning-mode and sync-mode are compatible

* cargo fmt

* parity-db: 0.3.11 -> 0.3.12

* sc-state-db: MetaDb::set_meta — a better doc-test

* cargo fmt

* make MetaDb read-only again!

* Remove the stray newline (and run the CI once again please)

* Last nitpicks

* A more comprehensive error message
2022-05-06 10:07:44 +00:00