Commit Graph

1713 Commits

Author SHA1 Message Date
Nazar Mokrynskyi 5896072b86 Network sync refactoring (part 4) (#11412)
* Remove direct dependency of `sc-network` on `sc-network-light`

* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`

* Move `WarpSyncProvider` trait and surrounding data structures into `sc-network-common`

* Create `sync` module in `sc-network-common`, create `ChainSync` trait there (not used yet), move a bunch of associated data structures from `sc-network-sync`

* Switch from concrete implementation to `ChainSync` trait from `sc-network-common`

* Introduce `OpaqueStateRequest`/`OpaqueStateResponse` to remove generics from `StateSync` trait

* Introduce `OpaqueBlockRequest`/`OpaqueBlockResponse`, make `scheme` module of `sc-network-sync` private

* Surface `sc-network-sync` into `sc-service` and make `sc-network` not depend on it anymore

* Remove now unnecessary dependency from `sc-network`

* Replace crate links with just text since dependencies are gone now

* Remove `warp_sync` re-export from `sc-network-common`

* Update copyright in network-related files

* Address review comments about documentation

* Apply review suggestion

* Rename `extra_requests` module to `metrics`

Co-authored-by: Bastian Köcher <info@kchr.de>
2022-07-12 20:34:17 +00:00
Roman 1902dc169d Upgrade libp2p to 0.46.1 (#11787)
* Update libp2p to 0.46.0

* Update libp2p to 0.46.1

* Fix telemetry initialization

* Fix tests
2022-07-12 17:00:00 +00:00
Xiliang Chen 47b27c292f CLI flag to configure tx ban duration (#11786)
* add tx-ban-seconds

* fix

* trigger CI

* trigger CI

* remove test print

* Update client/cli/src/params/transaction_pool_params.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2022-07-12 11:19:13 +00:00
Arkadiy Paronyan a1c60f0909 sync: Fixed clearing subsequent ranges (#11815)
* sync: Fixed clearing subsequent ranges

* Warp sync fix

* Better documentation
2022-07-12 11:14:42 +00:00
Andronik c9fd6b8e65 expose grandpa BeforeBestBlockBy voting rule (#11798) 2022-07-08 12:27:58 +00:00
Arkadiy Paronyan 64ebc12a3f Fix clearing queued blocks in the sync module (#11763) 2022-07-07 14:24:06 +00:00
Koute 3d89fa3605 Prevent unsoundness in environments with broken madvise(MADV_DONTNEED) (#11722)
* Prevend unsoundness in environments with broken `madvise(MADV_DONTNEED)`

* Add the `std` feature to `rustix` dependency

Apparently not having this breaks compilation on non-nightly toolchains.

* Autodetect the page size when checking whether `madvise` works

* Only make sure that the madvice check doesn't return `Err`
2022-06-28 06:54:56 +00:00
Nathan Whitaker 421b4b63d0 Avoid a duplicate block request when syncing from a fork (#11094)
* Separate queueing blocks for import from removal

* Add regression tests

* Remove unnecessary log

* Clear queued blocks when processed

* Move check out of match block

* Track queued block ranges

* Update client/network/sync/src/blocks.rs

* Update client/network/sync/src/blocks.rs

* Update client/network/sync/src/blocks.rs

* Update client/network/sync/src/blocks.rs

* FMT

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
2022-06-25 00:21:07 +02:00
Koute 025e1d76f1 Bump wasmtime to 0.38.0 and zstd to 0.11.2 (#11720)
* Bump `wasmtime` to 0.37.0 and `zstd` to 0.11.2

* Bump `wasmtime` to 0.38.0
2022-06-24 11:25:21 +00:00
André Silva 7080662f8f epochs: don't use gap when there's at least one genesis epoch imported (#11725)
* epochs: don't use gap when there's at least one genesis epoch imported

* epochs: add test for genesis gap fix
2022-06-22 16:03:03 +00:00
Davide Galassi c189bd15de More robust grandpa revert procedure (#11719)
* More robust revert procedure

Return an error if revert is called in a node that is not actively
running grandpa, i.e. grandpa genesis data has not been initialized.
Previous implementation was just firing an `unreachable!` code exception.

Furthermore we skip revert hassle if there is nothing to revert.

* Nit
2022-06-21 09:50:51 +00:00
Koute 9cee3c699d Pump the gossip engine while waiting for the BEEFY runtime pallet (memory leak fix) (#11694)
* Pump the gossip engine while waiting for the BEEFY runtime pallet

This fixes a memory leak when the BEEFY gadget is turned on, but
the runtime doesn't actually use BEEFY.

* Implement `FusedFuture` for `GossipEngine`

* Fuse futures outside of loops
2022-06-20 15:52:43 +00:00
Bastian Köcher 94976af3d5 Update syn and fix compilation (#11707)
* Update syn and fix compilation

* Bump pin-project
2022-06-20 09:53:41 +00:00
Zhenghao Lu eba0396182 Simplified code using existing APIs (#11702)
Signed-off-by: Emison Lu <lzh1633856298@gmail.com>
2022-06-18 23:56:48 +00:00
yjh 3f549bad6c chore: reducing codec times (#11688) 2022-06-18 00:14:50 +02:00
Roman 1988c4ac1d Upgrade to libp2p 0.45.1 (#11682)
* Upgrade to libp2p 0.45.1

* Limit max_negotiating_inbound_streams to 512

* Upgrade prost-build to 0.10

* Set max_negotiating_inbound_streams to 2048

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Fix authority discovery protobuf

* Fix comments in authority-discovery schema

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Add a comment about transport initialization

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2022-06-17 13:47:14 +00:00
ZhiYong e252c274c1 Add Event to Pallet Transaction Payment (#11618)
* add Event to Pallet Transaction Payment

* Fix tests in Pallet Balance

* Fix tests in Pallet Balance/Executive/Asset-tx-payment.

* Fix

* fmt

* Fix

* Fix

* Update Cargo.lock

* Fix tests in executor

* Update frame/transaction-payment/src/lib.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* update the name of the event and fmt.

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2022-06-16 01:30:49 +00:00
Bastian Köcher c013f0d6c1 Include the chain specs instead of trying to open them by path (#11625)
This makes it possible to run the tests manually, without them expecting to be run in a special
folder etc.
2022-06-14 23:22:16 +02:00
Niklas Adolfsson bed163dceb rpc servers: update jsonrpsee to fix host filtering + WS server-side pings (#11661)
* bump jsonrpsee to fix #11480

In addition it adds WebSocket server-side pings which is configured to
send out pings periodically every 30 seconds.

* use released crates.io version

* Update Cargo.toml
2022-06-14 20:43:25 +00:00
Nazar Mokrynskyi f9ea8b8d0f Make it possible to disable RocksDB completely (#11537)
* Make it possible to disable RocksDB completely

* Make ParityDB non-optional

* Address review comments
2022-06-14 16:55:56 +00:00
André Silva 06cf8ad076 grandpa: fix creation of justification with equivocating precommits in commit (#11302)
* grandpa: fix creation of justification ancestry

we would reject commits that have precommits targeting blocks lower than the
commit target. when there is an equivocation (or if it the commit is not
minimal) it is possible to have such precommits and we should assume that they
are the round base.

* grandpa: bump to 0.16.0

* grandpa: add test for justification with equivocation

* grandpa: fix failing test
2022-06-14 10:41:11 +00:00
Niklas Adolfsson e9e6916596 remove flaky rpc subscription tests (#11653) 2022-06-14 10:17:58 +02:00
Nazar Mokrynskyi e0e34ad45e Better transaction pool error (#11629)
* Better transaction pool error

* Do not use `Debug` when printing errors
2022-06-10 08:58:55 +00:00
Davide Galassi f2d578b678 Improve inspection and generation of node keys (#11525)
* Improve inspection and generation of node keys

* Lock stdout before write

* Fix typo

* Fix offset

* Remove useless code

* Set inspect-node-key 'network' option as obsolete
2022-06-09 19:55:02 +02:00
Nazar Mokrynskyi 9765f200cb Improve docs on --keep-blocks CLI parameter and related data structures (#11611)
* Improve docs on `--keep-blocks` CLI parameter and related data structures

* Update client/db/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-06-08 10:48:46 +02:00
Liu-Cheng Xu 6b1e871137 Expose ValidatedTransaction from transaction pool (#11588)
This is required to make a tx pool wrapper for some custom transaction validation, specificially
required to make a wrapper around the `LocalTransactionPool` implementation (https://github.com/subspace/subspace/blob/f54881a9b5/crates/subspace-service/src/pool.rs#L232-L269).

Related: https://github.com/paritytech/substrate/discussions/11520
2022-06-03 14:33:59 +02:00
Bastian Köcher dbf497f06d Sync: Improve major sync detection (#11547)
* Sync: Improve major sync detection

When we still have a full import queue, we should still report that we are in major sync mode,
otherwise validators may will already start producing blocks.

* Use median

* Review comments
2022-05-31 19:29:00 +01:00
Gavin Wood ecbd65fb95 Safe and sane multi-item storage removal (#11490)
* Fix overlay prefix removal result

* Second part of the overlay prefix removal fix.

* Report only items deleted from storage in clear_prefix

* Fix kill_prefix

* Formatting

* Remove unused code

* Fixes

* Fixes

* Introduce clear_prefix host function v3

* Formatting

* Use v2 for now

* Fixes

* Formatting

* Docs

* Child prefix removal should also hide v3 for now

* Fixes

* Fixes

* Formatting

* Fixes

* apply_to_keys_whle takes start_at

* apply_to_keys_whle takes start_at

* apply_to_keys_whle takes start_at

* Cursor API; force limits

* Use unsafe deprecated functions

* Formatting

* Fixes

* Grumbles

* Fixes

* Docs

* Some nitpicks 🙈

* Update primitives/externalities/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Formatting

* Fixes

* cargo fmt

* Fixes

* Update primitives/io/src/lib.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Formatting

* Fixes

Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2022-05-29 12:56:26 +01:00
Sergej Sakac ccbab1beea Fixed pruning docs (#11519) 2022-05-28 11:55:51 +02:00
Davide Galassi 35af8fd726 Fix Babe revert when last finalized block is a leaf (#11500)
* Fix Babe revert when a leaf is the last finalized block

Without this fix the last finalized block weight data is wrongly removed
on revert scenario where the last finalized block is a leaf.

* Remove redundant check

* Added test to exercise the fix

* Rename test

* Give variables better names
2022-05-24 17:24:55 +00:00
Kian Paimani 494167a05b Optimize offchain worker memory usage a bit. (#11454)
* add missing events to elections fallback

* Merged

* add some logs and stuff

* undo a bunch of things

* undo lock file

* remove unused err

* fix build
2022-05-24 06:23:05 +00:00
Koute 4f393171d7 Adjust maximum memory pages hard limit for the pooling instantiation strategy (#11482)
* Run `sc-executor-wasmtime` unit tests for all instantiation strategies

* Adjust maximum memory pages hard limit for the pooling instantiation strategy
2022-05-24 05:37:04 +00:00
Bastian Köcher 7ff7457f4b Make --dev listen by default on /ip4/0.0.0.0/tcp/30333 (#11492)
If `--validator` is passed we also listen on this address and as `--dev` is a shortcut for multiple
CLI args, including `--validator`, we should make it consistent.
2022-05-22 09:19:07 +02:00
Niklas Adolfsson c75474f7b8 rpc servers CLI: add --max--subscriptions--per--connection + fix a few bugs (#11461)
* cli: fix RPC CLI nits

* remove needless lines

* cargo fmt

* Update client/service/src/lib.rs

Co-authored-by: James Wilson <james@jsdw.me>

Co-authored-by: James Wilson <james@jsdw.me>
2022-05-21 08:45:57 +00:00
Qinxuan Chen 5497069d54 Unify rpc api and implementation name (#11469)
* Unify rpc api and implementation name

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* MauanlSeal ==> ManualSealRpc

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Remove extra Rpc naming in the structs

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Update doc

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* fix merge

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-05-21 08:13:09 +02:00
Alexandru Vasile 87de1e7e0d RPC: Mark storage methods as blocking (#11459)
* client/api: Make `storage_keys` blocking

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* client/api: Ensure `state_*` RPC methods are blocking

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* client/rpc: Ensure `childstate_*` RPC methods are blocking

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* client/rpc: `ChainApi` make RPC methods sync

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove unused async-traits

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* client/rpc-api: Make chain RPC methods blocking

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/rpc/src/state/state_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add `blocking` to `state_getKeysPaged` RPC call

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix build and warning

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove `async_trait` tidyup

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-05-20 19:16:35 +00:00
Koute dd854c16e2 Switch to pooling copy-on-write instantiation strategy for WASM (#11232)
* Switch to pooling copy-on-write instantiation strategy for WASM

* Fix benchmark compilation

* Fix `cargo fmt`

* Fix compilation of another benchmark I've missed

* Cleanups according to review comments

* Move `max_memory_size` to `Semantics`

* Set `memory_guaranteed_dense_image_size` to `max_memory_size`

* Rename `wasm_instantiation_strategy` to `wasmtime_instantiation_strategy`

* Update the doc-comments regarding the instantiation strategy

* Extend the integration tests to test every instantiation strategy

* Don't drop the temporary directory until the runtime is dropped in benchmarks

* Don't drop the temporary directory until the runtime is dropped in tests
2022-05-19 07:32:53 +00:00
Nathan Whitaker 3502bc5c1d Return a successful response on repeated small block request (#11429) 2022-05-18 09:42:53 +02:00
Qinxuan Chen fe4acb7924 Prune some duplicated dependencies in the dep graph (#11433)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2022-05-18 07:23:15 +00:00
Nazar Mokrynskyi 558daec697 Remove unnecessary RPC boxing (#11434) 2022-05-17 20:12:59 +00:00
Niklas Adolfsson 96006322bb fix: regression of sync_state_genSyncSpec #11435 (#11437)
* fix: #11435

* address grumbles: better safe than sorry
2022-05-17 16:29:21 +00:00
Oliver Tale-Yazdi 32d6d1b37c More benchmark machine args (#11428)
* Add ExecutionLimits to sc-sysinfo and return float

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Increase benchmarking duration and add options

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2022-05-17 13:57:08 +00:00
Shawn Tabrizi 101debbffa remove deprecated (#11432) 2022-05-17 09:50:03 +00:00
dependabot[bot] 83b774fa7f Bump prost from 0.9.0 to 0.10.3 (#11425)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.9.0 to 0.10.3.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.9.0...v0.10.3)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 13:09:12 +02:00
yjh bf694d3c0e remove unused error InvalidCode and improve docs (#11383) 2022-05-16 10:25:34 +02:00
dependabot[bot] a5d0a65bf4 Bump scale-info from 2.0.1 to 2.1.1 (#11420)
Bumps [scale-info](https://github.com/paritytech/scale-info) from 2.0.1 to 2.1.1.
- [Release notes](https://github.com/paritytech/scale-info/releases)
- [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/scale-info/commits/v2.1.1)

---
updated-dependencies:
- dependency-name: scale-info
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 10:07:00 +02:00
dependabot[bot] 4a3758d9b8 Bump log from 0.4.16 to 0.4.17 (#11369)
* Bump log from 0.4.16 to 0.4.17

Bumps [log](https://github.com/rust-lang/log) from 0.4.16 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits/0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-05-15 22:21:04 -04:00
dependabot[bot] ebc4c4417a Bump clap from 3.1.6 to 3.1.18 (#11376)
* Bump clap from 3.1.6 to 3.1.17

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.6 to 3.1.17.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.6...v3.1.17)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update utils/frame/try-runtime/cli/Cargo.toml

* Update frame/election-provider-support/solution-type/fuzzer/Cargo.toml

* use 3.1.18

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-05-15 21:37:23 -04:00
Nazar Mokrynskyi 8a45870878 Network sync refactoring (part 3) (#11347)
* Move `light.v1.proto` schema into new crate `sc-network-light`

* Move `sc_network::light_client_requests` and submodule to `sc_network_light::light_client_requests`

* Fix apparently outdated reference in documentation and visibility modifier

* Fix rustdoc check

* Update lock file
2022-05-14 07:17:53 +00:00
Bastian Köcher 9cb699bfe9 Make clippy happy on latest nightly (#11403)
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
2022-05-12 14:02:29 +03:00