# Description
Sometimes changing file descriptor limits is not allowed, but there is
no need to crash the node if/when this happens. Since `fdlimit`'s author
decided to use panics instead of returning `Result`, we need to catch
it.
# Checklist
- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
required)
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
The `BlockBuilderProvider` was a trait that was defined in
`sc-block-builder`. The trait was implemented for `Client`. This
basically meant that you needed to import `sc-block-builder` any way to
have access to the block builder. So, this trait was not providing any
real value. This pull request is removing the said trait. Instead of the
trait it introduces a builder for creating a `BlockBuilder`. The builder
currently has the quite fabulous name `BlockBuilderBuilder` (I'm open to
any better name 😅). The rest of the pull request is about
replacing the old trait with the new builder.
# Downstream code changes
If you used `new_block` or `new_block_at` before you now need to switch
it over to the new `BlockBuilderBuilder` pattern:
```rust
// `new` requires a type that implements `CallApiAt`.
let mut block_builder = BlockBuilderBuilder::new(client)
// Then you need to specify the hash of the parent block the block will be build on top of
.on_parent_block(at)
// The block builder also needs the block number of the parent block.
// Here it is fetched from the given `client` using the `HeaderBackend`
// However, there also exists `with_parent_block_number` for directly passing the number
.fetch_parent_block_number(client)
.unwrap()
// Enable proof recording if required. This call is optional.
.enable_proof_recording()
// Pass the digests. This call is optional.
.with_inherent_digests(digests)
.build()
.expect("Creates new block builder");
```
---------
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
This PR moves syncing-related code from `sc-network-common` to
`sc-network-sync`.
Unfortunately, some parts are tightly integrated with networking, so
they were left in `sc-network-common` for now:
1. `SyncMode` in `common/src/sync.rs` (used in `NetworkConfiguration`).
2. `BlockAnnouncesHandshake`, `BlockRequest`, `BlockResponse`, etc. in
`common/src/sync/message.rs` (used in `src/protocol.rs` and
`src/protocol/message.rs`).
More substantial refactoring is needed to decouple syncing and
networking completely, including getting rid of the hardcoded sync
protocol.
## Release notes
Move syncing-related code from `sc-network-common` to `sc-network-sync`.
Delete `ChainSync` trait as it's never used (the only implementation is
accessed directly from `SyncingEngine` and exposes a lot of public
methods that are not part of the trait). Some new trait(s) for syncing
will likely be introduced as part of Sync 2.0 refactoring to represent
syncing strategies.
Bumps [directories](https://github.com/soc/directories-rs) from 4.0.1 to
5.0.1.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/soc/directories-rs/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Submit the outstanding PRs from the old repos(these were already
reviewed and approved before the repo rorg, but not yet submitted):
Main PR: https://github.com/paritytech/substrate/pull/14014
Companion PRs: https://github.com/paritytech/polkadot/pull/7134,
https://github.com/paritytech/cumulus/pull/2489
The changes in the PR:
1. ChainSync currently calls into the block request handler directly.
Instead, move the block request handler behind a trait. This allows new
protocols to be plugged into ChainSync.
2. BuildNetworkParams is changed so that custom relay protocol
implementations can be (optionally) passed in during network creation
time. If custom protocol is not specified, it defaults to the existing
block handler
3. BlockServer and BlockDownloader traits are introduced for the
protocol implementation. The existing block handler has been changed to
implement these traits
4. Other changes:
[X] Make TxHash serializable. This is needed for exchanging the
serialized hash in the relay protocol messages
[X] Clean up types no longer used(OpaqueBlockRequest,
OpaqueBlockResponse)
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: command-bot <>
* chainHead/api: Make storage/body/call pure RPC methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Add mpsc channel between RPC methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscriptions: Extract mpsc::Sender via BlockGuard
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscriptions: Generate and provide the method operation ID
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Generate `chainHead_body` response
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Generate `chainHead_call` response
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Generate `chainHead_storage` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Propagate responses of methods to chainHead_follow
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust `chainHead_body` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust `chainHead_call` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust `chainHead_call` responses
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Ensure unique operation IDs across methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/events: Remove old method events
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscriptions: Add limit helper
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscription: Expose limits to `BlockGuard`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust testing to ongoing operations
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Make limits configurable via `ChainHeadConfig`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Adjust testing to `ChainHeadConfig`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/tests: Ensure operation limits discards items
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Improve documentation
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Rename `OngoingOperations` -> `LimitOperations`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Rename reserve -> reserve_at_most
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead: Use duration const instead of u64
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* chainHead/subscription: Use tokio::sync::Semaphore for limits
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update client/rpc-spec-v2/src/chain_head/subscription/inner.rs
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Fix std, runtime-benchmarks and try-runtime features
zepter lint propagate-feature --feature try-runtime --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="try-runtime:frame-try-runtime"
zepter lint propagate-feature --feature runtime-benchmarks --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
zepter lint propagate-feature --feature std --left-side-feature-missing=ignore --workspace --fix
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add propagate feature CI check
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Test CI by adding an error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Use --locked
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add help msg
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Test CI by adding an error"
This reverts commit cf4ff6cc0632269b0a109e547686e5e3314b02de.
* Test CI by adding an error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* No newline in help msg
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Test CI by adding an error"
This reverts commit 5daa06ada8e01f5bebafb9d1c76804dd79bc1006.
* Test CI by adding an error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Test CI by adding an error"
This reverts commit ca15de5729507a564f140a10ec2e87b19516ec4c.
* Fix msg
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert back to master
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Re-do with Zepter v0.7.4
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update Zepter to 0.7.4
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Disable rococo try-runtime check
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* More review fixes
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
* update libp2p to 0.52.0
* proto name now must implement `AsRef<str>`
* update libp2p version everywhere
* ToSwarm, FromBehaviour, ToBehaviour
also LocalProtocolsChange and RemoteProtocolsChange
* new NetworkBehaviour invariants
* replace `Vec<u8>` with `StreamProtocol`
* rename ConnectionHandlerEvent::Custom to NotifyBehaviour
* remove DialError & ListenError invariants
also fix pending_events
* use connection_limits::Behaviour
See https://github.com/libp2p/rust-libp2p/pull/3885
* impl `void::Void` for `BehaviourOut`
also use `Behaviour::with_codec`
* KademliaHandler no longer public
* fix StreamProtocol construction
* update libp2p-identify to 0.2.0
* remove non-existing methods from PollParameters
rename ConnectionHandlerUpgrErr to StreamUpgradeError
* `P2p` now contains `PeerId`, not `Multihash`
* use multihash-codetable crate
* update Cargo.lock
* reformat text
* comment out tests for now
* remove `.into()` from P2p
* confirm observed addr manually
See https://github.com/libp2p/rust-libp2p/blob/master/protocols/identify/CHANGELOG.md#0430
* remove SwarmEvent::Banned
since we're not using `ban_peer_id`, this can be safely removed.
we may want to introduce `libp2p::allow_block_list` module in the future.
* fix imports
* replace `libp2p` with smaller deps in network-gossip
* bring back tests
* finish rewriting tests
* uncomment handler tests
* Revert "uncomment handler tests"
This reverts commit 720a06815887f4e10767c62b58864a7ec3a48e50.
* add a fixme
* update Cargo.lock
* remove extra From
* make void uninhabited
* fix discovery test
* use autonat protocols
confirming external addresses manually is unsafe in open networks
* fix SyncNotificationsClogged invariant
* only set server mode manually in tests
doubt that we need to set it on node since we're adding public addresses
* address @dmitry-markin comments
* remove autonat
* removed unused var
* fix EOL
* update smallvec and sha2
in attempt to compile polkadot
* bump k256
in attempt to build cumulus
---------
Co-authored-by: parity-processbot <>
* Start
* More work!
* Moar
* More changes
* More fixes
* More worrk
* More fixes
* More fixes to make it compile
* Adds `NoOffchainStorage`
* Pass the extensions
* Small basti making small progress
* Fix merge errors and remove `ExecutionContext`
* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`
Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.
* Fix compilation
* Register the global extensions inside runtime api instance
* Fixes
* Fix `generate_initial_session_keys` by passing the keystore extension
* Fix the grandpa tests
* Fix more tests
* Fix more tests
* Don't set any heap pages if there isn't an override
* Fix small fallout
* FMT
* Fix tests
* More tests
* Offchain worker custom extensions
* More fixes
* Make offchain tx pool creation reusable
Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.
* Fixes
* Fixes
* Set offchain transaction pool in BABE before using it in the runtime
* Add the `offchain_tx_pool` to Grandpa as well
* Fix the nodes
* Print some error when using the old warnings
* Fix merge issues
* Fix compilation
* Rename `babe_link`
* Rename to `offchain_tx_pool_factory`
* Cleanup
* FMT
* Fix benchmark name
* Fix `try-runtime`
* Remove `--execution` CLI args
* Make clippy happy
* Forward bls functions
* Fix docs
* Update UI tests
* Update client/api/src/execution_extensions.rs
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update client/cli/src/params/import_params.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update client/api/src/execution_extensions.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Pass the offchain storage to the MMR RPC
* Update client/api/src/execution_extensions.rs
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Review comments
* Fixes
---------
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Make block sync service customizable
This change allows user to have their own network block logic, for example, we'd like to disable
the builtin block sync service when using another syncing approach.
* Remove unnecessary Box
* Revert network_block
* Expose build_system_rpc_future() and TransactionPoolAdaptor
* fmt
* Prepare `sc-network` for `ProtocolController`/`NotificationService`
The upcoming notification protocol refactoring requires that protocols
are able to communicate with `sc-network` over unique and direct links.
This means that `sc-network` side of the link has to be created before
`sc-network` is initialized and that it is allowed to consume the object
as the receiver half of the link may not implement `Clone`.
Remove request-response and notification protocols from `NetworkConfiguration`
and create a new object that contains the configurations of these protocols
and which is consumable by `sc-network`. This is needed needed because, e.g.,
the receiver half of `NotificationService` is not clonable so `sc-network`
must consume it when it's initializing the protocols in `Notifications`.
Similar principe applies to `PeerStore`/`ProtocolController`: as per current
design, protocols are created before the network so `Protocol` cannot be
the one creating the `PeerStore` object. `FullNetworkConfiguration` will be
used to store the objects that `sc-network` will use to communicate with
protocols and it will also allow protocols to allocate handles so they
can directly communicate with `sc-network`.
* Fixes
* Update client/service/src/builder.rs
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
* Updates
* Doc updates + cargo-fmt
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
* substrate-test-runtime migrated to pure-frame based
* test block builder: helpers added
* simple renaming
* basic_authorship test adjusted
* block_building storage_proof test adjusted
* babe: tests: should_panic expected added
* babe: tests adjusted
ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.
* beefy: tests adjusted
test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591
It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.
Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.
* grandpa: tests adjusted
test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591
It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.
Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.
* network:bitswap: test adjusted
The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.
* runtime apis versions adjusted
* storage keys used in runtime adjusted
* wasm vs native tests removed
* rpc tests: adjusted
Transfer transaction processing was slightly improved, test was
adjusted.
* tests: sizes adjusted
Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.
Sizes were adjusted in tests.
* cargo.lock update
cargo update -p substrate-test-runtime -p substrate-test-runtime-client
* warnings fixed
* builders cleanup: includes / std
* extrinsic validation cleanup
* txpool: benches performance fixed
* fmt
* spelling
* Apply suggestions from code review
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Apply code review suggestions
* Apply code review suggestions
* get rid of 1063 const
* renaming: UncheckedExtrinsic -> Extrinsic
* test-utils-runtime: further step to pure-frame
* basic-authorship: tests OK
* CheckSubstrateCall added + tests fixes
* test::Transfer call removed
* priority / propagate / no sudo+root-testing
* fixing warnings + format
* cleanup: build2/nonce + format
* final tests fixes
all tests are passing
* logs/comments removal
* should_not_accept_old_signatures test removed
* make txpool benches work again
* Cargo.lock reset
* format
* sudo hack removed
* txpool benches fix+cleanup
* .gitignore reverted
* rebase fixing + unsigned cleanup
* Cargo.toml/Cargo.lock cleanup
* force-debug feature removed
* mmr tests fixed
* make cargo-clippy happy
* network sync test uses unsigned extrinsic
* cleanup
* ".git/.scripts/commands/fmt/fmt.sh"
* push_storage_change signed call remove
* GenesisConfig cleanup
* fix
* fix
* GenesisConfig simplified
* storage_keys_works: reworked
* storage_keys_works: expected keys in vec
* storage keys list moved to substrate-test-runtime
* substrate-test: some sanity tests + GenesisConfigBuilder rework
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* Apply suggestions from code review
* Review suggestions
* fix
* fix
* beefy: generate_blocks_and_sync block_num sync with actaul value
* Apply suggestions from code review
Co-authored-by: Davide Galassi <davxy@datawok.net>
* Update test-utils/runtime/src/genesismap.rs
Co-authored-by: Davide Galassi <davxy@datawok.net>
* cargo update -p sc-rpc -p sc-transaction-pool
* Review suggestions
* fix
* doc added
* slot_duration adjusted for Babe::slot_duration
* small doc fixes
* array_bytes::hex used instead of hex
* tiny -> medium name fix
* Apply suggestions from code review
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* TransferData::try_from_unchecked_extrinsic -> try_from
* Update Cargo.lock
---------
Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Nazar Mokrynskyi