Part of https://github.com/paritytech/polkadot-sdk/issues/4126 we want
to safely increase the execute_workers_max_num gradually from chain to
chain and assess if there are any negative impacts.
This PR performs the necessary plumbing to be able to increase it based
on the chain id, it increase the number of execution workers from 2 to 4
on test network but lives kusama and polkadot unchanged until we gather
more data.
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
This introduces a check to ensure that the parachain code matches the
validation code stored in the relay chain state. If not, it will print a
warning. This should be mainly useful for parachain builders to make
sure they have setup everything correctly.
Currently, collators and their alongside nodes spin up a full-scale
overseer running a bunch of subsystems that are not needed if the node
is not a validator. That was considered to be harmless; however, we've
got problems with unused subsystems getting stalled for a reason not
currently known, resulting in the overseer exiting and bringing down the
whole node.
This PR aims to only run needed subsystems on such nodes, replacing the
rest with `DummySubsystem`.
It also enables collator-optimized availability recovery subsystem
implementation.
Partially solves #1730.
Currently the polkadot node will backoff from block authoring if
finality starts lagging. This PR disables this mechanism on production
networks (polkadot and kusama) and adds a flags to optionally force
enabling it.
* companion 14754: no-beefy flag moved to sc-cli
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* bump substrate ref
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* bump polkadot ref
---------
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* BlockId removal: refactor of runtime API
It changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
* fmt
* cargo update -p polkadot-primitives
* Revert "cargo update -p polkadot-primitives"
This reverts commit 58506ecd31e1e5e42b78c2db3982acbc4d8edc70.
* update lockfile for {"substrate", "polkadot"}
---------
Co-authored-by: parity-processbot <>
* Use primitives reexported from `polkadot_primitives` crate root
* restart CI
* Fixes after merge
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
* BlockId removal: refactor: HeaderBackend::status
It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
* BlockId removal: &Hash to Hash
It changes &Block::Hash argument to Block::Hash.
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
* Add minimal overseer gen with dummy subsystems
* Fix dependencies
* no-compile: only client transaction pool missing
* Remove unused imports
* Continue to hack towards PoC
* Continue
* Make mini node compile
* Compiling version with blockchainevents trait
* Continue
* Check in lockfile
* Block with tokio
* update patches
* Update polkadot patches
* Use polkadot-primitives v2
* Fix build problems
* First working version
* Adjust cargo.lock
* Add integration test
* Make integration test work
* Allow startinc collator without relay-chain args
* Make OverseerRuntimeClient async
* Create separate integration test
* Remove unused ChainSelection code
* Remove unused parameters on new-mini
* Connect collator node in test to relay chain nodes
* Make BlockChainRPCClient obsolete
* Clean up
* Clean up
* Reimplement blockchain-rpc-events
* Revert "Allow startinc collator without relay-chain args"
This reverts commit f22c70e16521f375fe125df5616d48ceea926b1a.
* Add `strict_record_validation` to AuthorityDiscovery
* Move network to cumulus
* Remove BlockchainRPCEvents
* Remove `BlockIdTo` and `BlockchainEvents`
* Make AuthorityDiscovery async
* Use hash in OverseerRuntime
* Adjust naming of runtime client trait
* Implement more rpc-client methods
* Improve error handling for `ApiError`
* Extract authority-discovery creationand cleanup
* RPC -> Rpc
* Extract bitswap
* Adjust to changes on master
* Implement `hash` method
* Introduce DummyChainSync, remove ProofProvider and BlockBackend
* Remove `HeaderMetadata` from blockchain-rpc-client
* Make ChainSync work
* Implement NetworkHeaderBackend
* Cleanup
* Adjustments after master merge
* Remove ImportQueue from network parameters
* Remove cargo patches
* Eliminate warnings
* Revert to HeaderBackend
* Add zombienet test
* Implement `status()` method
* Add more comments, improve readability
* Remove patches from Cargo.toml
* Remove integration test in favor of zombienet
* Remove unused dependencies, rename minimal node crate
* Adjust to latest master changes
* fmt
* Execute zombienet test on gitlab ci
* Reuse network metrics
* Chainsync metrics
* fmt
* Feed RPC node as boot node to the relay chain minimal node
* fmt
* Add bootnodes to zombienet collators
* Allow specification of relay chain args
* Apply review suggestions
* Remove unnecessary casts
* Enable PoV recovery for rpc full nodes
* Revert unwanted changes
* Make overseerHandle non-optional
* Add availability-store subsystem
* Add AuxStore and ChainApiSubsystem
* Add availability distribution subsystem
* Improve pov-recovery logging and add RPC nodes to tests
* fmt
* Make availability config const
* lock
* Enable debug logs for pov-recovery in zombienet
* Add log filters to test binary
* Allow wss
* Address review comments
* Apply reviewer comments
* Adjust to master changes
* Apply reviewer suggestions
* Bump polkadot
* Add builder method for minimal node
* Bump substrate and polkadot
* Clean up overseer building
* Add bootnode to two in pov_recovery test
* Fix missing quote in pov recovery zombienet test
* Improve zombienet pov test
* More debug logs for pov-recovery
* Remove reserved nodes like on original test
* Revert zombienet test to master
* Cleanup light client leftovers
* Readme: make it clear that Collator is full node of parachain
* cargo update -p sp-io
* cargo update -p polkadot-primitives
* Align to changes in Substrate
* Align to the newest changes in substrate
* Update `Cargo.lock`
* Add hwbenches to `parachain-template` too
* update lockfile for {"polkadot"}
Co-authored-by: parity-processbot <>
* Handle relocation of `ParachainHost` in Polkadot
`ParachainHost` is no longer versioned and is in `runtime_api` module.
This is a companion for
https://github.com/paritytech/polkadot/pull/5048
* Update dependencies
* Initial network interface preparations
* Implement get_storage_by_key
* Implement `validators` and `session_index_for_child`
* Implement persisted_validation_data and candidate_pending_availability
* Fix method name for persisted_validation_data and add encoded params
* Implement `retrieve_dmq_contents` and `retrieve_all_inbound_hrmp_channel_contents`
* Implement `prove_read`
* Introduce separate RPC client, expose JsonRpSee errors
* Simplify closure in call_remote_runtime_function
* Implement import stream, upgrade JsonRpSee
* Implement finality stream
* Remove unused method from interface
* Implement `is_major_syncing`
* Implement `wait_on_block`
* Fix tests
* Unify error handling `ApiError`
* Replace WaitError with RelayChainError
* Wrap BlockChainError in RelayChainError
* Unify error handling in relay chain intefaces
* Fix return type of proof method
* Improve error handling of new methods
* Improve error handling and move logging outside of interface
* Clean up
* Remove unwanted changes, clean up
* Remove unused import
* Add format for StatemachineError and remove nused From trait
* Use 'thiserror' crate to simplify error handling
* Expose error for overseer, further simplify error handling
* Reintroduce network interface
* Implement cli option
* Adjust call_state method to use hashes
* Disable PoV recovery when RPC is used
* Add integration test for network full node
* Use Hash instead of BlockId to ensure compatibility with RPC interface
* Fix cargo check warnings
* Implement retries
* Remove `expect` statements from code
* Update jsonrpsee to 0.8.0 and make collator keys optional
* Make cli arguments conflicting
* Remove unused `block_status` method
* Add clippy fixes
* Cargo fmt
* Validate relay chain rpc url
* Clean up dependencies and add one more integration test
* Clean up
* Clean up dependencies of relay-chain-network
* Use hash instead of blockid for rpc methods
* Fix tests
* Update client/cli/src/lib.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Improve error message of cli validation
* Add rpc client constructor
* Do not use debug formatting for errors
* Improve logging for remote runtime methods
* Only retry on transport problems
* Use PHash by value, rename test
* Improve tracing, return error on relay-chain-interface build
* Fix naming, use generics instead of deserializing manually
* Rename RelayChainLocal and RelayChainNetwork
* lock
* Format
* Use impl trait for encodable runtime payload
* Only instantiate full node in tests when we need it
* Upgrade scale-codec to 3.0.0
* Improve expect log
Co-authored-by: Koute <koute@users.noreply.github.com>
Alexandru Gheorghe