The `BlockBuilderProvider` was a trait that was defined in
`sc-block-builder`. The trait was implemented for `Client`. This
basically meant that you needed to import `sc-block-builder` any way to
have access to the block builder. So, this trait was not providing any
real value. This pull request is removing the said trait. Instead of the
trait it introduces a builder for creating a `BlockBuilder`. The builder
currently has the quite fabulous name `BlockBuilderBuilder` (I'm open to
any better name 😅). The rest of the pull request is about
replacing the old trait with the new builder.
# Downstream code changes
If you used `new_block` or `new_block_at` before you now need to switch
it over to the new `BlockBuilderBuilder` pattern:
```rust
// `new` requires a type that implements `CallApiAt`.
let mut block_builder = BlockBuilderBuilder::new(client)
// Then you need to specify the hash of the parent block the block will be build on top of
.on_parent_block(at)
// The block builder also needs the block number of the parent block.
// Here it is fetched from the given `client` using the `HeaderBackend`
// However, there also exists `with_parent_block_number` for directly passing the number
.fetch_parent_block_number(client)
.unwrap()
// Enable proof recording if required. This call is optional.
.enable_proof_recording()
// Pass the digests. This call is optional.
.with_inherent_digests(digests)
.build()
.expect("Creates new block builder");
```
---------
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
* Start
* More work!
* Moar
* More changes
* More fixes
* More worrk
* More fixes
* More fixes to make it compile
* Adds `NoOffchainStorage`
* Pass the extensions
* Small basti making small progress
* Fix merge errors and remove `ExecutionContext`
* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`
Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.
* Fix compilation
* Register the global extensions inside runtime api instance
* Fixes
* Fix `generate_initial_session_keys` by passing the keystore extension
* Fix the grandpa tests
* Fix more tests
* Fix more tests
* Don't set any heap pages if there isn't an override
* Fix small fallout
* FMT
* Fix tests
* More tests
* Offchain worker custom extensions
* More fixes
* Make offchain tx pool creation reusable
Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.
* Fixes
* Fixes
* Set offchain transaction pool in BABE before using it in the runtime
* Add the `offchain_tx_pool` to Grandpa as well
* Fix the nodes
* Print some error when using the old warnings
* Fix merge issues
* Fix compilation
* Rename `babe_link`
* Rename to `offchain_tx_pool_factory`
* Cleanup
* FMT
* Fix benchmark name
* Fix `try-runtime`
* Remove `--execution` CLI args
* Make clippy happy
* Forward bls functions
* Fix docs
* Update UI tests
* Update client/api/src/execution_extensions.rs
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update client/cli/src/params/import_params.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update client/api/src/execution_extensions.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Pass the offchain storage to the MMR RPC
* Update client/api/src/execution_extensions.rs
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Review comments
* Fixes
---------
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* BlockId removal: refactor of runtime API
It changes the arguments of:
- `ApiExt` methods: `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`
It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* BlockId removal: refactor of runtime API - tests
- tests adjusted to new runtime API,
- some tests migrated from block number to block hash
* benchmarking-cli: BlockId(0) migrated to info().genesis_hash
`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.
* trivial: api.call(BlockId) -> api.call(Hash)
- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,
* CallApiAtParams::at changed to Block::Hash
* missed doc updated
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* ".git/.scripts/commands/fmt/fmt.sh"
* BlockId removal: Benchmark::consumed_weight
Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.
* at_hash renamed
* wrong merge fixed
* beefy worker: merged with master
* beefy: tests: missing block problem fixed
* Apply review suggestion
* fix
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
* BlockId removal: Client::runtime_version_at
It changes the arguments of `Client::runtime_version_at` from: `BlockId<Block>` to: `Block::Hash`
* Apply suggestions from code review
Co-authored-by: Anton <anton.kalyaev@gmail.com>
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: parity-processbot <>
* Set genesis block data using the built genesis block
* Make resolve_state_version_from_wasm a separate function and some small refactorings
Useful for the commit following.
* Introduce trait BuildGenesisBlock
Substrate users can use this trait to implement their custom genesis block when constructing the
client.
* Make call_executor test compile
* cargo +nightly fmt --all
* Fix test
* Remove unnecessary clone
* FMT
* Apply review suggestions
* Revert changes to new_full_client() and new_full_parts() signature
* Remove needless `Block` type in `resolve_state_version_from_wasm`
* 1.Add pruning param "canonical" in sc-cli.
2.Make PruningMode's default value to ArchiveCanonical.
* Update tests in sc-state-db.
* Update tests in sc-state-db.
* 1.Add a new value `AllWithNonFinalized` in `enum BlocksPruning` which Corresponds to `blocks_pruning 0` in CLI .
2.Change value `All` to `AllFinalized` in `enum BlocksPruning` and make it to keep full finalized block history.
* Make some corresponding adjustments based on the content in the conversation.
* Update client/db/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Apply suggestions from code review.
* 1.Change `blocks_pruning` to be like `state_pruning` .
* Fmt and add some doc.
* Update client/cli/src/params/pruning_params.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update client/cli/src/params/pruning_params.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update doc.
* Change `new_test_with_tx_storage` to take `BlocksPruning`.
* Fmt
Co-authored-by: Bastian Köcher <git@kchr.de>
* trie state cache
* Also cache missing access on read.
* fix comp
* bis
* fix
* use has_lru
* remove local storage cache on size 0.
* No cache.
* local cache only
* trie cache and local cache
* storage cache (with local)
* trie cache no local cache
* Add state access benchmark
* Remove warnings etc
* Add trie cache benchmark
* No extra "clone" required
* Change benchmark to use multiple blocks
* Use patches
* Integrate shitty implementation
* More stuff
* Revert "Merge branch 'master' into trie_state_cache"
This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.
* Improve benchmark
* Adapt to latest changes
* Adapt to changes in trie
* Add a test that uses iterator
* Start fixing it
* Remove obsolete file
* Make it compile
* Start rewriting the trie node cache
* More work on the cache
* More docs and code etc
* Make data cache an optional
* Tests
* Remove debug stuff
* Recorder
* Some docs and a simple test for the recorder
* Compile fixes
* Make it compile
* More fixes
* More fixes
* Fix fix fix
* Make sure cache and recorder work together for basic stuff
* Test that data caching and recording works
* Test `TrieDBMut` with caching
* Try something
* Fixes, fixes, fixes
* Forward the recorder
* Make it compile
* Use recorder in more places
* Switch to new `with_optional_recorder` fn
* Refactor and cleanups
* Move `ProvingBackend` tests
* Simplify
* Move over all functionality to the essence
* Fix compilation
* Implement estimate encoded size for StorageProof
* Start using the `cache` everywhere
* Use the cache everywhere
* Fix compilation
* Fix tests
* Adds `TrieBackendBuilder` and enhances the tests
* Ensure that recorder drain checks that values are found as expected
* Switch over to `TrieBackendBuilder`
* Start fixing the problem with child tries and recording
* Fix recording of child tries
* Make it compile
* Overwrite `storage_hash` in `TrieBackend`
* Add `storage_cache` to the benchmarks
* Fix `no_std` build
* Speed up cache lookup
* Extend the state access benchmark to also hash a runtime
* Fix build
* Fix compilation
* Rewrite value cache
* Add lru cache
* Ensure that the cache lru works
* Value cache should not be optional
* Add support for keeping the shared node cache in its bounds
* Make the cache configurable
* Check that the cache respects the bounds
* Adds a new test
* Fixes
* Docs and some renamings
* More docs
* Start using the new recorder
* Fix more code
* Take `self` argument
* Remove warnings
* Fix benchmark
* Fix accounting
* Rip off the state cache
* Start fixing fallout after removing the state cache
* Make it compile after trie changes
* Fix test
* Add some logging
* Some docs
* Some fixups and clean ups
* Fix benchmark
* Remove unneeded file
* Use git for patching
* Make CI happy
* Update primitives/trie/Cargo.toml
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update primitives/state-machine/src/trie_backend.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Introduce new `AsTrieBackend` trait
* Make the LocalTrieCache not clonable
* Make it work in no_std and add docs
* Remove duplicate dependency
* Switch to ahash for better performance
* Speedup value cache merge
* Output errors on underflow
* Ensure the internal LRU map doesn't grow too much
* Use const fn to calculate the value cache element size
* Remove cache configuration
* Fix
* Clear the cache in between for more testing
* Try to come up with a failing test case
* Make the test fail
* Fix the child trie recording
* Make everything compile after the changes to trie
* Adapt to latest trie-db changes
* Fix on stable
* Update primitives/trie/src/cache.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Fix wrong merge
* Docs
* Fix warnings
* Cargo.lock
* Bump pin-project
* Fix warnings
* Switch to released crate version
* More fixes
* Make clippy and rustdocs happy
* More clippy
* Print error when using deprecated `--state-cache-size`
* 🤦
* Fixes
* Fix storage_hash linkings
* Update client/rpc/src/dev/mod.rs
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Review feedback
* encode bound
* Rework the shared value cache
Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.
* Improve of the cache by merging the old and new solution
* FMT
* Please stop coming back all the time :crying:
* Update primitives/trie/src/cache/shared_cache.rs
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Fixes
* Make clippy happy
* Ensure we don't deadlock
* Only use one lock to simplify the code
* Do not depend on `Hasher`
* Fix tests
* FMT
* Clippy 🤦
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Switch to pooling copy-on-write instantiation strategy for WASM
* Fix benchmark compilation
* Fix `cargo fmt`
* Fix compilation of another benchmark I've missed
* Cleanups according to review comments
* Move `max_memory_size` to `Semantics`
* Set `memory_guaranteed_dense_image_size` to `max_memory_size`
* Rename `wasm_instantiation_strategy` to `wasmtime_instantiation_strategy`
* Update the doc-comments regarding the instantiation strategy
* Extend the integration tests to test every instantiation strategy
* Don't drop the temporary directory until the runtime is dropped in benchmarks
* Don't drop the temporary directory until the runtime is dropped in tests
* sc-client-db: utils::open_database(...) — return OpenDbError so that the caller could tell the `OpenDbError::DoesNotExist` clearly
* sc-client-db: utils::open_database(..) — accept the `create: bool` argument
* sc-client-db: pruning — optional argument in the DatabaseSettings
* sc-state-db: Split `Error<E>` into separate `Error<E>` and `StateDbError`
* StateDb::open: choose the pruning-mode depending on the requested and stored values
* sc-state-db: test for different combinations of stored and requested pruning-modes
* CLI-argument: mark the unsafe-pruning as deprecated
* Fix tests
* tests: do not specify --pruning when running the substrate over the existing storage
* fix types for benches
* cargo fmt
* Check whether the pruning-mode and sync-mode are compatible
* cargo fmt
* parity-db: 0.3.11 -> 0.3.12
* sc-state-db: MetaDb::set_meta — a better doc-test
* cargo fmt
* make MetaDb read-only again!
* Remove the stray newline (and run the CI once again please)
* Last nitpicks
* A more comprehensive error message
* Split native executor stuff from wasm executor stuff
* Remove `native_runtime_version` in places
* Fix warning
* Fix test warning
* Remove redundant NativeRuntimeInfo trait
* Add a warning for use_native
* Run cargo fmt
* Revert "Add a warning for use_native"
This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.
* Make choosing an executor (native/wasm) an explicit part of service construction
* Add Cargo.lock
* Rename Executor to ExecutorDispatch
* Update bin/node/executor/src/lib.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Fix tests
* Fix minor node-executor error
* Fix node cli command thing
Co-authored-by: Squirrel <gilescope@gmail.com>
* implement "auto" database backend in client/db, in progress, #9201
* move fn supports_ref_counting from DatabaseSource enum to Database trait to make it work correctly for all types of dbs
* update kvdb_rocksdb to 0.13 and use it's new config feature to properly auto start existing database
* tests for auto database reopening
* introduce OpenDbError to cleanup opening database error handling and handle case when database is not enabled at the compile time
* cargo fmt strings again
* cargo fmt strings again
* rename DataSettingsSrc to fix test compilation
* fix the call to the new kvdb-rocksdb interdace in tests to fix compilation
* simplify OpenDbError and make it compile even when paritydb and rocksdb are disabled
* cargo fmt
* fix compilation without flag with-parity-db
* fix unused var compilation warning
* support different paths for rocksdb and paritydb in DatabaseSouce::Auto
* support "auto" database option in substrate cli
* enable Lz4 compression for some of the parity-db colums as per review suggestion
* applied review suggestions
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Make grandpa work
* Introduce `SharedData`
* Add test and fix bugs
* Switch to `SharedData`
* Make grandpa tests working
* More Babe work
* Make it async
* Fix fix
* Use `async_trait` in sc-consensus-slots
This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.
* Make grandpa tests compile
* More Babe tests work
* Fix network test
* Start fixing service test
* Finish service-test
* Fix sc-consensus-aura
* Fix fix fix
* More fixes
* Make everything compile *yeah*
* Fix build when we have Rust 1.51
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix warning
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Add MMR custom RPC.
* Change RuntimeApi to avoid hardcoding leaf type.
* Properly implement the new RuntimeAPI and wire up RPC.
* Extract Offchain DB as separate execution extension.
* Enable offchain DB access for offchain calls.
* Fix offchain_election tests.
* Skip block initialisation for proof generation.
* Fix integration test setup.
* Fix offchain tests. Not sure how I missed them earlier 🤷.
* Fix long line.
* One more test missing.
* Update mock for multi-phase.
* Address review grumbbles.
* Address review grumbles.
* Fix line width of a comment
* More extensible multiaddress format
* update name
* Don't depend on indices to define multiaddress type
* Use MultiAddress in Node Template too!
* reduce traits, fix build
* support multiple `StaticLookup`
* bump tx version
* feedback
* add system clear cache
* move to shared
* rearrange a little-a
* add few tricks also
* use tempdir for buf as well
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* WIP
* Making progress
* Almost ready
* Get service tests compiling
* Fix node screenshot
* Line widths
* Fix node cli tests
* Fix node cli warning
* ChainComponents -> ServiceComponents, fix tests
* make spawn_handle public
* Remove spawnnamed impl for taskmanager
* Move the keep alive stuff to the task manager
* Move the telemetry, base path, rpc keep_alive to the service builder
* Make the task manager keep alive an internal detail
* Rewrite the browser start_client future
* Remove run_node etc
* Revert my personal changes to browser-demo/build.sh
* use |config|
* Add a runtime_version function to SubstrateCli
* Reexport role and runtime version from sc cli
* Update Cargo.lock
* runtime_version -> native_runtime_version
* Pass chain spec to native_runtime_version for polkadot
* Fix line widths
* Traitify ServiceComponents Client
Michal Kucharczyk