mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-12 10:01:17 +00:00
Introduce trie level cache and remove state cache (#11407)
* trie state cache
* Also cache missing access on read.
* fix comp
* bis
* fix
* use has_lru
* remove local storage cache on size 0.
* No cache.
* local cache only
* trie cache and local cache
* storage cache (with local)
* trie cache no local cache
* Add state access benchmark
* Remove warnings etc
* Add trie cache benchmark
* No extra "clone" required
* Change benchmark to use multiple blocks
* Use patches
* Integrate shitty implementation
* More stuff
* Revert "Merge branch 'master' into trie_state_cache"
This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.
* Improve benchmark
* Adapt to latest changes
* Adapt to changes in trie
* Add a test that uses iterator
* Start fixing it
* Remove obsolete file
* Make it compile
* Start rewriting the trie node cache
* More work on the cache
* More docs and code etc
* Make data cache an optional
* Tests
* Remove debug stuff
* Recorder
* Some docs and a simple test for the recorder
* Compile fixes
* Make it compile
* More fixes
* More fixes
* Fix fix fix
* Make sure cache and recorder work together for basic stuff
* Test that data caching and recording works
* Test `TrieDBMut` with caching
* Try something
* Fixes, fixes, fixes
* Forward the recorder
* Make it compile
* Use recorder in more places
* Switch to new `with_optional_recorder` fn
* Refactor and cleanups
* Move `ProvingBackend` tests
* Simplify
* Move over all functionality to the essence
* Fix compilation
* Implement estimate encoded size for StorageProof
* Start using the `cache` everywhere
* Use the cache everywhere
* Fix compilation
* Fix tests
* Adds `TrieBackendBuilder` and enhances the tests
* Ensure that recorder drain checks that values are found as expected
* Switch over to `TrieBackendBuilder`
* Start fixing the problem with child tries and recording
* Fix recording of child tries
* Make it compile
* Overwrite `storage_hash` in `TrieBackend`
* Add `storage_cache` to the benchmarks
* Fix `no_std` build
* Speed up cache lookup
* Extend the state access benchmark to also hash a runtime
* Fix build
* Fix compilation
* Rewrite value cache
* Add lru cache
* Ensure that the cache lru works
* Value cache should not be optional
* Add support for keeping the shared node cache in its bounds
* Make the cache configurable
* Check that the cache respects the bounds
* Adds a new test
* Fixes
* Docs and some renamings
* More docs
* Start using the new recorder
* Fix more code
* Take `self` argument
* Remove warnings
* Fix benchmark
* Fix accounting
* Rip off the state cache
* Start fixing fallout after removing the state cache
* Make it compile after trie changes
* Fix test
* Add some logging
* Some docs
* Some fixups and clean ups
* Fix benchmark
* Remove unneeded file
* Use git for patching
* Make CI happy
* Update primitives/trie/Cargo.toml
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update primitives/state-machine/src/trie_backend.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Introduce new `AsTrieBackend` trait
* Make the LocalTrieCache not clonable
* Make it work in no_std and add docs
* Remove duplicate dependency
* Switch to ahash for better performance
* Speedup value cache merge
* Output errors on underflow
* Ensure the internal LRU map doesn't grow too much
* Use const fn to calculate the value cache element size
* Remove cache configuration
* Fix
* Clear the cache in between for more testing
* Try to come up with a failing test case
* Make the test fail
* Fix the child trie recording
* Make everything compile after the changes to trie
* Adapt to latest trie-db changes
* Fix on stable
* Update primitives/trie/src/cache.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Fix wrong merge
* Docs
* Fix warnings
* Cargo.lock
* Bump pin-project
* Fix warnings
* Switch to released crate version
* More fixes
* Make clippy and rustdocs happy
* More clippy
* Print error when using deprecated `--state-cache-size`
* 🤦
* Fixes
* Fix storage_hash linkings
* Update client/rpc/src/dev/mod.rs
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Review feedback
* encode bound
* Rework the shared value cache
Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.
* Improve of the cache by merging the old and new solution
* FMT
* Please stop coming back all the time :crying:
* Update primitives/trie/src/cache/shared_cache.rs
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Fixes
* Make clippy happy
* Ensure we don't deadlock
* Only use one lock to simplify the code
* Do not depend on `Hasher`
* Fix tests
* FMT
* Clippy 🤦
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
This commit is contained in:
Bastian Köcher
Generated
+22
-10
@@ -2798,9 +2798,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "hashbrown"
|
||||
version = "0.12.0"
|
||||
version = "0.12.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8c21d40587b92fa6a6c6e3c1bdbf87d75511db5672f9c93175574b3a00df1758"
|
||||
checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888"
|
||||
dependencies = [
|
||||
"ahash",
|
||||
]
|
||||
@@ -4352,7 +4352,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6566c70c1016f525ced45d7b7f97730a2bafb037c788211d0c186ef5b2189f0a"
|
||||
dependencies = [
|
||||
"hash-db",
|
||||
"hashbrown 0.12.0",
|
||||
"hashbrown 0.12.3",
|
||||
"parity-util-mem",
|
||||
]
|
||||
|
||||
@@ -6572,7 +6572,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c32561d248d352148124f036cac253a644685a21dc9fea383eb4907d7bd35a8f"
|
||||
dependencies = [
|
||||
"cfg-if 1.0.0",
|
||||
"hashbrown 0.12.0",
|
||||
"hashbrown 0.12.3",
|
||||
"impl-trait-for-tuples",
|
||||
"parity-util-mem-derive",
|
||||
"parking_lot 0.12.0",
|
||||
@@ -7505,7 +7505,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1f08c8062c1fe1253064043b8fc07bfea1b9702b71b4a86c11ea3588183b12e1"
|
||||
dependencies = [
|
||||
"bytecheck",
|
||||
"hashbrown 0.12.0",
|
||||
"hashbrown 0.12.3",
|
||||
"ptr_meta",
|
||||
"rend",
|
||||
"rkyv_derive",
|
||||
@@ -7884,7 +7884,9 @@ dependencies = [
|
||||
name = "sc-client-db"
|
||||
version = "0.10.0-dev"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"hash-db",
|
||||
"kitchensink-runtime",
|
||||
"kvdb",
|
||||
"kvdb-memorydb",
|
||||
"kvdb-rocksdb",
|
||||
@@ -7894,6 +7896,7 @@ dependencies = [
|
||||
"parity-scale-codec",
|
||||
"parking_lot 0.12.0",
|
||||
"quickcheck",
|
||||
"rand 0.8.4",
|
||||
"sc-client-api",
|
||||
"sc-state-db",
|
||||
"sp-arithmetic",
|
||||
@@ -9396,6 +9399,7 @@ dependencies = [
|
||||
"sp-state-machine",
|
||||
"sp-std",
|
||||
"sp-test-primitives",
|
||||
"sp-trie",
|
||||
"sp-version",
|
||||
"thiserror",
|
||||
]
|
||||
@@ -10060,6 +10064,7 @@ dependencies = [
|
||||
"sp-trie",
|
||||
"thiserror",
|
||||
"tracing",
|
||||
"trie-db",
|
||||
"trie-root",
|
||||
]
|
||||
|
||||
@@ -10157,16 +10162,23 @@ dependencies = [
|
||||
name = "sp-trie"
|
||||
version = "6.0.0"
|
||||
dependencies = [
|
||||
"ahash",
|
||||
"criterion",
|
||||
"hash-db",
|
||||
"hashbrown 0.12.3",
|
||||
"hex-literal",
|
||||
"lazy_static",
|
||||
"lru",
|
||||
"memory-db",
|
||||
"nohash-hasher",
|
||||
"parity-scale-codec",
|
||||
"parking_lot 0.12.0",
|
||||
"scale-info",
|
||||
"sp-core",
|
||||
"sp-runtime",
|
||||
"sp-std",
|
||||
"thiserror",
|
||||
"tracing",
|
||||
"trie-bench",
|
||||
"trie-db",
|
||||
"trie-root",
|
||||
@@ -10963,9 +10975,9 @@ checksum = "a7f741b240f1a48843f9b8e0444fb55fb2a4ff67293b50a9179dfd5ea67f8d41"
|
||||
|
||||
[[package]]
|
||||
name = "trie-bench"
|
||||
version = "0.30.0"
|
||||
version = "0.31.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "57ecec5d10427b35e9ae374b059dccc0801d02d832617c04c78afc7a8c5c4a34"
|
||||
checksum = "c5704f0d6130bd83608e4370c19e20c8a6ec03e80363e493d0234efca005265a"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"hash-db",
|
||||
@@ -10979,12 +10991,12 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "trie-db"
|
||||
version = "0.23.1"
|
||||
version = "0.24.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d32d034c0d3db64b43c31de38e945f15b40cd4ca6d2dcfc26d4798ce8de4ab83"
|
||||
checksum = "004e1e8f92535694b4cb1444dc5a8073ecf0815e3357f729638b9f8fc4062908"
|
||||
dependencies = [
|
||||
"hash-db",
|
||||
"hashbrown 0.12.0",
|
||||
"hashbrown 0.12.3",
|
||||
"log",
|
||||
"rustc-hex",
|
||||
"smallvec",
|
||||
|
||||
Reference in New Issue
Block a user