The `xcm` crate was renamed to `staging-xcm` to be able to publish it to
crates.io as someone as squatted `xcm`. The problem with this rename is
that the `TypeInfo` includes the crate name which ultimately lands in
the metadata. The metadata is consumed by downstream users like
`polkadot-js` or people building on top of `polkadot-js`. These people
are using the entire `path` to find the type in the type registry. Thus,
their code would break as the type path would now be [`staging_xcm`,
`VersionedXcm`] instead of [`xcm`, `VersionedXcm`]. This pull request
fixes this by renaming the path segment `staging_xcm` to `xcm`.
This requires: https://github.com/paritytech/scale-info/pull/197
---------
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
This PR updates:
- trie-db from 0.27.1 to 0.28.0
- trie-bench from 0.37.0 to 0.38.0 (deb-dependency)
While at it, also adapts the recorder to take into account the newly
added `TrieAccess::InlineValue`.
Needed by:
- https://github.com/paritytech/polkadot-sdk/pull/1153
@paritytech/subxt-team
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
* Fix std, runtime-benchmarks and try-runtime features
zepter lint propagate-feature --feature try-runtime --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="try-runtime:frame-try-runtime"
zepter lint propagate-feature --feature runtime-benchmarks --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
zepter lint propagate-feature --feature std --left-side-feature-missing=ignore --workspace --fix
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add propagate feature CI check
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Test CI by adding an error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Use --locked
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add help msg
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Test CI by adding an error"
This reverts commit cf4ff6cc0632269b0a109e547686e5e3314b02de.
* Test CI by adding an error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* No newline in help msg
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Test CI by adding an error"
This reverts commit 5daa06ada8e01f5bebafb9d1c76804dd79bc1006.
* Test CI by adding an error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Test CI by adding an error"
This reverts commit ca15de5729507a564f140a10ec2e87b19516ec4c.
* Fix msg
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert back to master
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Re-do with Zepter v0.7.4
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update Zepter to 0.7.4
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Disable rococo try-runtime check
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* More review fixes
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
First, we increase the size to 10MiB for the local cache. Second, we give the node cache a bigger
max size than the value cache, see the changed comment on why.
In general this should ensure that we are able to cache the `:code` and not directly throw it out of
the cache because it is too big (which currently happens when the size of the runtime > 2MiB). In
the future this should be improved to ensure that certain values are not removed from the cache at
all, like `:code`.
* TrieRecorder: Start adding support for transactions
* Adds `transactions` functions and some test
* More tests
* Docs
* Ensure that we rollback failed transactions in the storage proof
* FMT
* Update primitives/trie/src/recorder.rs
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
* Review comments
* Update primitives/trie/src/recorder.rs
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* ".git/.scripts/commands/fmt/fmt.sh"
* For the holy clippy!
* Update primitives/trie/src/recorder.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Speed up storage iteration from within the runtime
* Move the cached iterator into an `Option`
* Use `RefCell` in no_std
* Simplify the code slightly
* Use `Option::replace`
* Update doc comment for `next_storage_key_slow`
* Rework storage iterators
* Make sure storage iteration is also accounted for when benchmarking
* Use `trie-db` from crates.io
* Appease clippy
* Bump `trie-bench` to 0.35.0
* Fix tests' compilation
* Update comment to clarify how `IterArgs::start_at` works
* Add extra tests
* Fix iterators on `Client` so that they behave as before
* Add extra `unwrap`s in tests
* More clippy fixes
* Come on clippy, give me a break already
* Rename `allow_missing` to `stop_on_incomplete_database`
* Add `#[inline]` to `with_recorder_and_cache`
* Use `with_recorder_and_cache` in `with_trie_db`; add doc comment
* Simplify code: use `with_trie_db` in `next_storage_key_from_root`
* Remove `expect`s in the benchmarking CLI
* Add extra doc comments
* Move `RawIter` before `TrieBackendEssence` (no code changes; just cut-paste)
* Remove a TODO in tests
* Update comment for `StorageIterator::was_complete`
* Update `trie-db` to 0.25.1
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
With `StateVersion::V1` values over a certain size are not inlined and being put into the backend
with their own hash. When accessing a value in the trie with a recorder, we check if the value is maybe already
recorded and thus, we can check the cache. To check if a value is already recorded, we use the key
of the value to differentiate them. The problem is when there are multiple tries, like multiple
child tries that all have different values under the same key. Before this pull request we didn't
have differentiated for which trie we already had recorded a (key, value) pair. This is now done by also taking
the storage root into account in the recorder to differentiate the different (key, value) pair in
the tries.
* Guard some invalid node for proof decoding.
* only forbid bitmap with no children.
* change format
* scale error.
* small test
Co-authored-by: parity-processbot <>
* Use `array-bytes` for All Array/Bytes/Hex Operations
Signed-off-by: Xavier Lau <xavier@inv.cafe>
* Reorder
* Self Review
* Format
* Fix Tests
* Bump `array-bytes`
* Optimize large test res
Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
* trie state cache
* Also cache missing access on read.
* fix comp
* bis
* fix
* use has_lru
* remove local storage cache on size 0.
* No cache.
* local cache only
* trie cache and local cache
* storage cache (with local)
* trie cache no local cache
* Add state access benchmark
* Remove warnings etc
* Add trie cache benchmark
* No extra "clone" required
* Change benchmark to use multiple blocks
* Use patches
* Integrate shitty implementation
* More stuff
* Revert "Merge branch 'master' into trie_state_cache"
This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.
* Improve benchmark
* Adapt to latest changes
* Adapt to changes in trie
* Add a test that uses iterator
* Start fixing it
* Remove obsolete file
* Make it compile
* Start rewriting the trie node cache
* More work on the cache
* More docs and code etc
* Make data cache an optional
* Tests
* Remove debug stuff
* Recorder
* Some docs and a simple test for the recorder
* Compile fixes
* Make it compile
* More fixes
* More fixes
* Fix fix fix
* Make sure cache and recorder work together for basic stuff
* Test that data caching and recording works
* Test `TrieDBMut` with caching
* Try something
* Fixes, fixes, fixes
* Forward the recorder
* Make it compile
* Use recorder in more places
* Switch to new `with_optional_recorder` fn
* Refactor and cleanups
* Move `ProvingBackend` tests
* Simplify
* Move over all functionality to the essence
* Fix compilation
* Implement estimate encoded size for StorageProof
* Start using the `cache` everywhere
* Use the cache everywhere
* Fix compilation
* Fix tests
* Adds `TrieBackendBuilder` and enhances the tests
* Ensure that recorder drain checks that values are found as expected
* Switch over to `TrieBackendBuilder`
* Start fixing the problem with child tries and recording
* Fix recording of child tries
* Make it compile
* Overwrite `storage_hash` in `TrieBackend`
* Add `storage_cache` to the benchmarks
* Fix `no_std` build
* Speed up cache lookup
* Extend the state access benchmark to also hash a runtime
* Fix build
* Fix compilation
* Rewrite value cache
* Add lru cache
* Ensure that the cache lru works
* Value cache should not be optional
* Add support for keeping the shared node cache in its bounds
* Make the cache configurable
* Check that the cache respects the bounds
* Adds a new test
* Fixes
* Docs and some renamings
* More docs
* Start using the new recorder
* Fix more code
* Take `self` argument
* Remove warnings
* Fix benchmark
* Fix accounting
* Rip off the state cache
* Start fixing fallout after removing the state cache
* Make it compile after trie changes
* Fix test
* Add some logging
* Some docs
* Some fixups and clean ups
* Fix benchmark
* Remove unneeded file
* Use git for patching
* Make CI happy
* Update primitives/trie/Cargo.toml
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update primitives/state-machine/src/trie_backend.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Introduce new `AsTrieBackend` trait
* Make the LocalTrieCache not clonable
* Make it work in no_std and add docs
* Remove duplicate dependency
* Switch to ahash for better performance
* Speedup value cache merge
* Output errors on underflow
* Ensure the internal LRU map doesn't grow too much
* Use const fn to calculate the value cache element size
* Remove cache configuration
* Fix
* Clear the cache in between for more testing
* Try to come up with a failing test case
* Make the test fail
* Fix the child trie recording
* Make everything compile after the changes to trie
* Adapt to latest trie-db changes
* Fix on stable
* Update primitives/trie/src/cache.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Fix wrong merge
* Docs
* Fix warnings
* Cargo.lock
* Bump pin-project
* Fix warnings
* Switch to released crate version
* More fixes
* Make clippy and rustdocs happy
* More clippy
* Print error when using deprecated `--state-cache-size`
* 🤦
* Fixes
* Fix storage_hash linkings
* Update client/rpc/src/dev/mod.rs
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Review feedback
* encode bound
* Rework the shared value cache
Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.
* Improve of the cache by merging the old and new solution
* FMT
* Please stop coming back all the time :crying:
* Update primitives/trie/src/cache/shared_cache.rs
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Fixes
* Make clippy happy
* Ensure we don't deadlock
* Only use one lock to simplify the code
* Do not depend on `Hasher`
* Fix tests
* FMT
* Clippy 🤦
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* Update UI test output for rust 1.62.1
* switch ci to staging image to check that everything works
* fix artifacts node-bench-regression-guard
* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning
* Fix more clippy lints
* Make clippy happy by relying on auto-deref were possible
* Add tracking issue to the comments
* pin ci image
Co-authored-by: alvicsam <alvicsam@gmail.com>
* Add a test case for the determinism of recorded proof
* Replace HashMap with BTreeMap for the actual proof records
* cargo +nightly fmt --all
* Store the trie nodes in BTreeSet for StorageProof
* Nit
* Revert the BTreeMap changes and sort when converting to storage proof
* Remove PartialEq from StorageProof
* Remove unnecessary change
* Add `compare` method to StorageProof
* FMT
* Dummy change to trigger CI
* Use `BTreeSet` for StorageProof and keep using `Vec` for CompactProof
* Update comment on `iter_nodes`
* Revert `PartialEq` removal
Bastian Köcher