Commit Graph

105 Commits

Author SHA1 Message Date
Alexandru Vasile 0bebc8ae96 Update trie-db version to 0.28.0 (#1522)
This PR updates:
- trie-db from 0.27.1 to 0.28.0
- trie-bench from 0.37.0 to 0.38.0 (deb-dependency)


While at it, also adapts the recorder to take into account the newly
added `TrieAccess::InlineValue`.

Needed by:
- https://github.com/paritytech/polkadot-sdk/pull/1153

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-13 14:18:12 +03:00
dependabot[bot] 711132c961 Bump thiserror from 1.0.47 to 1.0.48 (#1396)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.47 to 1.0.48.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.47...1.0.48)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-09-05 09:27:19 +02:00
Oliver Tale-Yazdi dcda0e50f5 Fix build profiles (#1229)
* Fix build profiles

Closes https://github.com/paritytech/polkadot-sdk/issues/1155

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Manually set version to 1.0.0

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use workspace repo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* 'Authors and Edition from workspace

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-08-29 13:39:41 +02:00
alvicsam f441a5fc93 Diener workspacify
Signed-off-by: alvicsam <alvicsam@gmail.com>
2023-08-25 11:05:17 +02:00
Oliver Tale-Yazdi 8b9455465b Format and Sort features in Cargo.toml files (#14803)
* CI: Add feature sorting check

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Sort all features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Add some mistakes"

This reverts commit b2b1099f979f6decb22d09b46689c1554bb72e81.

* CI job naming

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add oneliner formatting

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Explain tool

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use latest version

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Better erorr message

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Format after master merge

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use --check option

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Messed up the merge commit...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-08-23 14:21:52 +00:00
Oliver Tale-Yazdi 77321288c3 Fix Substrate features (#14660)
* Fix std, runtime-benchmarks and try-runtime features

zepter lint propagate-feature --feature try-runtime --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="try-runtime:frame-try-runtime"
zepter lint propagate-feature --feature runtime-benchmarks --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
zepter lint propagate-feature --feature std --left-side-feature-missing=ignore --workspace --fix

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add propagate feature CI check

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test CI by adding an error

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use --locked

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add help msg

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Test CI by adding an error"

This reverts commit cf4ff6cc0632269b0a109e547686e5e3314b02de.

* Test CI by adding an error

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* No newline in help msg

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Test CI by adding an error"

This reverts commit 5daa06ada8e01f5bebafb9d1c76804dd79bc1006.

* Test CI by adding an error

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Test CI by adding an error"

This reverts commit ca15de5729507a564f140a10ec2e87b19516ec4c.

* Fix msg

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert back to master

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Re-do with Zepter v0.7.4

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update Zepter to 0.7.4

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Disable rococo try-runtime check

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* More review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-08-01 20:25:59 +00:00
Anton 59d8b86450 chore: update libp2p to 0.52.1 (#14429)
* update libp2p to 0.52.0

* proto name now must implement `AsRef<str>`

* update libp2p version everywhere

* ToSwarm, FromBehaviour, ToBehaviour

also LocalProtocolsChange and RemoteProtocolsChange

* new NetworkBehaviour invariants

* replace `Vec<u8>` with `StreamProtocol`

* rename ConnectionHandlerEvent::Custom to NotifyBehaviour

* remove DialError & ListenError invariants

also fix pending_events

* use connection_limits::Behaviour

See https://github.com/libp2p/rust-libp2p/pull/3885

* impl `void::Void` for `BehaviourOut`

also use `Behaviour::with_codec`

* KademliaHandler no longer public

* fix StreamProtocol construction

* update libp2p-identify to 0.2.0

* remove non-existing methods from PollParameters

rename ConnectionHandlerUpgrErr to StreamUpgradeError

* `P2p` now contains `PeerId`, not `Multihash`

* use multihash-codetable crate

* update Cargo.lock

* reformat text

* comment out tests for now

* remove `.into()` from P2p

* confirm observed addr manually

See https://github.com/libp2p/rust-libp2p/blob/master/protocols/identify/CHANGELOG.md#0430

* remove SwarmEvent::Banned

since we're not using `ban_peer_id`, this can be safely removed.
we may want to introduce `libp2p::allow_block_list` module in the future.

* fix imports

* replace `libp2p` with smaller deps in network-gossip

* bring back tests

* finish rewriting tests

* uncomment handler tests

* Revert "uncomment handler tests"

This reverts commit 720a06815887f4e10767c62b58864a7ec3a48e50.

* add a fixme

* update Cargo.lock

* remove extra From

* make void uninhabited

* fix discovery test

* use autonat protocols

confirming external addresses manually is unsafe in open networks

* fix SyncNotificationsClogged invariant

* only set server mode manually in tests

doubt that we need to set it on node since we're adding public addresses

* address @dmitry-markin comments

* remove autonat

* removed unused var

* fix EOL

* update smallvec and sha2

in attempt to compile polkadot

* bump k256

in attempt to build cumulus

---------

Co-authored-by: parity-processbot <>
2023-07-25 11:12:24 +00:00
Qinxuan Chen e8e2048b88 chore: update some deps to prune duplicated deps (#14515)
* chore: update some deps to prune duplicated deps

- array-bytes: 4.1.0 => 6.1.0
- cid: 0.8.6 => 0.9.0

* fix
2023-07-06 16:46:44 +02:00
Sebastian Kunert 30a7e6788c Make triecache generic and work for no-std environments (#14403)
* Remove `AsLocalTrieCache` trait

* Introduce new trait AsTrieDbCache

* Use AsTrieDbCache trait

* Make it compile

* Docs

* Improve naming of associated type, implement cache usage for no-std

* Improve naming

* Improve docs

* Allow construction with optional cache

* FMT

* Remove unused variable

* Revert unwanted change

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* More comment adjustments

* Docs

* Trigger CI

* ".git/.scripts/commands/fmt/fmt.sh"

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Review comments

* Review comments

* Apply suggestions from code review

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* fmt

* Bump trie-db again

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
2023-06-28 12:57:41 +00:00
Koute 9d1741eb3c Bump parity-scale-codec to 3.6.1 (#14428) 2023-06-21 11:37:11 +00:00
Alexandru Vasile 3da9449067 Bump sp-crates from latest crates.io version + release (#14265)
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-05-31 12:11:01 +00:00
Alexandru Vasile 98a0550ea2 Release: Bump SP crates to release on crates.io (#14237)
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-05-29 10:40:59 +00:00
Koute 6c3747ba34 Bump trie-db to 0.27.1 (fix storage iteration bug + change seek behavior with prefix) (#13630) 2023-03-17 16:09:28 +00:00
cheme ef165cec3e update trie deps (#13601) 2023-03-14 21:03:50 +00:00
Koute 55263fa2a1 Speed up storage iteration from within the runtime (#13479)
* Speed up storage iteration from within the runtime

* Move the cached iterator into an `Option`

* Use `RefCell` in no_std

* Simplify the code slightly

* Use `Option::replace`

* Update doc comment for `next_storage_key_slow`
2023-03-01 08:58:18 +00:00
Koute f8e3bdad3d Rework storage iterators (#13284)
* Rework storage iterators

* Make sure storage iteration is also accounted for when benchmarking

* Use `trie-db` from crates.io

* Appease clippy

* Bump `trie-bench` to 0.35.0

* Fix tests' compilation

* Update comment to clarify how `IterArgs::start_at` works

* Add extra tests

* Fix iterators on `Client` so that they behave as before

* Add extra `unwrap`s in tests

* More clippy fixes

* Come on clippy, give me a break already

* Rename `allow_missing` to `stop_on_incomplete_database`

* Add `#[inline]` to `with_recorder_and_cache`

* Use `with_recorder_and_cache` in `with_trie_db`; add doc comment

* Simplify code: use `with_trie_db` in `next_storage_key_from_root`

* Remove `expect`s in the benchmarking CLI

* Add extra doc comments

* Move `RawIter` before `TrieBackendEssence` (no code changes; just cut-paste)

* Remove a TODO in tests

* Update comment for `StorageIterator::was_complete`

* Update `trie-db` to 0.25.1
2023-02-22 07:49:25 +00:00
Gavin Wood 2d7fa18e73 Make DispatchError impl MEL (#13169)
* Make DispatchError impl MEL

* Upgrade SCALE codec to support `codec(skip)` for MEL

Co-authored-by: Bastian Köcher <info@kchr.de>
2023-01-19 16:47:47 +00:00
Qinxuan Chen 40c04cf11d *: update rand to v0.8.5 (#12962)
* *: update rand to v0.8.5

* *: remove useless deps from Cargo.toml

* fix pallet-session-benchmarking

* fix pallet-election-provider-support test

* remove useless rand from dev-dependencies
2022-12-19 07:38:51 +01:00
Niklas Adolfsson 2b8af8cb1a release sp-core 7.0.0 and sp-runtime 7.0.0 (#12599)
* chore(release): sp-core v7.0.0

* chore(release): sp-runtime v7.0.0

* fix bad merge
2022-11-15 14:54:14 +00:00
Xavier Lau 86198c5471 Use array-bytes for All Array/Bytes/Hex Operations (#12190)
* Use `array-bytes` for All Array/Bytes/Hex Operations

Signed-off-by: Xavier Lau <xavier@inv.cafe>

* Reorder

* Self Review

* Format

* Fix Tests

* Bump `array-bytes`

* Optimize large test res

Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
2022-09-21 06:12:20 +00:00
dependabot[bot] 5318bf53d4 Bump parking_lot from 0.12.0 to 0.12.1 (#12098)
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.12.0 to 0.12.1.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.12.0...0.12.1)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-25 23:52:19 +02:00
Bastian Köcher 73d9ae3284 Introduce trie level cache and remove state cache (#11407)
* trie state cache

* Also cache missing access on read.

* fix comp

* bis

* fix

* use has_lru

* remove local storage cache on size 0.

* No cache.

* local cache only

* trie cache and local cache

* storage cache (with local)

* trie cache no local cache

* Add state access benchmark

* Remove warnings etc

* Add trie cache benchmark

* No extra "clone" required

* Change benchmark to use multiple blocks

* Use patches

* Integrate shitty implementation

* More stuff

* Revert "Merge branch 'master' into trie_state_cache"

This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.

* Improve benchmark

* Adapt to latest changes

* Adapt to changes in trie

* Add a test that uses iterator

* Start fixing it

* Remove obsolete file

* Make it compile

* Start rewriting the trie node cache

* More work on the cache

* More docs and code etc

* Make data cache an optional

* Tests

* Remove debug stuff

* Recorder

* Some docs and a simple test for the recorder

* Compile fixes

* Make it compile

* More fixes

* More fixes

* Fix fix fix

* Make sure cache and recorder work together for basic stuff

* Test that data caching and recording works

* Test `TrieDBMut` with caching

* Try something

* Fixes, fixes, fixes

* Forward the recorder

* Make it compile

* Use recorder in more places

* Switch to new `with_optional_recorder` fn

* Refactor and cleanups

* Move `ProvingBackend` tests

* Simplify

* Move over all functionality to the essence

* Fix compilation

* Implement estimate encoded size for StorageProof

* Start using the `cache` everywhere

* Use the cache everywhere

* Fix compilation

* Fix tests

* Adds `TrieBackendBuilder` and enhances the tests

* Ensure that recorder drain checks that values are found as expected

* Switch over to `TrieBackendBuilder`

* Start fixing the problem with child tries and recording

* Fix recording of child tries

* Make it compile

* Overwrite `storage_hash` in `TrieBackend`

* Add `storage_cache` to  the benchmarks

* Fix `no_std` build

* Speed up cache lookup

* Extend the state access benchmark to also hash a runtime

* Fix build

* Fix compilation

* Rewrite value cache

* Add lru cache

* Ensure that the cache lru works

* Value cache should not be optional

* Add support for keeping the shared node cache in its bounds

* Make the cache configurable

* Check that the cache respects the bounds

* Adds a new test

* Fixes

* Docs and some renamings

* More docs

* Start using the new recorder

* Fix more code

* Take `self` argument

* Remove warnings

* Fix benchmark

* Fix accounting

* Rip off the state cache

* Start fixing fallout after removing the state cache

* Make it compile after trie changes

* Fix test

* Add some logging

* Some docs

* Some fixups and clean ups

* Fix benchmark

* Remove unneeded file

* Use git for patching

* Make CI happy

* Update primitives/trie/Cargo.toml

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update primitives/state-machine/src/trie_backend.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Introduce new `AsTrieBackend` trait

* Make the LocalTrieCache not clonable

* Make it work in no_std and add docs

* Remove duplicate dependency

* Switch to ahash for better performance

* Speedup value cache merge

* Output errors on underflow

* Ensure the internal LRU map doesn't grow too much

* Use const fn to calculate the value cache element size

* Remove cache configuration

* Fix

* Clear the cache in between for more testing

* Try to come up with a failing test case

* Make the test fail

* Fix the child trie recording

* Make everything compile after the changes to trie

* Adapt to latest trie-db changes

* Fix on stable

* Update primitives/trie/src/cache.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Fix wrong merge

* Docs

* Fix warnings

* Cargo.lock

* Bump pin-project

* Fix warnings

* Switch to released crate version

* More fixes

* Make clippy and rustdocs happy

* More clippy

* Print error when using deprecated `--state-cache-size`

* 🤦

* Fixes

* Fix storage_hash linkings

* Update client/rpc/src/dev/mod.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Review feedback

* encode bound

* Rework the shared value cache

Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.

* Improve of the cache by merging the old and new solution

* FMT

* Please stop coming back all the time :crying:

* Update primitives/trie/src/cache/shared_cache.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Fixes

* Make clippy happy

* Ensure we don't deadlock

* Only use one lock to simplify the code

* Do not depend on `Hasher`

* Fix tests

* FMT

* Clippy 🤦

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
2022-08-18 18:59:22 +00:00
Gavin Wood ecbd65fb95 Safe and sane multi-item storage removal (#11490)
* Fix overlay prefix removal result

* Second part of the overlay prefix removal fix.

* Report only items deleted from storage in clear_prefix

* Fix kill_prefix

* Formatting

* Remove unused code

* Fixes

* Fixes

* Introduce clear_prefix host function v3

* Formatting

* Use v2 for now

* Fixes

* Formatting

* Docs

* Child prefix removal should also hide v3 for now

* Fixes

* Fixes

* Formatting

* Fixes

* apply_to_keys_whle takes start_at

* apply_to_keys_whle takes start_at

* apply_to_keys_whle takes start_at

* Cursor API; force limits

* Use unsafe deprecated functions

* Formatting

* Fixes

* Grumbles

* Fixes

* Docs

* Some nitpicks 🙈

* Update primitives/externalities/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Formatting

* Fixes

* cargo fmt

* Fixes

* Update primitives/io/src/lib.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Formatting

* Fixes

Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2022-05-29 12:56:26 +01:00
dependabot[bot] 4a3758d9b8 Bump log from 0.4.16 to 0.4.17 (#11369)
* Bump log from 0.4.16 to 0.4.17

Bumps [log](https://github.com/rust-lang/log) from 0.4.16 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits/0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-05-15 22:21:04 -04:00
dependabot[bot] fcfcb06ce6 Bump pretty_assertions from 1.0.0 to 1.2.1 (#11360)
Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 1.0.0 to 1.2.1.
- [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases)
- [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/compare/v1.0.0...v1.2.1)

---
updated-dependencies:
- dependency-name: pretty_assertions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06 12:47:50 +02:00
Shawn Tabrizi 7d5202341b Basic TOML Lint (#11348)
* basic lint

* lint ordering
2022-05-04 13:38:54 +00:00
dependabot[bot] 5c4fa55f09 Bump log from 0.4.14 to 0.4.16 (#11236)
Bumps [log](https://github.com/rust-lang/log) from 0.4.14 to 0.4.16.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-20 15:39:28 +02:00
cheme 800cc1d4be State migration rpc (#10981)
* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* migration pallet

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* fix test

* fix child migration calls.

* useless closure

* remove remaining state_hash variables.

* Fix and add more tests

* Remove outdated comment

* useless inner hash

* fmt

* remote tests

* finally ksm works

* batches are broken

* clean the benchmarks

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>

* Update frame/state-trie-migration/src/lib.rs

* brand new version

* fix build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update primitives/storage/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* new test structure

* new testing stuff from emeric

* Add commit_all, still not working

* Fix all tests

* add comment

* we have PoV tracking baby

* document stuff, but proof size is still wrong

* FUCK YEAH

* a big batch of review comments

* add more tests

* tweak test

* update config

* some remote-ext stuff

* delete some of the old stuff

* sync more files with master to minimize the diff

* Fix all tests

* make signed migration a bit more relaxed

* add witness check to signed submissions

* allow custom migration to also go above limit

* Fix these pesky tests

* ==== removal of the unsigned stuff ====

* Make all tests work again

* separate the tests from the logic so it can be reused easier

* fix overall build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Slightly better termination

* some final tweaks

* Fix tests

* Restrict access to signed migrations

* mig rpc

* fix

* better rpc name

* Make rpc unsafe

* address most of the review comments

* fix defensive

* New simplified code

* Fix weights

* fmt

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* make the tests correctly fail

* Fix build

* Fix build

* try and fix the benchmarks

* fix build

* Fix cargo file

* Fix runtime deposit

* make rustdoc happy

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* update rpc deps, try to process empty keys.

* move rpc crate

* move check backend out of state machine

* Add primitive crate.

* module code

* fix runtime test

* StateMigrationStatusProvider

* Pass backend to rpc.

* fmt

* review changes

* move rpc crate

* try remove primitive crate

* Update utils/frame/rpc/state-trie-migration-rpc/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* review changes.

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
2022-03-16 11:43:24 +00:00
Andrew Jones a3a7fe9041 Release primitive crates (#10937)
* cargo set-version --bump major -p pallet-contracts-primitives

* cargo set-version --bump major -p sp-core

* cargo set-version --bump major -p sp-runtime-interface

* cargo set-version --bump major -p sp-wasm-interface

* cargo set-version --bump major -p sp-runtime

* cargo set-version --bump major -p sp-storage

* cargo set-version --bump major -p sp-rpc

* cargo set-version --bump major -p sp-io

* cargo set-version --bump major -p sp-trie

* cargo set-version -p sp-state-machine  -- 0.12.0

* cargo set-version -p sp-externalities  -- 0.12.0

* cargo set-version -p sp-keystore -- 0.12.0

* cargo set-version --bump major -p sp-keyring

* cargo set-version --bump major -p sp-version

* cargo set-version --bump major -p sp-tracing

* cargo set-version --bump major -p sp-application-crypto

* cargo set-version --bump major -p sp-arithmetic

* cargo unleash version bump-major -p sp-runtime-interface-proc-macro

* Add codec max-encoded-len feature to sp-arithmetic

* cargo unleash version bump-major -p sp-core-hashing-proc-macro
2022-03-01 14:35:23 +01:00
dependabot[bot] 5ae3ed3b95 Bump smallvec from 1.7.0 to 1.8.0 (#10949)
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.7.0...v1.8.0)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-01 10:03:31 +00:00
wigy 535325d2e6 Upgrading parity-scale-codec to v3 (#10825)
* Upgraded dependencies

* Adapting code to scale v3

* Empty commit to trigger CI

* Triggering CI

* Fixing UI test

* Remove superfluous dev-dep added by #9228

* Cryout for CI
2022-02-24 15:34:34 +01:00
Alexander Theißen 3938c1cb62 Release pallet-contracts-primitives (#10772)
* Bump sp-storage

* Bump sp-rpc

* Bump sp-io

* Bump sp-trie

* Bump sp-state-machine

* Bump sp-externalities

* Bump sp-keystore

* Bump sp-application-crypto

* Bump pallet-contracts-primitives

* Bump sp-core

* Bump sp-runtime-interface

* Bump sp-wasm-interface

* Bump sp-runtime

* Bump sp-storage

* Update lock file

* Revert "Bump sp-storage"

This reverts commit f86c6db1ae334e28207b658fd90714ade0332e3b.

* Update lock file
2022-02-01 20:38:02 +01:00
Nazar Mokrynskyi 6b60c3dbff Relax types on DigestItemRef, such that byte slices can be used in addition to vector references (#10536)
* Relax types on `DigestItemRef`, such that byte slices can be used in addition to vector references

* Apply clippy suggestions
2022-01-20 11:02:22 +01:00
cheme 4c651637f2 Inner hashing of value in state trie (runtime versioning). (#9732)
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* useless closure

* remove remaining state_hash variables.

* Remove outdated comment

* useless inner hash

* fmt

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* Apply review change (docs and error).

* fmt

* use explicit runtime_interface in doc test

* fix ui test

* fix doc test

* fmt

* use default for path and specname when resolving version.

* small review related changes.

* doc value size requirement.

* rename old_state feature

* Remove macro changes

* feature rename

* state version as host function parameter

* remove flag for client api

* fix tests

* switch storage chain proof to V1

* host functions, pass by state version enum

* use WrappedRuntimeCode

* start

* state_version in runtime version

* rust fmt

* Update storage proof of max size.

* fix runtime version rpc test

* right intent of convert from compat

* fix doc test

* fix doc test

* split proof

* decode without replay, and remove some reexports.

* Decode with compatibility by default.

* switch state_version to u8. And remove RuntimeVersionBasis.

* test

* use api when reading embedded version

* fix decode with apis

* extract core version instead

* test fix

* unused import

* review changes.

Co-authored-by: kianenigma <kian@parity.io>
2021-12-24 08:54:07 +00:00
dependabot[bot] b3648e88e8 Bump parking_lot from 0.11.1 to 0.11.2 (#10335)
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.11.1...0.11.2)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-15 13:45:30 +01:00
David edf73199ab Prepare sp-runtime for publication (#10451)
* Bump versions of sp-core and dependencies to v4.0.0

* Update references from `4.0.0-dev` –> `4.0.0`

* Funny whitespace

* Funny whitespace 2

* Prepare `sp-runtime` for publication
2021-12-15 12:36:17 +01:00
Bastian Köcher f6f58f95e1 inspect-key: Adds support for expect-public (#10430)
* Introduce `SecretUri`

* `inspect-key`: Adds support for `expect-public`

`expect-public` can be used to check that a given secret uri corresponds to the given public key.
This is mainly useful when the secret uri is protected by a password and a new derived account
should be generated. With `--expect-public` the user can pass the public key/account-id of the
"base" secret uri aka the one without any derivation to ensure the correct password was inserted.

* Fixes

* 🤦

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Review feedback

* FMT

* Bump the versions

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-12-11 08:13:18 +01:00
David 92dc635f3c Release sp-core 4.0.0 (#10447)
* Bump versions of sp-core and dependencies to v4.0.0

* Update references from `4.0.0-dev` –> `4.0.0`

* Funny whitespace

* Funny whitespace 2
2021-12-08 21:06:50 +01:00
dependabot[bot] 423cde8ad0 Bump hex-literal from 0.3.3 to 0.3.4 (#10246)
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.3.3 to 0.3.4.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.3.3...hex-literal-v0.3.4)

---
updated-dependencies:
- dependency-name: hex-literal
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-13 00:32:44 +00:00
dependabot[bot] 4aae801ccf Bump thiserror from 1.0.26 to 1.0.30 (#10240)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.26 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.26...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-11 23:32:27 +00:00
Benjamin Kampmann 2168a6d0c9 Intend to reactivate cargo-unleash check (#10167)
* Intend to reactivate cargo-unleash check

It appears the bug it was deactivated for has been resolved a while ago. Trying to reactivate the checks.

* adding missing cargo.toml metadata for BEEFY crates

* fix wrong version reference

* matching up versions

* disable faulty cache

* switching more versions to prerelease

* Revert "disable faulty cache"

This reverts commit 411a12ae444a9695a8bfea4458a868438d870b06.

* bump minor of sc-allocator to fix already-published-issue

* fixup another pre-released dependency problem

* temp switch to latest unleash

* fixing dependency version and features

* prometheus endpoint has also been changed

* fixing proposer metrics versioning

* fixing hex feature for beefy

* fix generate-bags feature selection

* fixup Cargo.lock

* upgrade prometheus dependencies

* missed one

* switch to latest release
2021-11-10 17:30:24 +01:00
Bastian Köcher b08d3017a8 Switch to Rust 2021 (#10170)
* Switch to Rust 2021

* Update trybuild to fix errors
2021-11-04 19:54:48 +01:00
Dan Shields 8db7ec9cc9 Migrate to docs.substrate.io throughout (#10007)
* all reverences of substrate.dev replaced with *correct* docs.substrate.io links

* fix links in non-cargo files, errors in replace

* homepage https://substrate.io

* cargo +nightly fmt

* FMT

Co-authored-by: Bastian Köcher <info@kchr.de>
2021-10-13 12:56:56 -06:00
dependabot[bot] ee7b4f06d7 Bump hex-literal from 0.3.1 to 0.3.3 (#9974)
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.3.1 to 0.3.3.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.3.1...hex-literal-v0.3.3)

---
updated-dependencies:
- dependency-name: hex-literal
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-09 14:23:57 +00:00
dependabot[bot] cec88b7144 Bump tracing from 0.1.25 to 0.1.29 (#9956)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.25 to 0.1.29.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.25...tracing-0.1.29)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-07 17:18:13 +02:00
Squirrel 07761c22f7 remove dep on unmaintained difference crate (#9946) 2021-10-06 18:56:37 +02:00
Bastian Köcher 2deed49706 Update wasmtime to 0.29.0 (#9552)
* Start

* Move to ctx

* Make it compile for now

* More work

* Get rid off state-holder

* Use less Refcells

* 🤦

* Don't use RefCell

* Use names for parameters

* Fixes after merge

* Fixes after merge

* Review feedback

* FMT
2021-09-29 12:30:46 +00:00
dependabot[bot] 9605a6db3a Bump smallvec from 1.6.1 to 1.7.0 (#9894)
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-29 09:54:10 +00:00
Guillaume Thiolliere be3c59f59c remove unused state machine (#9747) 2021-09-10 16:32:46 +02:00
cheme 0a8b3ddedc Fix compact proof decoding unaccessed last child trie. (#9715)
* fix no child proof attached but root included.

* small stress test for proof of child tries.

* rust fmt
2021-09-07 16:21:42 +00:00