mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-04-26 15:47:58 +00:00
4eae8dc1cd815e467d87ca875ff788f4d330aa9f
29 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Koute
|
f8e3bdad3d |
Rework storage iterators (#13284)
* Rework storage iterators * Make sure storage iteration is also accounted for when benchmarking * Use `trie-db` from crates.io * Appease clippy * Bump `trie-bench` to 0.35.0 * Fix tests' compilation * Update comment to clarify how `IterArgs::start_at` works * Add extra tests * Fix iterators on `Client` so that they behave as before * Add extra `unwrap`s in tests * More clippy fixes * Come on clippy, give me a break already * Rename `allow_missing` to `stop_on_incomplete_database` * Add `#[inline]` to `with_recorder_and_cache` * Use `with_recorder_and_cache` in `with_trie_db`; add doc comment * Simplify code: use `with_trie_db` in `next_storage_key_from_root` * Remove `expect`s in the benchmarking CLI * Add extra doc comments * Move `RawIter` before `TrieBackendEssence` (no code changes; just cut-paste) * Remove a TODO in tests * Update comment for `StorageIterator::was_complete` * Update `trie-db` to 0.25.1 |
||
|
Vivek Pandya
|
bc53b9a03a |
Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022 * Fix incorrect update of copyright year * Remove years from copy right header * Fix remaining files * Fix typo in a header and remove update-copyright.sh |
||
|
Shawn Tabrizi
|
1b27ae9549 |
Add Proof Size to Weight Output (#11637)
* initial impl * add template test * linear fit proof size * always record proof when tracking storage * calculate worst case pov * remove duplicate worst case * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs * more comment output * add cli for worst case map size * update name * clap does not support underscores * rename * expose worst case map values * improve some comments * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs * update template * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix fmt * more fmt * more fmt * Dont panic when there is no proof Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix test features Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Whitelist :extrinsic_index Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use whitelist when recording proof Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add logs Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add PoV testing pallet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Deploy PoV testing pallet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Storage benches reside in the PoV pallet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Linear regress PoV per component Splits the PoV calculation into "measured" and "estimated". The measured part is reported by the Proof recorder and linear regressed over all components at once. The estimated part is calculated as worst-case by using the max PoV size per storage access and calculating one linear regress per component. This gives each component a (possibly) independent PoV. For now the measured size will always be lower than the PoV on Polkadot since it is measured on an empty snapshot. The measured part is therefor only used as diagnostic for debugging. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Put PoV into the weight templates Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Extra alanysis choise for PoV Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add+Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Make benches faster Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use same template comments Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/bench-bot.sh" pallet dev pallet_balances * ".git/.scripts/bench-bot.sh" pallet dev pallet_democracy * Update referenda mock BlockWeights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Take measured value size into account Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/bench-bot.sh" pallet dev pallet_scheduler * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * proof_size: None Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ugly, but works Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * wup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * WIP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add pov_mode attribute to the benchmarks! macro Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use pov_mode attribute in PoV benchmarking Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Scheduler, Whitelist: Add pov_mode attr Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update PoV weights * Add CLI arg: default-pov-mode Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Update PoV weights" This reverts commit 2f3ac2387396470b118122a6ff8fa4ee12216f4b. * Revert "WIP" This reverts commit c34b538cd2bc45da4544e887180184e30957904a. * Revert first approach This reverts commit range 8ddaa2fffe5930f225a30bee314d0b7c94c344dd^..4c84f8748e5395852a9e0e25b0404953fee1a59e Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add extra benchmarks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_alliance * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_whitelist * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_scheduler * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Clippy 🤦 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add reference benchmarks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix doc comments Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Undo logging Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add 'Ignored' pov_mode Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Allow multiple attributes per benchmark Turns out that the current benchmarking syntax does not support multiple attributes per bench 🤦. Changing it to support that since otherwise the `pov_mode` would conflict with the others. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Validate pov_mode syntax Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Ignore PoV for all contract benchmarks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Bump macro recursion limit Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update contract weights They dont have a PoV component anymore. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix test ffs Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * pov_mode is unsupported in V2 syntax Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix pallet ui tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * update pallet ui Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix pallet ui tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> Co-authored-by: Your Name <you@example.com> |
||
|
Kian Paimani
|
c1ed1901e1 |
Try-runtime Revamp and Facelift (#12537)
* fix online/offline confusion * unified cache file * multi-threaded babyyy * checkpoint for niklas * compiles * all tests pass with --test-threads 1 * child-tree scrape is also multi-threaded now. * better thread splitting * some suggestions (#12532) * some suggestions * tokio multithread * move unused dependencies * snapshot command * fix rem * a bit of cleanup * support optional checks * fix * OCW command migrated to wasm-only, as an example * state-version management fully in remote-ext * almost everything move to wasm executor, some CLI flags reduced * follow-chain works as well * Master.into() * everything builds now * concurrent insertion and download for remote builds * minor fix * fix a bug * checkpoint * some updates * fmt * review comments * fmt * fix * fmt * update * fmt * rename * fix the damn UI tests * fmt * remoe the thread abstraction for the time being * cleanup * fix CI * fmt * fix * fix a few more things * tweak log levels * better error handling * address grumbles: use futures::mpsc * review comments * fmt * Apply suggestions from code review Co-authored-by: Bastian Köcher <git@kchr.de> * Update utils/frame/try-runtime/cli/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * better api version stuff * some doc update * a whole lot of docs * fmt * fix all docs * fmt * rpc rebase: Try-runtime Revamp and Facelift (#12921) * Introduce sensible weight constants (#12868) * Introduce sensible weight constants * cargo fmt * Remove unused import * Add missing import * ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery Co-authored-by: command-bot <> * Checkout to the branch HEAD explicitly in `build-linux-substrate` (#12876) * cli: Improve pruning documentation (#12819) * cli: Improve pruning documentation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Keep `finalized` notation and remove `canonical` one * cli: Fix cargo doc * cli: `PruningModeClap` IR enum Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Convert PruningModeClap into pruning modes Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Use `PruningModeClap` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Rename to `DatabasePruningMode` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Implement `FromStr` instead of `clap::ValueEnum` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update client/cli/src/params/pruning_params.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Fix clippy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Add option documentation back Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Apply suggestions from code review Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> * Revert "Move LockableCurrency trait to fungibles::Lockable and deprecate LockableCurrency (#12798)" (#12882) This reverts commit |
||
|
Sebastian Kunert
|
f687db40f7 |
Bump clap to 4.0.x and adjust to best practices (#12381)
* Bump clap to 3.2.22 * Replace `from_os_str` with `value_parser` * Replace `from_str` and `try_from_str` with `value_parser` * Move possible_values to the new format * Remove unwanted print * Add missing match branch * Update clap to 4.0.9 and make it compile * Replace deprecated `clap` macro with `command` and `value` * Move remaining `clap` attributes to `arg` * Remove no-op value_parsers * Adjust value_parser for state_version * Remove "deprecated" feature flag and bump to 4.0.11 * Improve range Co-authored-by: Bastian Köcher <git@kchr.de> * Apply suggestions * Trigger CI * Fix unused error warning * Fix doc errors * Fix ArgGroup naming conflict * Change default_value to default_value_t * Use 1.. instead of 0.. Co-authored-by: Bastian Köcher <git@kchr.de> |
||
|
Sam Johnson
|
c0e007b50f |
Add special tag to exclude runtime storage items from benchmarking (#12205)
* initial setup * add WhitelistedStorageKeys trait * add (A, B) tuple implementation for whitelisted_storage_keys() * fix formatting * implement WhitelistedStorageKeys for all tuple combinations * impl_for_tuples up to 128 for WhitelistedStorageKeys * refactor to #[benchmarking(cached)] * tweak error message and mark BlockNumber as cached * add benchmarking(cached) to the other default types * add docs for benchmarking(cached) * properly parse storage type declaration * make storage_alias structs public so we can use them in this macro * use BTreeMap since TrackedStorageKey missing Ord outside of std * make WhitelistedStorageKeys accessible * basic detection of benchmarking(cached) 💥 * proper parsing of #[benchmarking(cached)] from pallet parse macro * store presence of #[benchmarking(cached)] macro on StorageDef * will be used for later expansion * compiling blank impl for WhitelistedStorageKeys * move impl to expand_pallet_struct * use frame_support::sp_std::vec::Vec properly * successfully compiling with storage info loaded into a variable 💥 * plausible implementation for whitelisted_storage_keys() * depends on the assumption that storage_info.encode() can be loaded into TrackedStorageKey::new(..) * use Pallet::whitelisted_storage_keys() instead of hard-coded list * AllPallets::whitelisted_storage_keys() properly working 💥 * collect storage names * whitelisted_storage_keys() impl working 💥 * clean up * fix compiler error * just one compiler error * fix doc compiler error * use better import path * fix comment * whoops * whoops again * fix macro import issue * cargo fmt * mark example as ignore * use keyword tokens instead of string parsing * fix keyword-based parsing of benchmarking(cached) * preliminary spec for check_whitelist() * add additional test for benchmarking whitelist * add TODO note * remove irrelevant line from example * use filter_map instead of filter and map * simplify syntax Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> * clean up * fix test * fix tests * use keyword parsing instead of string parsing * use collect() instead of a for loop Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * fix compiler error * clean up benchmarking(cached) marking code * use cloned() * refactor to not use panic! and remove need for pub types in storage_alias * remove unneeded use Co-authored-by: Bastian Köcher <info@kchr.de> * remove unneeded visibility changes * don't manually hard code hash for treasury account as hex * proper Ord, PartialOrd, and Hash impls for TrackedStorageKey * now based just on key, and available in no-std * use BTreeSet instead of BTreeMap * fix comments * cargo fmt * switch to pallet::whitelist and re-do it basti's way :D * make PartialOrd for TrackedStorageKey consistent with Ord * more correct implementation of hash-related traits for TrackedStorageKey * fix integration test * update TODO * remove unused keyword * remove more unused keywords * use into_iter() Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> * Update frame/support/procedural/src/pallet/parse/mod.rs Co-authored-by: Bastian Köcher <info@kchr.de> * add docs for whitelisted * fix comment Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Bastian Köcher <info@kchr.de> |
||
|
Falco Hirschenberger
|
b581604aa7 |
Apply some clippy lints (#11154)
* Apply some clippy hints * Revert clippy ci changes * Update client/cli/src/commands/generate.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/cli/src/commands/inspect_key.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/db/src/bench.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/db/src/bench.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/service/src/client/block_rules.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/service/src/client/block_rules.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/network/src/transactions.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/network/src/protocol.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Revert due to missing `or_default` function. * Fix compilation and simplify code * Undo change that corrupts benchmark. * fix clippy * Update client/service/test/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/state-db/src/noncanonical.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/state-db/src/noncanonical.rs remove leftovers! * Update client/tracing/src/logging/directives.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update utils/fork-tree/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * added needed ref * Update frame/referenda/src/benchmarking.rs * Simplify byte-vec creation * let's just not overlap the ranges * Correction * cargo fmt * Update utils/frame/benchmarking-cli/src/shared/stats.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update utils/frame/benchmarking-cli/src/pallet/command.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update utils/frame/benchmarking-cli/src/pallet/command.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Giles Cope <gilescope@gmail.com> |
||
|
Qinxuan Chen
|
41c84b0b41 |
Remove useless imports (#11136)
Because `TryInto`/`TryFrom` are in prelude by default from edition 2021 Signed-off-by: koushiro <koushiro.cqx@gmail.com> |
||
|
Kian Paimani
|
6cfdbe5caa |
Trie version migration pallet (#10073)
* starting * Updated from other branch. * setting flag * flag in storage struct * fix flagging to access and insert. * added todo to fix * also missing serialize meta to storage proof * extract meta. * Isolate old trie layout. * failing test that requires storing in meta when old hash scheme is used. * old hash compatibility * Db migrate. * runing tests with both states when interesting. * fix chain spec test with serde default. * export state (missing trie function). * Pending using new branch, lacking genericity on layout resolution. * extract and set global meta * Update to branch 4 * fix iterator with root flag (no longer insert node). * fix trie root hashing of root * complete basic backend. * Remove old_hash meta from proof that do not use inner_hashing. * fix trie test for empty (force layout on empty deltas). * Root update fix. * debug on meta * Use trie key iteration that do not include value in proofs. * switch default test ext to use inner hash. * small integration test, and fix tx cache mgmt in ext. test failing * Proof scenario at state-machine level. * trace for db upgrade * try different param * act more like iter_from. * Bigger batches. * Update trie dependency. * drafting codec changes and refact * before removing unused branch no value alt hashing. more work todo rename all flag var to alt_hash, and remove extrinsic replace by storage query at every storage_root call. * alt hashing only for branch with value. * fix trie tests * Hash of value include the encoded size. * removing fields(broken) * fix trie_stream to also include value length in inner hash. * triedbmut only using alt type if inner hashing. * trie_stream to also only use alt hashing type when actually alt hashing. * Refactor meta state, logic should work with change of trie treshold. * Remove NoMeta variant. * Remove state_hashed trigger specific functions. * pending switching to using threshold, new storage root api does not make much sense. * refactoring to use state from backend (not possible payload changes). * Applying from previous state * Remove default from storage, genesis need a special build. * rem empty space * Catch problem: when using triedb with default: we should not revert nodes: otherwhise thing as trie codec cannot decode-encode without changing state. * fix compilation * Right logic to avoid switch on reencode when default layout. * Clean up some todos * remove trie meta from root upstream * update upstream and fix benches. * split some long lines. * UPdate trie crate to work with new design. * Finish update to refactored upstream. * update to latest triedb changes. * Clean up. * fix executor test. * rust fmt from master. * rust format. * rustfmt * fix * start host function driven versioning * update state-machine part * still need access to state version from runtime * state hash in mem: wrong * direction likely correct, but passing call to code exec for genesis init seem awkward. * state version serialize in runtime, wrong approach, just initialize it with no threshold for core api < 4 seems more proper. * stateversion from runtime version (core api >= 4). * update trie, fix tests * unused import * clean some TODOs * Require RuntimeVersionOf for executor * use RuntimeVersionOf to resolve genesis state version. * update runtime version test * fix state-machine tests * TODO * Use runtime version from storage wasm with fast sync. * rustfmt * fmt * fix test * revert useless changes. * clean some unused changes * fmt * removing useless trait function. * remove remaining reference to state_hash * fix some imports * Follow chain state version management. * trie update, fix and constant threshold for trie layouts. * update deps * Update to latest trie pr changes. * fix benches * Verify proof requires right layout. * update trie_root * Update trie deps to latest * Update to latest trie versioning * Removing patch * update lock * extrinsic for sc-service-test using layout v0. * Adding RuntimeVersionOf to CallExecutor works. * fmt * error when resolving version and no wasm in storage. * use existing utils to instantiate runtime code. * migration pallet * Patch to delay runtime switch. * Revert "Patch to delay runtime switch." This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da. * fix test * fix child migration calls. * useless closure * remove remaining state_hash variables. * Fix and add more tests * Remove outdated comment * useless inner hash * fmt * remote tests * finally ksm works * batches are broken * clean the benchmarks * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com> * Update frame/state-trie-migration/src/lib.rs * brand new version * fix build * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Update primitives/storage/src/lib.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * fmt and opt-in feature to apply state change. * feature gate core version, use new test feature for node and test node * Use a 'State' api version instead of Core one. * fix merge of test function * use blake macro. * Fix state api (require declaring the api in runtime). * Opt out feature, fix macro for io to select a given version instead of latest. * run test nodes on new state. * fix * new test structure * new testing stuff from emeric * Add commit_all, still not working * Fix all tests * add comment * we have PoV tracking baby * document stuff, but proof size is still wrong * FUCK YEAH * a big batch of review comments * add more tests * tweak test * update config * some remote-ext stuff * delete some of the old stuff * sync more files with master to minimize the diff * Fix all tests * make signed migration a bit more relaxed * add witness check to signed submissions * allow custom migration to also go above limit * Fix these pesky tests * ==== removal of the unsigned stuff ==== * Make all tests work again * separate the tests from the logic so it can be reused easier * fix overall build * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Slightly better termination * some final tweaks * Fix tests * Restrict access to signed migrations * address most of the review comments * fix defensive * New simplified code * Fix weights * fmt * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * make the tests correctly fail * Fix build * Fix build * try and fix the benchmarks * fix build * Fix cargo file * Fix runtime deposit * make rustdoc happy * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Parity Bot <admin@parity.io> |
||
|
Bastian Köcher
|
f3662b4bba | Happy new year 2022 (#10573) | ||
|
cheme
|
4c651637f2 |
Inner hashing of value in state trie (runtime versioning). (#9732)
* starting * Updated from other branch. * setting flag * flag in storage struct * fix flagging to access and insert. * added todo to fix * also missing serialize meta to storage proof * extract meta. * Isolate old trie layout. * failing test that requires storing in meta when old hash scheme is used. * old hash compatibility * Db migrate. * runing tests with both states when interesting. * fix chain spec test with serde default. * export state (missing trie function). * Pending using new branch, lacking genericity on layout resolution. * extract and set global meta * Update to branch 4 * fix iterator with root flag (no longer insert node). * fix trie root hashing of root * complete basic backend. * Remove old_hash meta from proof that do not use inner_hashing. * fix trie test for empty (force layout on empty deltas). * Root update fix. * debug on meta * Use trie key iteration that do not include value in proofs. * switch default test ext to use inner hash. * small integration test, and fix tx cache mgmt in ext. test failing * Proof scenario at state-machine level. * trace for db upgrade * try different param * act more like iter_from. * Bigger batches. * Update trie dependency. * drafting codec changes and refact * before removing unused branch no value alt hashing. more work todo rename all flag var to alt_hash, and remove extrinsic replace by storage query at every storage_root call. * alt hashing only for branch with value. * fix trie tests * Hash of value include the encoded size. * removing fields(broken) * fix trie_stream to also include value length in inner hash. * triedbmut only using alt type if inner hashing. * trie_stream to also only use alt hashing type when actually alt hashing. * Refactor meta state, logic should work with change of trie treshold. * Remove NoMeta variant. * Remove state_hashed trigger specific functions. * pending switching to using threshold, new storage root api does not make much sense. * refactoring to use state from backend (not possible payload changes). * Applying from previous state * Remove default from storage, genesis need a special build. * rem empty space * Catch problem: when using triedb with default: we should not revert nodes: otherwhise thing as trie codec cannot decode-encode without changing state. * fix compilation * Right logic to avoid switch on reencode when default layout. * Clean up some todos * remove trie meta from root upstream * update upstream and fix benches. * split some long lines. * UPdate trie crate to work with new design. * Finish update to refactored upstream. * update to latest triedb changes. * Clean up. * fix executor test. * rust fmt from master. * rust format. * rustfmt * fix * start host function driven versioning * update state-machine part * still need access to state version from runtime * state hash in mem: wrong * direction likely correct, but passing call to code exec for genesis init seem awkward. * state version serialize in runtime, wrong approach, just initialize it with no threshold for core api < 4 seems more proper. * stateversion from runtime version (core api >= 4). * update trie, fix tests * unused import * clean some TODOs * Require RuntimeVersionOf for executor * use RuntimeVersionOf to resolve genesis state version. * update runtime version test * fix state-machine tests * TODO * Use runtime version from storage wasm with fast sync. * rustfmt * fmt * fix test * revert useless changes. * clean some unused changes * fmt * removing useless trait function. * remove remaining reference to state_hash * fix some imports * Follow chain state version management. * trie update, fix and constant threshold for trie layouts. * update deps * Update to latest trie pr changes. * fix benches * Verify proof requires right layout. * update trie_root * Update trie deps to latest * Update to latest trie versioning * Removing patch * update lock * extrinsic for sc-service-test using layout v0. * Adding RuntimeVersionOf to CallExecutor works. * fmt * error when resolving version and no wasm in storage. * use existing utils to instantiate runtime code. * Patch to delay runtime switch. * Revert "Patch to delay runtime switch." This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da. * useless closure * remove remaining state_hash variables. * Remove outdated comment * useless inner hash * fmt * fmt and opt-in feature to apply state change. * feature gate core version, use new test feature for node and test node * Use a 'State' api version instead of Core one. * fix merge of test function * use blake macro. * Fix state api (require declaring the api in runtime). * Opt out feature, fix macro for io to select a given version instead of latest. * run test nodes on new state. * fix * Apply review change (docs and error). * fmt * use explicit runtime_interface in doc test * fix ui test * fix doc test * fmt * use default for path and specname when resolving version. * small review related changes. * doc value size requirement. * rename old_state feature * Remove macro changes * feature rename * state version as host function parameter * remove flag for client api * fix tests * switch storage chain proof to V1 * host functions, pass by state version enum * use WrappedRuntimeCode * start * state_version in runtime version * rust fmt * Update storage proof of max size. * fix runtime version rpc test * right intent of convert from compat * fix doc test * fix doc test * split proof * decode without replay, and remove some reexports. * Decode with compatibility by default. * switch state_version to u8. And remove RuntimeVersionBasis. * test * use api when reading embedded version * fix decode with apis * extract core version instead * test fix * unused import * review changes. Co-authored-by: kianenigma <kian@parity.io> |
||
|
Kian Paimani
|
4775f11edc |
allow try-runtime and TestExternalities to report PoV size (#10372)
* allow try-runtime and test-externalities to report proof size * self review * fix test * Fix humanized dispaly of bytes * Fix some test * Fix some review grumbles * last of the review comments * fmt * remove unused import * move test * fix import * Update primitives/state-machine/src/testing.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * last touches * fix Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> |
||
|
Arkadiy Paronyan
|
4cbbf0cf43 |
Kill the light client, CHTs and change tries. (#10080)
* Remove light client, change tries and CHTs * Update tests * fmt * Restore changes_root * Fixed benches * Cargo fmt * fmt * fmt |
||
|
Bastian Köcher
|
7b56ab15b4 |
Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base * Second run * Add CI check * Fix compilation * More unnecessary braces * Handle weights * Use --all * Use correct attributes... * Fix UI tests * AHHHHHHHHH * 🤦 * Docs * Fix compilation * 🤷 * Please stop * 🤦 x 2 * More * make rustfmt.toml consistent with polkadot Co-authored-by: André Silva <andrerfosilva@gmail.com> |
||
|
Shawn Tabrizi
|
b42b8fc5fb |
Include StorageInfo in Benchmarking Pipeline (#9090)
* extend storageinfo * extend_storage_info * use vec * add storage info to pipeline * get read and written keys * undo storageinfo move * refactor keytracker * return read / write count * playing with key matching * add basic `StorageInfo` constructor * add whitelisted to returned info * fix some test stuff * pipe comments into benchmark data * add_storage_comments * add comments to template * track only storage prefix * Update frame/benchmarking/src/lib.rs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix test * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs * remove test logs * add temp benchmark script * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * remove keytracker and use trackedstoragekey * add comment for unknown keys * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_timestamp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/timestamp/src/weights.rs --template=./.maintain/frame-weight-template.hbs * remove duplicate comments with unknown keys * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_timestamp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/timestamp/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs * refactor bench tracker, and fix results * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix child tries in new tracker * extra newline * fix unused warning * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_timestamp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/timestamp/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix master merge * storage info usage refactor * remove now unused * fix refactor * use a vec for prefix * fix tests * also update writer to use vec * disable read and written keys for now * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/system/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Update frame/system/src/weights.rs * fix test * Delete weights.rs * reset weights Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> |
||
|
Kian Paimani
|
b3fabde4e3 |
make remote ext use batch ws-client (#8916)
* make remote ext use batch ws-client * Add debug log for key length * better assertions * new sanity_checl * try and make it work with batch * update test * remove exctra uri * add missing at * remove unused rpc stuff * improve Co-authored-by: emostov <32168567+emostov@users.noreply.github.com> |
||
|
Kian Paimani
|
33b435894a |
Better identifier and logging for runtime upgrades (#8123)
* A clean new attempt * Checkpoint to move remote. * A lot of dependency wiring to make it feature gated. * bad macro, bad macro. * Undo the DB mess. * Update frame/support/src/traits.rs Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * Apply suggestions from code review Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * unbreak the build * Better logging and ids for migrations * Fix doc. * Test * Update frame/try-runtime/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update utils/frame/try-runtime/cli/Cargo.toml Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/try-runtime/Cargo.toml Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Address most review grumbles. * Fix build * Add some comments * Remove allowing one pallet at a time. * Rework the PR * nit * Slightly better error handling. * Remove files * Update utils/frame/remote-externalities/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update frame/support/src/dispatch.rs * Update frame/support/src/dispatch.rs * Fix test * Make extension trait. * Bring back try-runtime/std * remove bincode * Remove warning * Change test features Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> |
||
|
Kian Paimani
|
16a27c28a9 |
Migration testing runtime API/Bot (#8038)
* A clean new attempt * Checkpoint to move remote. * A lot of dependency wiring to make it feature gated. * bad macro, bad macro. * Undo the DB mess. * Update frame/support/src/traits.rs Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * Apply suggestions from code review Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * unbreak the build * Update frame/try-runtime/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update utils/frame/try-runtime/cli/Cargo.toml Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/try-runtime/Cargo.toml Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Address most review grumbles. * Fix build * Add some comments * Remove allowing one pallet at a time. * More grumbles. * relocate remote-ext * Fix build Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> |
||
|
cheme
|
1922dab050 |
Fix clear prefix check to avoid erasing child trie roots. (#7848)
* Fix clear prefix check to avoid erasing child trie roots. * Renaming and extend existing test with check. * last nitpicks. |
||
|
Bastian Köcher
|
e3e651f72c |
Happy new year (#7814)
* Happy new year Updates the copyright years and fixes wrong license headers. * Fix the template * Split HEADER into HEADER-APACHE & HEADER-GPL |
||
|
Shawn Tabrizi
|
481ad884d6 |
Dynamic Benchmarking DB Whitelist (#6815)
* Add `get_whitelist` api * add whitelisted caller * Whitelist caller * remove caller 0 * initial piping of origin (not actual value yet) * remove attempt to pass origin around * Add whitelist for `DidUpdate` storage on `pallet_timestamp` * fix traits * only add to whitelist if !contains * PassBy not implemented error * Whitelist read/writes explicitly per key * update docs * reduce trait constraint * copy pasta * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * rename functions @apopiak * missed some renaming * enable doc tests * Update docs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> |
||
|
Benjamin Kampmann
|
be8c96adec | Relax substrate licensing scheme (#5947) | ||
|
Bastian Köcher
|
bc9707b4c5 |
Use storage::append in the implementation of the storage types (#5889)
* Start improving `storage_append` * Fix some stuff * Fix compilation * Update docs and add new test * More tests * Test kill + append |
||
|
Bastian Köcher
|
5b8d3607e9 |
Adds export-state subcommand (#5842)
* Export state cli * More work * Fix tests * Make it work * Fix compilation * Apply suggestions from code review |
||
|
cheme
|
4ffcf98d8d |
Child trie api changes BREAKING (#4857)
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com> |
||
|
Bastian Köcher
|
26fa6e3c76 |
Move runtime upgrade to frame-executive (#5197)
* Move runtime upgrade to `frame-executive` Instead of storing the runtime upgraded in the space of `frame-system`, this moves it to `frame-executive`. We also start storing the `spec_version` and `impl_version` the last runtime upgrade was applied. This scheme has multiple advantages: - We don't need to make sure that runtime upgrade was set. - This will work in the future when we want to test a runtime upgrade, because the `on_runtime_upgrade` will be executed automatically when the runtime changes. * Move `LAST_RUNTIME_UPGRADE` key to `frame-executive` * Add some more documentation * Make sure `spec_version` always increases between runtime upgrades * Fix test * Upgrade `parity-multiaddr` to fix compilation * Also check if the `spec_name` changed * Remove `RuntimeUpgraded` storage entry * Don't run on genesis |
||
|
Shawn Tabrizi
|
7efa62725e | Update copyright year (#4532) | ||
|
cheme
|
0ece5d9e17 |
Fix key collision for child trie (#4162)
* In progress, runtime io must switch to future proof root + child_specific (unique id) + u32 type. * Switch interface, sr-io seems ok, rpc could use similar interface to sr-io, genesis json broken if there is child trie in existing encoding genesis. * test from previous implementation. * fix proving test. * Restore Keyspacedb from other branch, only apply to child trie. * Removing unneeded child_info from child root (child info are stored if things changed, otherwhise the root does not change). * Switch rpc to use same format as ext: more future proof. * use root from child info for trie backend essence. * Breaking long lines. * Update doc and clean pr a bit. * fix error type * Restore removed doc on merge and update sr-io doc. * Switch child storage api to use directly unique id, if managed id where to be put in place, the api will change at this time. * Clean deprecated host interface from child. * Removing assertion on child info (can fail depending on root memoization). * merging child info in the overlay when possible. * child iteration by prefix using child_info. * Using ChainInfo in frame support. ChainInfo gets redesign to avoid buffers allocation on every calls. * Add length of root to the data of child info. * comments * Encode compact. * Remove child info with root. * Fix try_update condition. * Comment Ext child root caching. * Replace tuples by struct with field * remove StorageTuple alias. * Fix doc tests, and remove StorageOverlay and ChildStorageOverlay aliases. |
||
|
Benjamin Kampmann
|
7773daaf5b |
Clean up crate names and locations (#4361)
* rename crate: sp-transaction-pool-api -> sp-transaction-pool * move primitives/core/derive-debug -> primitives/derive-debug; primitives/core/storage -> primitives/storage * rename crate sp-core-storage -> sp-storage * rename and move: test/utils/transaction-factory -> client/transaction-factory * move transaction-factory -> node/transaction-factory * fix missing rename * Move chain-spec-builder into bin/utils * move subkey into bin/utils * Update new subkey location * Update docs to reflect new location for utils * fixing import name |