* starting
* Updated from other branch.
* setting flag
* flag in storage struct
* fix flagging to access and insert.
* added todo to fix
* also missing serialize meta to storage proof
* extract meta.
* Isolate old trie layout.
* failing test that requires storing in meta when old hash scheme is used.
* old hash compatibility
* Db migrate.
* runing tests with both states when interesting.
* fix chain spec test with serde default.
* export state (missing trie function).
* Pending using new branch, lacking genericity on layout resolution.
* extract and set global meta
* Update to branch 4
* fix iterator with root flag (no longer insert node).
* fix trie root hashing of root
* complete basic backend.
* Remove old_hash meta from proof that do not use inner_hashing.
* fix trie test for empty (force layout on empty deltas).
* Root update fix.
* debug on meta
* Use trie key iteration that do not include value in proofs.
* switch default test ext to use inner hash.
* small integration test, and fix tx cache mgmt in ext.
test failing
* Proof scenario at state-machine level.
* trace for db upgrade
* try different param
* act more like iter_from.
* Bigger batches.
* Update trie dependency.
* drafting codec changes and refact
* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.
* alt hashing only for branch with value.
* fix trie tests
* Hash of value include the encoded size.
* removing fields(broken)
* fix trie_stream to also include value length in inner hash.
* triedbmut only using alt type if inner hashing.
* trie_stream to also only use alt hashing type when actually alt hashing.
* Refactor meta state, logic should work with change of trie treshold.
* Remove NoMeta variant.
* Remove state_hashed trigger specific functions.
* pending switching to using threshold, new storage root api does not
make much sense.
* refactoring to use state from backend (not possible payload changes).
* Applying from previous state
* Remove default from storage, genesis need a special build.
* rem empty space
* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.
* fix compilation
* Right logic to avoid switch on reencode when default layout.
* Clean up some todos
* remove trie meta from root upstream
* update upstream and fix benches.
* split some long lines.
* UPdate trie crate to work with new design.
* Finish update to refactored upstream.
* update to latest triedb changes.
* Clean up.
* fix executor test.
* rust fmt from master.
* rust format.
* rustfmt
* fix
* start host function driven versioning
* update state-machine part
* still need access to state version from runtime
* state hash in mem: wrong
* direction likely correct, but passing call to code exec for genesis
init seem awkward.
* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.
* stateversion from runtime version (core api >= 4).
* update trie, fix tests
* unused import
* clean some TODOs
* Require RuntimeVersionOf for executor
* use RuntimeVersionOf to resolve genesis state version.
* update runtime version test
* fix state-machine tests
* TODO
* Use runtime version from storage wasm with fast sync.
* rustfmt
* fmt
* fix test
* revert useless changes.
* clean some unused changes
* fmt
* removing useless trait function.
* remove remaining reference to state_hash
* fix some imports
* Follow chain state version management.
* trie update, fix and constant threshold for trie layouts.
* update deps
* Update to latest trie pr changes.
* fix benches
* Verify proof requires right layout.
* update trie_root
* Update trie deps to latest
* Update to latest trie versioning
* Removing patch
* update lock
* extrinsic for sc-service-test using layout v0.
* Adding RuntimeVersionOf to CallExecutor works.
* fmt
* error when resolving version and no wasm in storage.
* use existing utils to instantiate runtime code.
* migration pallet
* Patch to delay runtime switch.
* Revert "Patch to delay runtime switch."
This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.
* fix test
* fix child migration calls.
* useless closure
* remove remaining state_hash variables.
* Fix and add more tests
* Remove outdated comment
* useless inner hash
* fmt
* remote tests
* finally ksm works
* batches are broken
* clean the benchmarks
* Apply suggestions from code review
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Apply suggestions from code review
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
* Update frame/state-trie-migration/src/lib.rs
* brand new version
* fix build
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update primitives/storage/src/lib.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* fmt and opt-in feature to apply state change.
* feature gate core version, use new test feature for node and test node
* Use a 'State' api version instead of Core one.
* fix merge of test function
* use blake macro.
* Fix state api (require declaring the api in runtime).
* Opt out feature, fix macro for io to select a given version
instead of latest.
* run test nodes on new state.
* fix
* new test structure
* new testing stuff from emeric
* Add commit_all, still not working
* Fix all tests
* add comment
* we have PoV tracking baby
* document stuff, but proof size is still wrong
* FUCK YEAH
* a big batch of review comments
* add more tests
* tweak test
* update config
* some remote-ext stuff
* delete some of the old stuff
* sync more files with master to minimize the diff
* Fix all tests
* make signed migration a bit more relaxed
* add witness check to signed submissions
* allow custom migration to also go above limit
* Fix these pesky tests
* ==== removal of the unsigned stuff ====
* Make all tests work again
* separate the tests from the logic so it can be reused easier
* fix overall build
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Slightly better termination
* some final tweaks
* Fix tests
* Restrict access to signed migrations
* address most of the review comments
* fix defensive
* New simplified code
* Fix weights
* fmt
* Update frame/state-trie-migration/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* make the tests correctly fail
* Fix build
* Fix build
* try and fix the benchmarks
* fix build
* Fix cargo file
* Fix runtime deposit
* make rustdoc happy
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
* starting
* Updated from other branch.
* setting flag
* flag in storage struct
* fix flagging to access and insert.
* added todo to fix
* also missing serialize meta to storage proof
* extract meta.
* Isolate old trie layout.
* failing test that requires storing in meta when old hash scheme is used.
* old hash compatibility
* Db migrate.
* runing tests with both states when interesting.
* fix chain spec test with serde default.
* export state (missing trie function).
* Pending using new branch, lacking genericity on layout resolution.
* extract and set global meta
* Update to branch 4
* fix iterator with root flag (no longer insert node).
* fix trie root hashing of root
* complete basic backend.
* Remove old_hash meta from proof that do not use inner_hashing.
* fix trie test for empty (force layout on empty deltas).
* Root update fix.
* debug on meta
* Use trie key iteration that do not include value in proofs.
* switch default test ext to use inner hash.
* small integration test, and fix tx cache mgmt in ext.
test failing
* Proof scenario at state-machine level.
* trace for db upgrade
* try different param
* act more like iter_from.
* Bigger batches.
* Update trie dependency.
* drafting codec changes and refact
* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.
* alt hashing only for branch with value.
* fix trie tests
* Hash of value include the encoded size.
* removing fields(broken)
* fix trie_stream to also include value length in inner hash.
* triedbmut only using alt type if inner hashing.
* trie_stream to also only use alt hashing type when actually alt hashing.
* Refactor meta state, logic should work with change of trie treshold.
* Remove NoMeta variant.
* Remove state_hashed trigger specific functions.
* pending switching to using threshold, new storage root api does not
make much sense.
* refactoring to use state from backend (not possible payload changes).
* Applying from previous state
* Remove default from storage, genesis need a special build.
* rem empty space
* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.
* fix compilation
* Right logic to avoid switch on reencode when default layout.
* Clean up some todos
* remove trie meta from root upstream
* update upstream and fix benches.
* split some long lines.
* UPdate trie crate to work with new design.
* Finish update to refactored upstream.
* update to latest triedb changes.
* Clean up.
* fix executor test.
* rust fmt from master.
* rust format.
* rustfmt
* fix
* start host function driven versioning
* update state-machine part
* still need access to state version from runtime
* state hash in mem: wrong
* direction likely correct, but passing call to code exec for genesis
init seem awkward.
* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.
* stateversion from runtime version (core api >= 4).
* update trie, fix tests
* unused import
* clean some TODOs
* Require RuntimeVersionOf for executor
* use RuntimeVersionOf to resolve genesis state version.
* update runtime version test
* fix state-machine tests
* TODO
* Use runtime version from storage wasm with fast sync.
* rustfmt
* fmt
* fix test
* revert useless changes.
* clean some unused changes
* fmt
* removing useless trait function.
* remove remaining reference to state_hash
* fix some imports
* Follow chain state version management.
* trie update, fix and constant threshold for trie layouts.
* update deps
* Update to latest trie pr changes.
* fix benches
* Verify proof requires right layout.
* update trie_root
* Update trie deps to latest
* Update to latest trie versioning
* Removing patch
* update lock
* extrinsic for sc-service-test using layout v0.
* Adding RuntimeVersionOf to CallExecutor works.
* fmt
* error when resolving version and no wasm in storage.
* use existing utils to instantiate runtime code.
* Patch to delay runtime switch.
* Revert "Patch to delay runtime switch."
This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.
* useless closure
* remove remaining state_hash variables.
* Remove outdated comment
* useless inner hash
* fmt
* fmt and opt-in feature to apply state change.
* feature gate core version, use new test feature for node and test node
* Use a 'State' api version instead of Core one.
* fix merge of test function
* use blake macro.
* Fix state api (require declaring the api in runtime).
* Opt out feature, fix macro for io to select a given version
instead of latest.
* run test nodes on new state.
* fix
* Apply review change (docs and error).
* fmt
* use explicit runtime_interface in doc test
* fix ui test
* fix doc test
* fmt
* use default for path and specname when resolving version.
* small review related changes.
* doc value size requirement.
* rename old_state feature
* Remove macro changes
* feature rename
* state version as host function parameter
* remove flag for client api
* fix tests
* switch storage chain proof to V1
* host functions, pass by state version enum
* use WrappedRuntimeCode
* start
* state_version in runtime version
* rust fmt
* Update storage proof of max size.
* fix runtime version rpc test
* right intent of convert from compat
* fix doc test
* fix doc test
* split proof
* decode without replay, and remove some reexports.
* Decode with compatibility by default.
* switch state_version to u8. And remove RuntimeVersionBasis.
* test
* use api when reading embedded version
* fix decode with apis
* extract core version instead
* test fix
* unused import
* review changes.
Co-authored-by: kianenigma <kian@parity.io>
* deadlock, need to ask someone to help now
* Finally it seems to be working.. at least for a few blocks
* self-review
* major mega revamp
* some small fixes
* another mega refactor
* add license
* Apply suggestions from code review
* hack around signature verification
* Some fixes
* Update utils/frame/try-runtime/cli/src/lib.rs
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
* Update utils/frame/try-runtime/cli/src/lib.rs
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
* Update utils/frame/try-runtime/cli/src/lib.rs
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
* final tweaks, hopefully.
* a little self-review
* Add the ext root check
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Initial piping of returning amount of keys killed
* One more test for `None` limit
* forgot to update
* fix return value
* use version 3
* Update to return `KillOutcome`
* Update name to KillChildStorageResult
* A clean new attempt
* Checkpoint to move remote.
* A lot of dependency wiring to make it feature gated.
* bad macro, bad macro.
* Undo the DB mess.
* Update frame/support/src/traits.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Apply suggestions from code review
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* unbreak the build
* Update frame/try-runtime/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update utils/frame/try-runtime/cli/Cargo.toml
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Update frame/try-runtime/Cargo.toml
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Address most review grumbles.
* Fix build
* Add some comments
* Remove allowing one pallet at a time.
* More grumbles.
* relocate remote-ext
* Fix build
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Moving offchain change set to state machine overlay change set,
preparing use of change set internally.
* Make change set generic over key and value, and use it for offchain
indexing.
* test ui change
* remaining delta
* generating with standard method
* Remove 'drain_committed' function, and documentation.
* Default constructor for enabling offchain indexing.
* Remove offchain change specific iterators.
* remove pub accessor
* keep previous hierarchy, just expose iterator instead.
* Update primitives/state-machine/src/overlayed_changes/mod.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* fix line break
* missing renamings
* fix import
* fix new state-machine tests.
* Don't expose InnerValue type.
* Add test similar to set_storage.
* Remove conditional offchain storage (hard to instantiate correctly).
* fix
* offchain as children cannot fail if top doesn't
Co-authored-by: Addie Wagenknecht <addie@nortd.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Make offchain indexing work
This fixes some bugs with offchain indexing to make it actually working ;)
* Fix tests
* Fix browser build
* Update client/db/src/offchain.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Remove seperation between prefix and key
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Remove runtime registered extensions after execution
This prevents a bug when an extension was registered in native, but the
native execution aborted without removing the extension again. Now, when
executing the wasm code the extension is still registered and it fails
of being registered. So, the wasm execution fails as well. This can
happen for example for the `VerificationExt`.
* Make it better
* Add transactional storage functionality to OverlayChanges
A collection already has a natural None state. No need to
wrap it with an option.
* Add storage transactions runtime interface
* Add frame support for transactions
* Fix committed typo
* Rename 'changes' variable to 'overlay'
* Fix renaming change
* Fixed strange line break
* Rename clear to clear_where
* Add comment regarding delete value on mutation
* Add comment which changes are covered by a transaction
* Do force the arg to with_transaction return a Result
* Use rust doc comments on every documentable place
* Fix wording of insert_diry doc
* Improve doc on start_transaction
* Rename value to overlayed in close_transaction
* Inline negation
* Improve wording of close_transaction comments
* Get rid of an expect by using get_or_insert_with
* Remove trailing whitespace
* Rename should to expected in tests
* Rolling back a transaction must mark the overlay as dirty
* Protect client initiated storage tx from being droped by runtime
* Review nits
* Return Err when entering or exiting runtime fails
* Documentation fixup
* Remove close type
* Move enter/exit runtime to excute_aux in the state-machine
* Rename Discard -> Rollback
* Move child changeset creation to constructor
* Move child spawning into the closure
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fixup for code suggestion
* Unify re-exports
* Rename overlay_changes to mod.rs and move into subdir
* Change proof wording
* Adapt a new test from master to storage-tx
* Suggestions from the latest round of review
* Fix warning message
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Hide internal structure of OverlayChanges
* Fix tests for OverlayChanges refactor
* Do not clone pending changes
Discarding prospective changes should be equivalent as a state machine
is not to be called with peding changes.
This will be replaced by a storage transaction that is rolled back before
executing the call the second time removing this constraint.
* Doc fixes
* Remove overlong line
* Revert "Do not clone pending changes"
This reverts commit 4799491f4ac16f8517287a0fcf4a3f84ad56f46e.
* Deduplicate chield tries returned from child_infos()
* Remove redundant type annotation
* Avoid changing the storage root in tests
* Preserve extrinsic indices in trie build test
* Swap order of comitted and prospective in fn child_infos
This is only for consistency and does not impact the result.
* Rename set_pending to replace_pending for clearity
This reduces the usage of `Blake2Hasher` in the code base and replaces
it with `BlakeTwo256`. The most important change is the removal of the
custom extern function for `Blake2Hasher`. The runtime `Hash` trait is
now also simplified and directly requires that the implementing type
implements `Hashable`.
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* In progress, runtime io must switch to future proof root +
child_specific (unique id) + u32 type.
* Switch interface, sr-io seems ok, rpc could use similar interface to
sr-io, genesis json broken if there is child trie in existing encoding
genesis.
* test from previous implementation.
* fix proving test.
* Restore Keyspacedb from other branch, only apply to child trie.
* Removing unneeded child_info from child root (child info are stored
if things changed, otherwhise the root does not change).
* Switch rpc to use same format as ext: more future proof.
* use root from child info for trie backend essence.
* Breaking long lines.
* Update doc and clean pr a bit.
* fix error type
* Restore removed doc on merge and update sr-io doc.
* Switch child storage api to use directly unique id, if managed id
where to be put in place, the api will change at this time.
* Clean deprecated host interface from child.
* Removing assertion on child info (can fail depending on root
memoization).
* merging child info in the overlay when possible.
* child iteration by prefix using child_info.
* Using ChainInfo in frame support. ChainInfo gets redesign to avoid
buffers allocation on every calls.
* Add length of root to the data of child info.
* comments
* Encode compact.
* Remove child info with root.
* Fix try_update condition.
* Comment Ext child root caching.
* Replace tuples by struct with field
* remove StorageTuple alias.
* Fix doc tests, and remove StorageOverlay and ChildStorageOverlay
aliases.
* The storage runtime interface should not enforce a hash type
Currently the runtime interface enforces `H256` as hash type, but in the
future people could use whatever they want as hash type. The hash type
always needs to match between the runtime and the node, but that is
already required.
* Update primitives/externalities/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Gavin Wood