* Switch to pooling copy-on-write instantiation strategy for WASM
* Fix benchmark compilation
* Fix `cargo fmt`
* Fix compilation of another benchmark I've missed
* Cleanups according to review comments
* Move `max_memory_size` to `Semantics`
* Set `memory_guaranteed_dense_image_size` to `max_memory_size`
* Rename `wasm_instantiation_strategy` to `wasmtime_instantiation_strategy`
* Update the doc-comments regarding the instantiation strategy
* Extend the integration tests to test every instantiation strategy
* Don't drop the temporary directory until the runtime is dropped in benchmarks
* Don't drop the temporary directory until the runtime is dropped in tests
* Add extra WASM heap pages when precompiling the runtime blob
* Fix compilation
* Fix rustdoc
* Fix rustdoc for real this time
* Fix benches compilation
* Improve the builder in `sc-executor-wasmtime`'s tests
* Add a new host function for reporting fatal errors
* Fix one of the wasmtime executor tests
* Have `#[runtime_interface(wasm_only)]` actually mean WASM-only, and not no_std-only
* Print out errors through `Display` instead of `Debug`
* Switch one more trait to require `Error` for its error instead of only `Debug`
* Align to review comments
* starting
* Updated from other branch.
* setting flag
* flag in storage struct
* fix flagging to access and insert.
* added todo to fix
* also missing serialize meta to storage proof
* extract meta.
* Isolate old trie layout.
* failing test that requires storing in meta when old hash scheme is used.
* old hash compatibility
* Db migrate.
* runing tests with both states when interesting.
* fix chain spec test with serde default.
* export state (missing trie function).
* Pending using new branch, lacking genericity on layout resolution.
* extract and set global meta
* Update to branch 4
* fix iterator with root flag (no longer insert node).
* fix trie root hashing of root
* complete basic backend.
* Remove old_hash meta from proof that do not use inner_hashing.
* fix trie test for empty (force layout on empty deltas).
* Root update fix.
* debug on meta
* Use trie key iteration that do not include value in proofs.
* switch default test ext to use inner hash.
* small integration test, and fix tx cache mgmt in ext.
test failing
* Proof scenario at state-machine level.
* trace for db upgrade
* try different param
* act more like iter_from.
* Bigger batches.
* Update trie dependency.
* drafting codec changes and refact
* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.
* alt hashing only for branch with value.
* fix trie tests
* Hash of value include the encoded size.
* removing fields(broken)
* fix trie_stream to also include value length in inner hash.
* triedbmut only using alt type if inner hashing.
* trie_stream to also only use alt hashing type when actually alt hashing.
* Refactor meta state, logic should work with change of trie treshold.
* Remove NoMeta variant.
* Remove state_hashed trigger specific functions.
* pending switching to using threshold, new storage root api does not
make much sense.
* refactoring to use state from backend (not possible payload changes).
* Applying from previous state
* Remove default from storage, genesis need a special build.
* rem empty space
* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.
* fix compilation
* Right logic to avoid switch on reencode when default layout.
* Clean up some todos
* remove trie meta from root upstream
* update upstream and fix benches.
* split some long lines.
* UPdate trie crate to work with new design.
* Finish update to refactored upstream.
* update to latest triedb changes.
* Clean up.
* fix executor test.
* rust fmt from master.
* rust format.
* rustfmt
* fix
* start host function driven versioning
* update state-machine part
* still need access to state version from runtime
* state hash in mem: wrong
* direction likely correct, but passing call to code exec for genesis
init seem awkward.
* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.
* stateversion from runtime version (core api >= 4).
* update trie, fix tests
* unused import
* clean some TODOs
* Require RuntimeVersionOf for executor
* use RuntimeVersionOf to resolve genesis state version.
* update runtime version test
* fix state-machine tests
* TODO
* Use runtime version from storage wasm with fast sync.
* rustfmt
* fmt
* fix test
* revert useless changes.
* clean some unused changes
* fmt
* removing useless trait function.
* remove remaining reference to state_hash
* fix some imports
* Follow chain state version management.
* trie update, fix and constant threshold for trie layouts.
* update deps
* Update to latest trie pr changes.
* fix benches
* Verify proof requires right layout.
* update trie_root
* Update trie deps to latest
* Update to latest trie versioning
* Removing patch
* update lock
* extrinsic for sc-service-test using layout v0.
* Adding RuntimeVersionOf to CallExecutor works.
* fmt
* error when resolving version and no wasm in storage.
* use existing utils to instantiate runtime code.
* Patch to delay runtime switch.
* Revert "Patch to delay runtime switch."
This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.
* useless closure
* remove remaining state_hash variables.
* Remove outdated comment
* useless inner hash
* fmt
* fmt and opt-in feature to apply state change.
* feature gate core version, use new test feature for node and test node
* Use a 'State' api version instead of Core one.
* fix merge of test function
* use blake macro.
* Fix state api (require declaring the api in runtime).
* Opt out feature, fix macro for io to select a given version
instead of latest.
* run test nodes on new state.
* fix
* Apply review change (docs and error).
* fmt
* use explicit runtime_interface in doc test
* fix ui test
* fix doc test
* fmt
* use default for path and specname when resolving version.
* small review related changes.
* doc value size requirement.
* rename old_state feature
* Remove macro changes
* feature rename
* state version as host function parameter
* remove flag for client api
* fix tests
* switch storage chain proof to V1
* host functions, pass by state version enum
* use WrappedRuntimeCode
* start
* state_version in runtime version
* rust fmt
* Update storage proof of max size.
* fix runtime version rpc test
* right intent of convert from compat
* fix doc test
* fix doc test
* split proof
* decode without replay, and remove some reexports.
* Decode with compatibility by default.
* switch state_version to u8. And remove RuntimeVersionBasis.
* test
* use api when reading embedded version
* fix decode with apis
* extract core version instead
* test fix
* unused import
* review changes.
Co-authored-by: kianenigma <kian@parity.io>
* Statically register host WASM functions
* Fix `substrate-test-client` compilation
* Move `ExtendedHostFunctions` to `sp-wasm-interface`
* Fix `sp-runtime-interface` tests' compilation
* Fix `sc-executor-wasmtime` tests' compilation
* Use `runtime_interface` macro in `test-runner`
* Fix `sc-executor` tests' compilation
* Reformatting/`rustfmt`
* Add an extra comment regarding the `H` generic arg in `create_runtime`
* Even more `rustfmt`
* Depend on `wasmtime` without default features in `sp-wasm-interface`
* Bump version of `sp-wasm-interface` to 4.0.1
* Bump `sp-wasm-interface` in `Cargo.lock` too
* Bump all of the `sp-wasm-interface` requirements to 4.0.1
Maybe this will appease cargo-unleash?
* Revert "Bump all of the `sp-wasm-interface` requirements to 4.0.1"
This reverts commit 0f7ccf8e0f371542861121b145ab87af6541ac30.
* Make `cargo-unleash` happy (maybe)
* Use `cargo-unleash` to bump the crates' versions
* Align to review comments
* sc-executor-wasmtime: upgrade wasmtime to 0.30.0
Changes related to memory64 proposal implementation,
for additional details see bytecodealliance/wasmtime#3153
* sc-executor-wasmtime: introduce parallel_compilation flag
* typos
* Focus on correctness lints. This relies on a compiler patch that landed 8th July 2021.
If people are using an earlier version of the compiler everything will still work unless they try and run clippy.
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Start
* Move to ctx
* Make it compile for now
* More work
* Get rid off state-holder
* Use less Refcells
* 🤦
* Don't use RefCell
* Use names for parameters
* Fixes after merge
* Fixes after merge
* Review feedback
* FMT
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* NaN canonicalization
* Introduce a simple stack depth metering
* Be explicit about the wasm features we enable
* Pull the latest latast fix for the pwasm-utils crate
* Disable `wasm_threads` as well.
* Factor out deterministic stack params
* Add more docs
* Remove redundant dep
* Refine comments
* Typo
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Andronik Ordian <write@reusable.software>
* Add function for embedding the runtime version in a wasm blob
This function can be used to add the custom section to a wasm blob with
the runtime version in it.
* Review nitpick
* Update wasmtime to 0.27
A couple of notes:
- Now we are fair about unsafeness of runtime creation via an compiled artifact.
This change was prompted by the change in wasmtime which made
`deserialize` rightfully unsafe. Now `CodeSupplyMode` was hidden and
the `create_runtime` now takes the blob again and there is now a new
fn for creating a runtime with a compiled artifact.
- This is a big change for wasmtime. They switched to the modern backend
for code generation. While this can bring performance improvements, it
can also introduce some problems. In fact, 0.27 fixed a serious issue
that could lead to sandbox escape. Hence we need a proper burn in.
This would require a change to PVF validation host as well.
* Filter regalloc logging
* emit a custom section from impl_runtime_apis!
This change emits a custom section from the impl_runtime_apis! proc macro.
Each implemented API will result to emitting a link section `runtime_apis`.
During linking all sections with this name will be concatenated and
placed into the final wasm binary under the same name.
* Introduce `runtime_version` proc macro
This macro takes an existing `RuntimeVersion` const declaration, parses
it and emits the version information in form of a linking section.
Ultimately such a linking section will result into a custom wasm
section.
* Parse custom wasm section for runtime version
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Fix sc-executor integration tests
* Nits
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Refactor apis section deserialization
* Fix version decoding
* Reuse uncompressed value for CallInWasm
* Log on decompression error
* Simplify if
* Reexport proc-macro from sp_version
* Merge ReadRuntimeVersionExt
* Export `read_embedded_version`
* Fix test
* Simplify searching for custom section
Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Establish the runtime_blob module
Seed it with the existing contents of the `util` module.
* Port wasmtime mutable globals instrumentation into runtime blob APIs
* Opt-out from fast instance reuse
* Minor clean up
* Spaces
* Docs clean up
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Factor out the expects
* Fix the suggestion
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Use CLI to configure max instances cache
* Fix tests
* Move default value into CLI
* Use SmallVec
* Apply review comments
* Get rid of `SmallVec`
Co-authored-by: Bastian Köcher <git@kchr.de>
* Don't include `:code` by default in storage proofs (#5060)
* Adds test to verify that the runtime currently is always contained in
the proof
* Start passing the runtime wasm code from the outside
* Fix compilation
* More build fixes
* Make the test work as expected now :)
* Last fixes
* Fixes benchmarks
* Review feedback
* Apply suggestions from code review
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Review feedback
* Fix compilation
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
* Fix compilation and change the way `RuntimeCode` works
* Fix tests
* Switch to `Cow`
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
* Adds test to verify that the runtime currently is always contained in
the proof
* Start passing the runtime wasm code from the outside
* Fix compilation
* More build fixes
* Make the test work as expected now :)
* Last fixes
* Fixes benchmarks
* Review feedback
* Apply suggestions from code review
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Review feedback
* Fix compilation
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
* Migrate wasmtime backend to wasmtime-api
* Port to a newer version of wasmtime
* Update to the latest changes.
* Rejig the sandbox module a bit
* Materialze
* Fixes.
* executor wasm_runtime fix
* Refactor everything
* More refactoring
* Even more refactorings
* More cleaning.
* Update to the latest wasmtime
* Reformat
* Renames
* Refactoring and comments.
* Docs
* Rename FunctionExecutor to host.
* Imrpove docs.
* fmt
* Remove panic
* Assert the number of arguments are equal between wasmtime and hostfunc.
* Comment a possible panic if there is no corresponding value variant.
* Check signature of the entrypoint.
* Use git version of wasmtime
* Refine and doc the sandbox code.
* Comment RefCells.
* Update wasmtime to the latest-ish master.
This may solve a problem with segfaults.
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Use full SHA1 hash of wasmtime commit.
* Add a panic message.
* Add some documentation
* Update wasmtime version to include SIGSEGV fix
* Update to crates.io version of wasmtime
* Make it work.
* Move the creation of memory into `InstanceWrapper::new`
* Make `InstanceWrapper` !Send & !Sync
* Avoid using `take_mut`
* Update client/executor/wasmtime/Cargo.toml
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Limit maximum size of memory.
* Rename `init_state` to `with_initialized_state`
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Implement runtime version checks in `set_code`
Check that the new runtime code given to `set_code` fullfills some
requirements:
- `spec_name` matches
- `spec_version` does not decreases
- `impl_version` does not decreases
- Either `spec_version` and `impl_version` increase
* Make tests almost work
* Some fixes after master merge
* Fix tests
* Add missed file
* Make depedency check happy?
* Remove leftover `sc-executor`
* AHHHHH
* Reset debug stuff
* Remove some 'static
* More 'static
* Some docs
* Update `Cargo.lock`
* Clean imports in wasmi_execution
* Replace `interpret_runtime_api_result` with `pointer_and_len_from_u64`.
* Extract sc-executor-common crate
* Extract `sc-executor-wasmi` into its own crate
* Extract `sc-executor-wasmtime` into its own crate.
* Add missing headers.
* Clean and docs
* Docs for sc-executor-wasmi
* Expand a comment about sandboxing
* Fix assert_matches
* Rename (un)pack_ptr_and_len and move them into util module
* Remove wasmtime errors in sc-executor-common
* The storage runtime interface should not enforce a hash type
Currently the runtime interface enforces `H256` as hash type, but in the
future people could use whatever they want as hash type. The hash type
always needs to match between the runtime and the node, but that is
already required.
* Update primitives/externalities/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Koute