* Adds first version of traits for generating the host functions
* First steps of the procedural macro
* Implements generation of the host extern functions
* Prefix ext host function with snake case trait name
* Implement host functions implementation on the host
* Change `HostFunctions` interface
* Implement `HostFunctions` for tuples
* Make `WasmExecutor` generic over the host functions
* Begin to add a test and make it compile
* Make the test succeed
* Add test to ensure that host functions are not found
* It's alive! Make the `set_storage` test work
* Add test for mutable references
* Code cleanup and documentation etc
* Add marker trait for types that should be passed as SCALE encoded
* Inherit the visibility from the trait and more improvements
* More impls and move them into their own file
* Code simplification by dropping one trait
* Give it a better name
* Implement traits for arrays
* Refactor code to support pass by codec/inner
* Docs
* Implement pass by inner for some crypto types and add a test
* Implement exchangeable function support
* Rewrite sr-io with as runtime interface
* Start reworking after master merge
* Adds `PassByCodec` derive
* Adds `PassByInner` derive
* Fix compilation errors
* More implementations
* Implement runtime interface traits for `str`
* Make `sr-io` compile again
* Fix more compilation errors
* More progress on getting stuff back to compile
* More compilation fixes
* Fix warnings
* Remove le conversions
* Add support for `wasm_only` interfaces
* Implement `Allocator` interface
* Improve error message
* Move `WasmAllocator` to `sr-io` and more clean ups
* Use correct function signature for wasm functions
* Store the host functions with the Wasm runtime
* Docs update
* Fix compilation after master merge
* Remove `sr-io/without_std`
* Make `srml-support` tests run again
* More compilation error fixes
* Use correct doc syntax
* Fix test-runtime
* Fix compilation
* Catch native panics when executing the wasm runtime
As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.
* Fix compilation
* Don't panic in a panic
* Move `sr-sandbox` to new runtime interface
* Fixes tests after sandbox changes
* Make sure we detect invalid utf8
* Fixes after master merge
* Adds pass by enum strategy
* Fix wasmtime integration
* Some macro structure clean up
* Rework and test exchangebale host functions
* PassBy derive macros documentation
* Docs for `runtime_interface` macro
* Support wild card argument names
* Adds ui tests
* Make sure that we are backwards compatible to the old runtime interfaces
* Documentation
* Fixes after latest master merge
* Make `wasmtime` happy
* Make `full_crypto` work
* Make the new interface versionable
* Rename `Sanboxing` to `Sandbox`
* Don't finalize in test while importing
* Fix Performance regression
* Fix test
* executor: Move runtime caching out of WasmRuntime interface.
The runtime version is now fetched and cached at a higher level, not
within the WasmRuntime trait implementations.
* executor: Require successful querying of runtime version.
* executor: Use non wasmi-specific execution in tests.
* executor: Move all runtime execution tests into tests file.
* executor: Use test_case macro to easily execute tests with different
Wasm execution methods.
* executor: Convert errors to strings with Display, not Debug.
* node-executor: Rewrite benchmarks with criterion.
They were not passing compilation before and criterion seems to be more
widely used in Substrate.
* executor: Begin implementation of Wasm runtime.
The implementation demonstrates the outline of the execution, but does
not link against the external host functions.
* executor: Define and implement basic FunctionExecutor.
The SandboxCapabilities::invoke is still left unimplemented.
* executor: Implement host function trampoline generation.
* executor: Instantiate and link runtime module to env module.
* executor: Provide input data during wasmtime execution.
* executor: Implement SandboxCapabilites::invoke for wasmtime executor.
* executor: Integrate and test wasmtime execution method.
* executor: Improve FunctionExecution error messages.
* Scope the unsafe blocks to be smaller.
* Rename TrampolineState to EnvState.
* Let EnvState own its own compiler instead of unsafe lifetime cast.
* Refactor out some common wasmi/wasmtime logic.
* Typos and cosmetic changes.
* More trampoline comments.
* Cargo.lock update.
* cli: CLI option for running Substrate with compiled Wasm execution.
* executor: Switch dependency from fork to official wasmtime repo.
* Quiet down cranelift logs.
* Explicitly catch panics during host calls.
We do this to ensure that panics do not cross language boundaries.
* Additional checks and clarifications in make_trampoline.
* Fixes after merge from master and panic safety for wasmtime
instantiation.
* fix child_storage_hash
* extract common implementation for ext and testing
* cleaning impl.
* replace ExtBasisMut by actual Ext
* remove extbasis.
* Update tests to use Ext from test externalities.
* use Ext constructor for getting ext from TestExternalities.
* Add missing extensions from ext.
* fix wasmi test
* Fix merge error.
* Make `wasmi_execution` public to use it from tests
* Make `WasmRuntime` accessible as well
* Add `call_in_wasm` instead of making stuff public
* Use `WasmRuntime`
* Move test
* More feedback
* Move `Externalities` into `substrate-externalities`
- `Externalities` now support generic extensions
- Split of `primtives-storage` for storage primitive types
* Move the externalities scoping into `substrate-externalities`
* Fix compilation
* Review feedback
* Adds macro for declaring extensions
* Fix benchmarks
* Introduce `ExtensionStore` trait
* Last review comments
* Implement it for `ExtensionStore`
* executor: Move definitions of externals out of wasm_executor module.
* executor: Create WasmRuntime trait.
This will be used to decouple the runtime cache from wasmi execution.
* executor: Remove WasmExecutor and move methods to wasmi_execution.
These will now be crate-internal functions and there is no need
for the struct.
* executor: Set default default_heap_pages in NativeExecutor.
* cli: CLI configuration for Wasm execution method.
* executor: Remove wasmi-specific code from wasm_runtime.
* Respond to review comments.
* Adds new wrapper traits for wasm executor
* Add new crate `substrate-wasm-interface`
Thew new crate holds types and traits for the communicating between the
wasm runtime and the host.
* Rewrite externals with new macro etc
* Fix vec initialization
* Make executor tests working
* Remove unused code + warnings
* Introduce `Pointer` and `WordSize` for working with wasm
* Fix tests and compilation
* Fix compilation
* Apply suggestions from code review
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Review feedback
* Remove unused conversions
* Make each host function its own struct
`HostFunctions` now just returns these function structs. Each function
can be executed by using one of the function structs. The inherent host
functions are now moved to the "normal" host functions.
* Remove byteorder
* Add floating point types
* Make pointer interface more safe
* Add type alias for wasm-interface Result
* More review comments
* Add test from original bug report
Original is from @pepyakin in 3d7b27f3421818e8d6de568e02fbc2947a06246b.
I adapted it to work with the latest master.
* No longer cleanup module instance
* Replace runtime cache with synchronous clone
* Fix test
* Preserve initial runtime memory and restore it on fetch
* Remove leftover comment
* Fix style
* Improve variable naming
* Replace get_into() with get()
* Handle missing memory export better
* Return earlier when creating runtime first time
* Improve comments
* fmt
* Fix#2967.
* Eradicate `code` from `Error::InvalidCode`
* tidy
* A state snapshot doc.
* Store multiple runtimes by hash.
* Get rid of deref.
* Docs
* Use Self for instantiate_module
* REVERT ME
* Should be ok
* Commit
* Remove dbg
* Use fast-memory's erase
* Clean and undo hacks.
* Introduce a dedicated error for heap_base
* Ban the start function.
* Clean, docs and refactor
* Add rustflags.
* Update Cargo.lock
* Apply Basti's suggestions
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Rename allocates_huge_stack_array
* Extend TestClientBuilder with set_heap_pages
* Update the test.
* Update core/executor/src/wasm_executor.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update core/executor/src/wasm_runtimes_cache.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update core/executor/src/error.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update core/executor/src/error.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix tests.
* Update cargo-lock
* Use wasmi master
* Use master wasmi
* Move tests.
* Use wasmi crates.io
* Update Cargo.lock
* Fix build.rs
* Bump runtime version
* Revert initial_heap_pages renaming
* Bump wasmi up to 0.5.0
* Bump runtime version
* Don't restore an instance every now and then
* Update core/executor/src/wasm_runtimes_cache.rs
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
* Propagate error in CacheError
* Clarify the get_heap_base call in instantiation
* Supply --export=__heap_base
See https://reviews.llvm.org/D62744
Co-authored-by: Jim Posen <jim.posen@gmail.com>
* Bump version.
* Use combinators for segments.
* Fix build.rs
* Fix build.rs for runtime-test
* Link substrate issue tracker in panic
* Replace allocator with freeing-bump allocator
* Revert me: Panic on double allocate/free
* Revert me: Add shallow benchmark for a first impression
* Revert "Revert me: Add shallow benchmark for a first impression"
This reverts commit 5f0d4df39395eb55c9ec2d6fe1ed992533475fec.
* Revert "Revert me: Panic on double allocate/free"
This reverts commit a114df7d6cfc21d60af396ccca02e5c8205db6ce.
* Rename heap to FreeingBumpHeapAllocator
* Rename heap.rs to allocator.rs
* Use sandbox heap
* Move functions
* Move variables into constructor
* Revert "Move variables into constructor"
This reverts commit f46fa0d0cdf4ea97760ccce58003b0d33f433743.
* Remove unnecessary casts
* Add comment for new parameter
* Improve typing
* Move variables into constructor
* Avoid dynamic allocation
* Remove unused variables
* Revert "Link substrate issue tracker in panic"
This reverts commit 32dfa1d02bcf881d1d514a930fcc0fdf3c5f8e08.
In the meantime this was fixed in https://github.com/paritytech/substrate/pull/1667.
* Improve naming
* Only assert in debug mode
* Remove dynamic allocation
* Decrease bucket size
A bucket size of 8192 bytes is quite large and it turned
out that this can exhaust the available heap space too
too quickly.
This is because even for allocating 1 byte a bucket of
8192 bytes is allocated/wasted.
* Return 0 if requested size too large
* Improve test
The test didn't use an offset when setting up the heap.
Hence the first successfully allocated pointer was
always `0`.
This is unfortunate since `0` is also the return value
when there is an error.
This lead to us not noticing that the test was failing,
because it did not distinguish between success and error.
* Revert to linear allocator
* Better allocator for wasm (#1460)
* Add Arch Linux installation instructions
* Enable tracing heap size
* Extract heap
* Replace linear allocator with buddy allocator
* Fix test
The purose of this big block is for the tests
to test a failure when the block is too big
though. The improved buddy allocation strategy
results in this block fitting on the heap now.
Hence the increase.
* Get rid of memcpy in to_vec()
* fixup: Style and comments
* fixup: Split Linux instructions by distribution
To prevent misunderstandings of people executing both.
* fixup: Remove unnecessary types and code
* fixup: Make Pointers start from 1, remove some panics, code improvements
* fixup: Return 0 on errors
* fixup: Move loop to separate function
* fixup: Use FnvHashMap instead of HashMap
* fixup: Fix error handling
* fixup: Use current_size() instead of used_size()
* fixup: Fix and document allocation offset
* fixup: Remove unnecessary multiplication
* fixup: Fix comments
* fixup: Remove Arch installation instructions
* Revert "Fix test"
This reverts commit 292c177df8efaa4658293748a13b1ab1c0b76ea8.
* fixup: Remove unused code, improve import
* fixup: Proper alignment
* fixup: Do not use internal constant in public description
* fixup: Add comment regarding invariants
* fixup: Move assertion to compile-time check
* Fix bug in necessary tree level calculation
The tree levels necessary to house a number
of nodes was calculated incorrectly.
* Improve naming
* Add Arch Linux installation instructions
* Enable tracing heap size
* Extract heap
* Replace linear allocator with buddy allocator
* Fix test
The purose of this big block is for the tests
to test a failure when the block is too big
though. The improved buddy allocation strategy
results in this block fitting on the heap now.
Hence the increase.
* Get rid of memcpy in to_vec()
* fixup: Style and comments
* fixup: Split Linux instructions by distribution
To prevent misunderstandings of people executing both.
* fixup: Remove unnecessary types and code
* fixup: Make Pointers start from 1, remove some panics, code improvements
* fixup: Return 0 on errors
* fixup: Move loop to separate function
* fixup: Use FnvHashMap instead of HashMap
* fixup: Fix error handling
* fixup: Use current_size() instead of used_size()
* fixup: Fix and document allocation offset
* fixup: Remove unnecessary multiplication
* fixup: Fix comments
* fixup: Remove Arch installation instructions
* Revert "Fix test"
This reverts commit 292c177df8efaa4658293748a13b1ab1c0b76ea8.
* fixup: Remove unused code, improve import
* fixup: Proper alignment
* fixup: Do not use internal constant in public description
* fixup: Add comment regarding invariants
* fixup: Move assertion to compile-time check
* docs: Remove safe-mix since is no longer local dependency
* docs: Fix existing packages link. Still only includes Substrate Core
* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme
* docs: Remove link to old docs
* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names
* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory
* Change example to use package name that opens all packages in RustDocs
* Fix list of packages names that are available to open in RustDocs
* fix typo
* fix ordering of CLI options and add missing backslash
* Remove unneeded script
* Rename Substrate Demo -> Substrate
* Rename demo -> node
* Build wasm from last rename.
* Merge ed25519 into substrate-primitives
* Minor tweak
* Rename substrate -> core
* Move substrate-runtime-support to core/runtime/support
* Rename/move substrate-runtime-version
* Move codec up a level
* Rename substrate-codec -> parity-codec
* Move environmental up a level
* Move pwasm-* up to top, ready for removal
* Remove requirement of s-r-support from s-r-primitives
* Move core/runtime/primitives into core/runtime-primitives
* Remove s-r-support dep from s-r-version
* Remove dep of s-r-support from bft
* Remove dep of s-r-support from node/consensus
* Sever all other core deps from s-r-support
* Forgot the no_std directive
* Rename non-SRML modules to sr-* to avoid match clashes
* Move runtime/* to srml/*
* Rename substrate-runtime-* -> srml-*
* Move srml to top-level
Bastian Köcher