* Improve sandbox internal api
This improves the internal sandbox api for the executor implementations.
The main point is to hide the tls in the internal api and not having it
exposed to the outside.
This is especially needed for wasmtime 0.29.0
* Fmt
* Make it nicer
* Add comments and refactor Sandbox module
* Adds some comments
* Add wasmtime instance to the sandbox and delegate calls
* Adds module imports stub
* WIP state holder via *mut
* My take at the problem
* Brings back invoke and instantiate implementation details
* Removes redundant bound
* Code cleanup
* Fixes invoke closure
* Refactors FunctionExecutor to eliminate lifetime
* Wraps `FunctionExecutor::sandbox_store` in `RefCell`
* Renames `FunctionExecutor::heap` to `allocator`
* Wraps `FunctionExecutor::allocator` in `RefCell`
* Refactors FunctionExecutor to `Rc<Inner>` pattern
* Implements scoped TLS for FunctionExecutor
* Fixes wasmi instancing
* Fixes sandbox asserts
* Makes sandbox compile after wasmtime API change
* Uses Vurich/wasmtime for the Lightbeam backend
* Uses wasmtime instead of wasmi for sandbox API results
* Refactors sandbox to use one of the execution backends at a time
* Fixes wasmtime module instantiation
* TEMP vurich branch stuff
* Adds wasmer impl stub
* Adds get global
* Fixes warnings
* Adds wasmer invoke impl
* Implements host function interface for wasmer
* Fixes wasmer instantiation result
* Adds workaround to remove debug_assert
* Fixes import object generation for wasmer
* Attempt to propagate wasmer::Store through sandbox::Store
* Wraps `sandbox::Store::memories` in `RefCell`
* Moves `sandbox::instantiate` to `sandbox::Store`
* Eliminate `RefCell<memories>`
* Implements `HostState::memory_get/set`, removes accidental `borrow_mut`
* Fixes sandbox memory handling for wasmi
* Fix memory allocation
* Resets Cargo.lock to match master
* Fixes compilation
* Refactors sandbox to use TLS for dispatch_thunk propagation to wasmer
* Pass dispatch thunk to the sandbox as a TLS
* Initialize dispatch thunk holder in `SandboxInstance`
* Comment out Wasmtime/Lightbeam sandbox backend
* Revert wasmtime back to mainstream
* Adds SandboxExecutionMethod enum for cli param
* Cleanup sandbox code
* Allow wasmi to access wasmer memory regions
* More cleanup
* Remove debug logging, replace asserts with runtime errors
* Revert "Adds SandboxExecutionMethod enum for cli param"
This reverts commit dcb2b1d3b54145ab51ad2e3fef0d980ba215b596.
* Fixes warnings
* Fixes indentation and line width
* Fix return types condition
* Puts everything related under the `wasmer-sandbox` feature flag
* Fixes warnings
* Address grumbles
* Split instantiate per backend
* More splits
* Refacmemory allocation
* Nitpicks
* Attempt to wrap wasmer memory in protoco enforcing type
* Revert renaming
* WIP wasm buffer proxy API
* Reimplement util::wasmer::MemoryRef to use buffers instead of memory slices
* Adds WasmiMemoryWrapper and MemoryTransfer trait
* Refactor naming
* Perform all memory transfers using MemoryTransfer
* Adds allocating `read`
* Adds comments
* Removes unused imports
* Removes now unused function
* Pulls Cargo.lock from origin/master
* Fix rustdoc
* Removes unused `TransferError`
* Update Cargo.lock
* Removes unused import
* cargo fmt
* Fix feature dependency graph
* Feature should flow from the top level crate
* We should not assume a specific workspace structure
* sc-executor-wasmi does not use the feature
* sc-executor-wasmtime should not know about the feature
* Fix doc typo
* Enable wasmer-sandbox by default (for now)
It will be removed before merge. It is so that the benchbot
uses the wasmer sandbox.
* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Revert "cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs"
This reverts commit d713590ba45387c4204b2ad97c8bd6f6ebabda4e.
* cargo fmt
* Add ci-check to prevent wasmer sandbox build breaking
* Run tests with wasmer-sandbox enabled
* Revert "Run tests with wasmer-sandbox enabled"
This reverts commit cff63156a162f9ffdab23e7cb94a30f44e320f8a.
Co-authored-by: Sergei Shulepov <s.pepyakin@gmail.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.
Updated code related to deprecated warning on tracing-subscriber `scope()`
( See https://github.com/tokio-rs/tracing/issues/1429 )
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* mark template and utils as non-publish
* switch to development version for testing
* activate unleash check
* maybe if I disable all rules...
* Fix isolated compilation of `max-encoded-len-derive` with `syn`
error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
--> src/lib.rs:88:29
|
88 | .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
| --------- ^^ ----------------------------------- _
| |
| syn::Path
error: aborting due to previous error
For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`
* WIP: bump changes crates since v3 tag to next breaking
cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0
FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`
* Update lockfile
* WIP: Bump sp-transaction-pool as well
* WIP: Bump sp-offchain as well
* WIP: Bump frame-system-rpc-runtime-api as well
* WIP: Bump sp-authority-discovery as well
* Manually deactivate dev-deps before `cargo unleash check`
Otherwise we run into `Cycle detected` error.
* Bump sp-consensus-slots
* Add missing Cargo.lock change
* Bump sp-consensus-vrf as well
* Bump sp-keyring as well
* Bump sp-consensus-pow as well
* Try to speed up the `unleash-check` job
Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help
* fixup: Ensure the temp target dir exists for unleash check
* Bump pallet-transaction-payment-rpc-runtime-api as well
Needed for Polkadot
* Bump pallet-transaction-payment-rpc as well
Needed for Polkadot
* Try updating crates after patching in the Polkadot CI job
* Use another approach to update patched Substrate crates
* Try to update all sp-core versions in Polkadot CI job
* Simplify sp-core version checking
* Apply another shellcheck lint
* Just do the simplest thing I guess
* Welp don't do --offline then
* Clean up `unleash-check` job triggers
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Fix a note in unleash-check cache step
* Add a note about temporary optimization in cargo-unleash
* Pin a newer version of cargo-unleash
Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* NaN canonicalization
* Introduce a simple stack depth metering
* Be explicit about the wasm features we enable
* Pull the latest latast fix for the pwasm-utils crate
* Disable `wasm_threads` as well.
* Factor out deterministic stack params
* Add more docs
* Remove redundant dep
* Refine comments
* Typo
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Andronik Ordian <write@reusable.software>
* Move alloc primitive (not used in /pallets)
* Move to alternative location as not shared
* moved crates to different dir
* ren sp_chain_spec to sc_chain_spec_primatives
* merged sc-chain-spec and moved allocation up one.
* no no_std
* nudge
* Bump CI
* Decommit instance memory after a runtime call on Linux
* Update documentation for the test
* Remove unfinished comment
* Use saturating_sub.
Also update the doc comment.
* Precise RSS tracking in the test
Instead of tracking RSS for the whole process we just look at the particular mapping that is associated with the linear memory of the runtime instance
* Remove unused import
* Fix unused imports
* Fix the unused imports error for good
* Rollback an accidental change to benches
* Fix the test
* Remove now unneeded code
* Update wasmtime to 0.27
A couple of notes:
- Now we are fair about unsafeness of runtime creation via an compiled artifact.
This change was prompted by the change in wasmtime which made
`deserialize` rightfully unsafe. Now `CodeSupplyMode` was hidden and
the `create_runtime` now takes the blob again and there is now a new
fn for creating a runtime with a compiled artifact.
- This is a big change for wasmtime. They switched to the modern backend
for code generation. While this can bring performance improvements, it
can also introduce some problems. In fact, 0.27 fixed a serious issue
that could lead to sandbox escape. Hence we need a proper burn in.
This would require a change to PVF validation host as well.
* Filter regalloc logging
* Establish the runtime_blob module
Seed it with the existing contents of the `util` module.
* Port wasmtime mutable globals instrumentation into runtime blob APIs
* Opt-out from fast instance reuse
* Minor clean up
* Spaces
* Docs clean up
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Factor out the expects
* Fix the suggestion
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Re-generate READMEs to fix doc links; set readme field in package manifests
* Re-generate READMEs to fix doc links; set readme field in package manifests
* Re-generate READMEs to fix doc links; set readme field in package manifests
* Re-generate READMEs to fix doc links; set readme field in package manifests
* Revert stuff that shouldn't have been committed
* Revert stuff that shouldn't have been committed
* Fix parent relative link generation
* Manually remove this incorrect link for now.
* Bump version
* update test-utils crates to be ready for publishing
* adding changelog
* Adding automaticly generated READMEs
* fixing versions
* another version mishap
* refactor globals snapshot
* ignore test
* update pwasm-utils ref
* line width
* add doc comment for internal struct
* add explanation for iteration
* Demote rustdoc to a comment
* use 0.14
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Bastian Köcher