* FreeingBumpAllocator: Initialize the heads to `u32::max_value()`
`self.heads` can point to an element with the index `0` in the heap.
This would make the allocator fail to reuse this element.
* Simplify the `PREFIX_SIZE` handling
* Fix memory leak in runtime interface
We used `slice::from_raw_parts` in runtime-interface which did not free
the memory afterwards. This pr changes it to `Vec::from_raw_parts` to
make sure `drop` is called properly and the values are freed.
* Check that `len` is non-zero
* Adds comment
When messing with the signature, we need to make sure that we acutally
mess-up the signature. As the generated private/public key is random,
the signature is random as well. It can happen that `bytes[0] ==
bytes[2]` which makes the test fail. We fix this problem by just
inverting the bytes at `0` and `2`.
* Expose information about the extrinsic in the metadata
This pr exposes some information about the extrinsic used in the runtime
via metadata. The following information are exposed:
- Version of the extrinsic
- List of all signed extensions used by the extrinsic.
* Increment `spec_version`
* Print extrinsic failed error always, not just on import
Before we printed the error of a transaction only when importing a
block, this pr changes it to print the error also at building the block.
* Increment `impl_version`
* WASM runtime switch to import memory
Up to now runtimes have exported their memory. To unify it with
sandboxing, this pr switches runtimes to import memory as well.
From a functional perspective, exporting/importing memory makes no
difference to the runtime.
To provide backwards compatibility, WASM exported memory is still supported.
* Revert debug stuff
* Revert some stuff
* Introduces `author_hasKey` and `author_hasSessionKeys` rpc endpoints
Both endpoints can be used to check if a key is present in the keystore.
- `hasKey` works on with an individual public key and key type. It
checks if a private key for the given combination exists in the
keystore.
- `hasSessionKeys` works with the full encoded session key blob stored
on-chain in `nextKeys`. This requires that the given blob can be decoded
by the runtime. It will return `true`, iff all public keys of the
session key exist in the storage.
Fixes: https://github.com/paritytech/substrate/issues/4696
* Update client/rpc-api/src/author/error.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Indentation
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
This moves the wasm-allocator (`FreeingBumpHeapAllocator`) into its own
crate `sp-allocator`. This new crate can theoretically provide multiple
different allocators. Besides moving the allocator, this pr also makes
`FreeingBumpHeapAllocator` compile on `no_std`.
* Support `u128`/`i128` in runtime interface
This implements support for `u128`/`i128` as parameters/return value in
runtime interfaces. As we can not pass them as identity, as for the
other primitives types, we pass them as an pointer to an `[u8; 16]` array.
* Remove some unsafe code usage
* Make debug builds more usable
This pr makes debug builds more usable in terms of `cargo run -- --dev`.
1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.
* Make sure we only overwrite default values
* Make it work
* Apply suggestion
Some fixes after: https://github.com/paritytech/substrate/pull/4502
This removes the unwanted `expect`s from `MultiSigner`. Instead we
convert from full to compressed in `TryFrom` and can return an error on
invalid input.
By using `cfg(doc)`, we can generate docs for modules which are only
available from `no_std`. This drastically improves the documentation for
the developers.
* Implement runtime version checks in `set_code`
Check that the new runtime code given to `set_code` fullfills some
requirements:
- `spec_name` matches
- `spec_version` does not decreases
- `impl_version` does not decreases
- Either `spec_version` and `impl_version` increase
* Make tests almost work
* Some fixes after master merge
* Fix tests
* Add missed file
* Make depedency check happy?
* Remove leftover `sc-executor`
* AHHHHH
* Reset debug stuff
* Remove some 'static
* More 'static
* Some docs
* Update `Cargo.lock`
Instead of having these weird implementation of `BuildStorage for
&ChainSpec` we should just take the `genesis_storage` by ref. The
`BuildStorage` trait changed some time ago to take a self ref anyway,
instead of a self value.
Also fixes warnings in frame-staking
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* Make wasm-builder remove invalid members
Wasm-builder now removes members that point to packages that do not
exist anymore or that were renamed. Up to now, the build would fail and
required manual fixing.
* Fixes build
* Fix cli for structopt 0.3.7 and pin to that version
This is just some hotfix to make everything compile. In the future it
will require another pr to not depend on internals of StructOpt, but
that will probably also require some additions to StructOpt itself. To
not break the code again with another StructOpt, this also pins the
StructOpt version.
* Fix benches
* Fix for fix
* Make `decl_error!` errors usable
This pr implements support for returning errors of different pallets in
a pallet. These errors need to be declared with `decl_error!`.
The pr changes the following:
- Each dispatchable function now returns a `DispatchResult` which is an
alias for `Result<(), DispatchError>`.
- `DispatchError` is an enum that has 4 variants:
- `Other`: For storing string error messages
- `CannotLookup`: Variant that is returned when something returns a
`sp_runtime::LookupError`
- `BadOrigin`: Variant that is returned for any kind of bad origin
- `Module`: The error of a specific module. Contains the `index`,
`error` and the `message`. The index is the index of the module in
`construct_runtime!`. `error` is the index of the error in the error
enum declared by `decl_error!`. `message` is the message to the error
variant (this will not be encoded).
- `construct_runtime!` now creates a new struct `ModuleToIndex`. This
struct implements the trait `ModuleToIndex`.
- `frame_system::Trait` has a new associated type: `ModuleToIndex` that
expects the `ModuleToIndex` generated by `construct_runtime!`.
- All error strings returned in any module are being converted now to `DispatchError`.
- `BadOrigin` is the default error returned by any type that implements `EnsureOrigin`.
* Fix frame system benchmarks
* Check in block authoring that we can author with current authoring version
* Update client/consensus/pow/src/lib.rs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* Fix compilation
* The storage runtime interface should not enforce a hash type
Currently the runtime interface enforces `H256` as hash type, but in the
future people could use whatever they want as hash type. The hash type
always needs to match between the runtime and the node, but that is
already required.
* Update primitives/externalities/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Switch to new rocksdb with some hacks to resolve linking errors
Rocksdb and `wasmtime-environ` both link `zstd`. In the final link step,
this leads to linking errors because of duplicate symbols. The linked
Rocksdb fixes this by using `zstd-sys` as well. However, this currently
also requires modifications in `zstd-sys`. Someone will need to come up
with a better implementation.
* Switch to new version of kvdb-rocksdb
* Update client/db/src/utils.rs
Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>
When we enabled arbitary precision for `serde_json`, the feature leaked
through the whole workspace. The given test fails because of some bug in
serde json when the feature is enabled.
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Adds first version of traits for generating the host functions
* First steps of the procedural macro
* Implements generation of the host extern functions
* Prefix ext host function with snake case trait name
* Implement host functions implementation on the host
* Change `HostFunctions` interface
* Implement `HostFunctions` for tuples
* Make `WasmExecutor` generic over the host functions
* Begin to add a test and make it compile
* Make the test succeed
* Add test to ensure that host functions are not found
* It's alive! Make the `set_storage` test work
* Add test for mutable references
* Code cleanup and documentation etc
* Add marker trait for types that should be passed as SCALE encoded
* Inherit the visibility from the trait and more improvements
* More impls and move them into their own file
* Code simplification by dropping one trait
* Give it a better name
* Implement traits for arrays
* Refactor code to support pass by codec/inner
* Docs
* Implement pass by inner for some crypto types and add a test
* Implement exchangeable function support
* Rewrite sr-io with as runtime interface
* Start reworking after master merge
* Adds `PassByCodec` derive
* Adds `PassByInner` derive
* Fix compilation errors
* More implementations
* Implement runtime interface traits for `str`
* Make `sr-io` compile again
* Fix more compilation errors
* More progress on getting stuff back to compile
* More compilation fixes
* Fix warnings
* Remove le conversions
* Add support for `wasm_only` interfaces
* Implement `Allocator` interface
* Improve error message
* Move `WasmAllocator` to `sr-io` and more clean ups
* Use correct function signature for wasm functions
* Store the host functions with the Wasm runtime
* Docs update
* Fix compilation after master merge
* Remove `sr-io/without_std`
* Make `srml-support` tests run again
* More compilation error fixes
* Use correct doc syntax
* Fix test-runtime
* Fix compilation
* Catch native panics when executing the wasm runtime
As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.
* Fix compilation
* Don't panic in a panic
* Move `sr-sandbox` to new runtime interface
* Fixes tests after sandbox changes
* Make sure we detect invalid utf8
* Fixes after master merge
* Adds pass by enum strategy
* Fix wasmtime integration
* Some macro structure clean up
* Rework and test exchangebale host functions
* PassBy derive macros documentation
* Docs for `runtime_interface` macro
* Support wild card argument names
* Adds ui tests
* Make sure that we are backwards compatible to the old runtime interfaces
* Documentation
* Fixes after latest master merge
* Make `wasmtime` happy
* Make `full_crypto` work
* Make the new interface versionable
* Rename `Sanboxing` to `Sandbox`
* Don't finalize in test while importing
* Fix Performance regression
* Fix test
- Add new crate `substrate-build-script-utils` to unify the code of
`node`, `node-template` and `polkadot-node`.
- The `node-cli` build script needs to search upwards for the
`.git/HEAD` file to find it.
As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.
* Remove footgun around session keys/handlers
- `OpaqueKeys` now has an associated type `KeyTypeIdProviders`. This can
be used in the runtime as input for `SessionHandler` from the session
trait.
- `impl_opaque_keys` now works with modules and extracts the `KeyTypeId`
from the module directly.
- Added some checks to the `session` storage initialization that checks
that the `SessionHandler` and `Keys` use the same number of keys and
that the order is equal.
* Update core/sr-primitives/src/traits.rs
* Adds function `to_substrate_wasm_fn_return_value`
Instead of replicating this piece of code over and over again, just move
it to a function that does it.
* Feedback
* Comment
* Wasm-builder-runner unset `CARGO_TARGET_DIR` and release 1.0.4
`CARGO_TARGET_DIR` needs to be unset or otherwise cargo deadlocks,
because cargo always holds an exclusive lock on target dir.
* Commit missing version up
* Lock file
* Make `wasmi_execution` public to use it from tests
* Make `WasmRuntime` accessible as well
* Add `call_in_wasm` instead of making stuff public
* Use `WasmRuntime`
* Move test
* More feedback
The macro generates the functions with the signature we expect for wasm
functions. This macro is useful for tests where we need to call into
wasm. Parameter passing is done by SCALE encoding the input and output
parameters.
This function executes the given closure in a context where the test
externalities are set. This makes the srml tests easier to write, as the
test externalities need to be created anyway.
* Move `Externalities` into `substrate-externalities`
- `Externalities` now support generic extensions
- Split of `primtives-storage` for storage primitive types
* Move the externalities scoping into `substrate-externalities`
* Fix compilation
* Review feedback
* Adds macro for declaring extensions
* Fix benchmarks
* Introduce `ExtensionStore` trait
* Last review comments
* Implement it for `ExtensionStore`
So, instead of:
`type ALL_MODULES = (Module1, Module2, Module3)`;
Generate it like this:
`type ALL_MODULES = ((Module1, Module2), Module3,);`
This should make `construct_runtime!` support an "unlimited" number of
modules.
* Give more information why conversion between native and runtime failed
This adds the SCALE error description to the error when the conversion
between native and runtime failed.
* Fixes tests
`debug-assertions` can also be enabled in for release builds. This
introduces a new build-script that extracts the build type from the
`PROFILE` env variable and sets the `build_type` cfg.
By accident it was already enabled in master for quite some time. To
make sure that we don't blow up the wasm binary size, I compiled the
binary with the feature enabled and disabled. With nice panic messages
enabled, the binary size increases by 908 bytes. Given the value that
this feature brings, I think it is okay to have these panic messages
enabled by default.
* Implement support for `patch` section in wasm-builder
* Update core/utils/wasm-builder/src/wasm_project.rs
* Support patch by path
* Go down to the actual data
* Switch to new proc-macro crate for implementing traits for tuples
* Switch back to stable `Cargo.lock` format
* Reduce tuple number to match rusts default implementation
Bastian Köcher