* Expose information about the extrinsic in the metadata
This pr exposes some information about the extrinsic used in the runtime
via metadata. The following information are exposed:
- Version of the extrinsic
- List of all signed extensions used by the extrinsic.
* Increment `spec_version`
* Print extrinsic failed error always, not just on import
Before we printed the error of a transaction only when importing a
block, this pr changes it to print the error also at building the block.
* Increment `impl_version`
* WASM runtime switch to import memory
Up to now runtimes have exported their memory. To unify it with
sandboxing, this pr switches runtimes to import memory as well.
From a functional perspective, exporting/importing memory makes no
difference to the runtime.
To provide backwards compatibility, WASM exported memory is still supported.
* Revert debug stuff
* Revert some stuff
* Introduces `author_hasKey` and `author_hasSessionKeys` rpc endpoints
Both endpoints can be used to check if a key is present in the keystore.
- `hasKey` works on with an individual public key and key type. It
checks if a private key for the given combination exists in the
keystore.
- `hasSessionKeys` works with the full encoded session key blob stored
on-chain in `nextKeys`. This requires that the given blob can be decoded
by the runtime. It will return `true`, iff all public keys of the
session key exist in the storage.
Fixes: https://github.com/paritytech/substrate/issues/4696
* Update client/rpc-api/src/author/error.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Indentation
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
This moves the wasm-allocator (`FreeingBumpHeapAllocator`) into its own
crate `sp-allocator`. This new crate can theoretically provide multiple
different allocators. Besides moving the allocator, this pr also makes
`FreeingBumpHeapAllocator` compile on `no_std`.
* Support `u128`/`i128` in runtime interface
This implements support for `u128`/`i128` as parameters/return value in
runtime interfaces. As we can not pass them as identity, as for the
other primitives types, we pass them as an pointer to an `[u8; 16]` array.
* Remove some unsafe code usage
* Make debug builds more usable
This pr makes debug builds more usable in terms of `cargo run -- --dev`.
1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.
* Make sure we only overwrite default values
* Make it work
* Apply suggestion
Some fixes after: https://github.com/paritytech/substrate/pull/4502
This removes the unwanted `expect`s from `MultiSigner`. Instead we
convert from full to compressed in `TryFrom` and can return an error on
invalid input.
By using `cfg(doc)`, we can generate docs for modules which are only
available from `no_std`. This drastically improves the documentation for
the developers.
* Implement runtime version checks in `set_code`
Check that the new runtime code given to `set_code` fullfills some
requirements:
- `spec_name` matches
- `spec_version` does not decreases
- `impl_version` does not decreases
- Either `spec_version` and `impl_version` increase
* Make tests almost work
* Some fixes after master merge
* Fix tests
* Add missed file
* Make depedency check happy?
* Remove leftover `sc-executor`
* AHHHHH
* Reset debug stuff
* Remove some 'static
* More 'static
* Some docs
* Update `Cargo.lock`
Instead of having these weird implementation of `BuildStorage for
&ChainSpec` we should just take the `genesis_storage` by ref. The
`BuildStorage` trait changed some time ago to take a self ref anyway,
instead of a self value.
Also fixes warnings in frame-staking
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* Make wasm-builder remove invalid members
Wasm-builder now removes members that point to packages that do not
exist anymore or that were renamed. Up to now, the build would fail and
required manual fixing.
* Fixes build
* Fix cli for structopt 0.3.7 and pin to that version
This is just some hotfix to make everything compile. In the future it
will require another pr to not depend on internals of StructOpt, but
that will probably also require some additions to StructOpt itself. To
not break the code again with another StructOpt, this also pins the
StructOpt version.
* Fix benches
* Fix for fix
* Make `decl_error!` errors usable
This pr implements support for returning errors of different pallets in
a pallet. These errors need to be declared with `decl_error!`.
The pr changes the following:
- Each dispatchable function now returns a `DispatchResult` which is an
alias for `Result<(), DispatchError>`.
- `DispatchError` is an enum that has 4 variants:
- `Other`: For storing string error messages
- `CannotLookup`: Variant that is returned when something returns a
`sp_runtime::LookupError`
- `BadOrigin`: Variant that is returned for any kind of bad origin
- `Module`: The error of a specific module. Contains the `index`,
`error` and the `message`. The index is the index of the module in
`construct_runtime!`. `error` is the index of the error in the error
enum declared by `decl_error!`. `message` is the message to the error
variant (this will not be encoded).
- `construct_runtime!` now creates a new struct `ModuleToIndex`. This
struct implements the trait `ModuleToIndex`.
- `frame_system::Trait` has a new associated type: `ModuleToIndex` that
expects the `ModuleToIndex` generated by `construct_runtime!`.
- All error strings returned in any module are being converted now to `DispatchError`.
- `BadOrigin` is the default error returned by any type that implements `EnsureOrigin`.
* Fix frame system benchmarks
* Check in block authoring that we can author with current authoring version
* Update client/consensus/pow/src/lib.rs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* Fix compilation
* The storage runtime interface should not enforce a hash type
Currently the runtime interface enforces `H256` as hash type, but in the
future people could use whatever they want as hash type. The hash type
always needs to match between the runtime and the node, but that is
already required.
* Update primitives/externalities/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Switch to new rocksdb with some hacks to resolve linking errors
Rocksdb and `wasmtime-environ` both link `zstd`. In the final link step,
this leads to linking errors because of duplicate symbols. The linked
Rocksdb fixes this by using `zstd-sys` as well. However, this currently
also requires modifications in `zstd-sys`. Someone will need to come up
with a better implementation.
* Switch to new version of kvdb-rocksdb
* Update client/db/src/utils.rs
Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>
When we enabled arbitary precision for `serde_json`, the feature leaked
through the whole workspace. The given test fails because of some bug in
serde json when the feature is enabled.
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Adds first version of traits for generating the host functions
* First steps of the procedural macro
* Implements generation of the host extern functions
* Prefix ext host function with snake case trait name
* Implement host functions implementation on the host
* Change `HostFunctions` interface
* Implement `HostFunctions` for tuples
* Make `WasmExecutor` generic over the host functions
* Begin to add a test and make it compile
* Make the test succeed
* Add test to ensure that host functions are not found
* It's alive! Make the `set_storage` test work
* Add test for mutable references
* Code cleanup and documentation etc
* Add marker trait for types that should be passed as SCALE encoded
* Inherit the visibility from the trait and more improvements
* More impls and move them into their own file
* Code simplification by dropping one trait
* Give it a better name
* Implement traits for arrays
* Refactor code to support pass by codec/inner
* Docs
* Implement pass by inner for some crypto types and add a test
* Implement exchangeable function support
* Rewrite sr-io with as runtime interface
* Start reworking after master merge
* Adds `PassByCodec` derive
* Adds `PassByInner` derive
* Fix compilation errors
* More implementations
* Implement runtime interface traits for `str`
* Make `sr-io` compile again
* Fix more compilation errors
* More progress on getting stuff back to compile
* More compilation fixes
* Fix warnings
* Remove le conversions
* Add support for `wasm_only` interfaces
* Implement `Allocator` interface
* Improve error message
* Move `WasmAllocator` to `sr-io` and more clean ups
* Use correct function signature for wasm functions
* Store the host functions with the Wasm runtime
* Docs update
* Fix compilation after master merge
* Remove `sr-io/without_std`
* Make `srml-support` tests run again
* More compilation error fixes
* Use correct doc syntax
* Fix test-runtime
* Fix compilation
* Catch native panics when executing the wasm runtime
As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.
* Fix compilation
* Don't panic in a panic
* Move `sr-sandbox` to new runtime interface
* Fixes tests after sandbox changes
* Make sure we detect invalid utf8
* Fixes after master merge
* Adds pass by enum strategy
* Fix wasmtime integration
* Some macro structure clean up
* Rework and test exchangebale host functions
* PassBy derive macros documentation
* Docs for `runtime_interface` macro
* Support wild card argument names
* Adds ui tests
* Make sure that we are backwards compatible to the old runtime interfaces
* Documentation
* Fixes after latest master merge
* Make `wasmtime` happy
* Make `full_crypto` work
* Make the new interface versionable
* Rename `Sanboxing` to `Sandbox`
* Don't finalize in test while importing
* Fix Performance regression
* Fix test
- Add new crate `substrate-build-script-utils` to unify the code of
`node`, `node-template` and `polkadot-node`.
- The `node-cli` build script needs to search upwards for the
`.git/HEAD` file to find it.
As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.
* Remove footgun around session keys/handlers
- `OpaqueKeys` now has an associated type `KeyTypeIdProviders`. This can
be used in the runtime as input for `SessionHandler` from the session
trait.
- `impl_opaque_keys` now works with modules and extracts the `KeyTypeId`
from the module directly.
- Added some checks to the `session` storage initialization that checks
that the `SessionHandler` and `Keys` use the same number of keys and
that the order is equal.
* Update core/sr-primitives/src/traits.rs
* Adds function `to_substrate_wasm_fn_return_value`
Instead of replicating this piece of code over and over again, just move
it to a function that does it.
* Feedback
* Comment
* Wasm-builder-runner unset `CARGO_TARGET_DIR` and release 1.0.4
`CARGO_TARGET_DIR` needs to be unset or otherwise cargo deadlocks,
because cargo always holds an exclusive lock on target dir.
* Commit missing version up
* Lock file
* Make `wasmi_execution` public to use it from tests
* Make `WasmRuntime` accessible as well
* Add `call_in_wasm` instead of making stuff public
* Use `WasmRuntime`
* Move test
* More feedback
The macro generates the functions with the signature we expect for wasm
functions. This macro is useful for tests where we need to call into
wasm. Parameter passing is done by SCALE encoding the input and output
parameters.
This function executes the given closure in a context where the test
externalities are set. This makes the srml tests easier to write, as the
test externalities need to be created anyway.
* Move `Externalities` into `substrate-externalities`
- `Externalities` now support generic extensions
- Split of `primtives-storage` for storage primitive types
* Move the externalities scoping into `substrate-externalities`
* Fix compilation
* Review feedback
* Adds macro for declaring extensions
* Fix benchmarks
* Introduce `ExtensionStore` trait
* Last review comments
* Implement it for `ExtensionStore`
So, instead of:
`type ALL_MODULES = (Module1, Module2, Module3)`;
Generate it like this:
`type ALL_MODULES = ((Module1, Module2), Module3,);`
This should make `construct_runtime!` support an "unlimited" number of
modules.
* Give more information why conversion between native and runtime failed
This adds the SCALE error description to the error when the conversion
between native and runtime failed.
* Fixes tests
`debug-assertions` can also be enabled in for release builds. This
introduces a new build-script that extracts the build type from the
`PROFILE` env variable and sets the `build_type` cfg.
By accident it was already enabled in master for quite some time. To
make sure that we don't blow up the wasm binary size, I compiled the
binary with the feature enabled and disabled. With nice panic messages
enabled, the binary size increases by 908 bytes. Given the value that
this feature brings, I think it is okay to have these panic messages
enabled by default.
* Implement support for `patch` section in wasm-builder
* Update core/utils/wasm-builder/src/wasm_project.rs
* Support patch by path
* Go down to the actual data
* Switch to new proc-macro crate for implementing traits for tuples
* Switch back to stable `Cargo.lock` format
* Reduce tuple number to match rusts default implementation
* Adds new wrapper traits for wasm executor
* Add new crate `substrate-wasm-interface`
Thew new crate holds types and traits for the communicating between the
wasm runtime and the host.
* Rewrite externals with new macro etc
* Fix vec initialization
* Make executor tests working
* Remove unused code + warnings
* Introduce `Pointer` and `WordSize` for working with wasm
* Fix tests and compilation
* Fix compilation
* Apply suggestions from code review
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Review feedback
* Remove unused conversions
* Make each host function its own struct
`HostFunctions` now just returns these function structs. Each function
can be executed by using one of the function structs. The inherent host
functions are now moved to the "normal" host functions.
* Remove byteorder
* Add floating point types
* Make pointer interface more safe
* Add type alias for wasm-interface Result
* More review comments
* srml-system checks
* wip
* more modules compiles
* node-runtime checks
* build.sh passes
* include dispatch error in failed event
* revert some unnecessary changes
* refactor based on comments
* more compile error fixes
* avoid unnecessary into
* reorder code
* fixes some tests
* manually implement encode & decode to avoid i8 workaround
* more test fixes
* more fixes
* more error fixes
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* address comments
* test for DispatchError encoding
* tyep alias for democracy
* make error printable
* line width
* fix balances tests
* fix executive test
* fix system tests
* bump version
* ensure consistent method signature
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <github@gavwood.com>
* changes based on review
* Add issue number for TODOs
* fix
* line width
* fix test
* Update core/sr-primitives/src/lib.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update core/sr-primitives/src/traits.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update srml/council/src/motions.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Update srml/council/src/motions.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* update based on review
* More concrete macro matching
* fix test build issue
* Update hex-literal dependency version. (#3141)
* Update hex-literal dep version.
* Update lock file.
* Start to rework the new error handling
* More work to get it back compiling
* Start to fix after master merge
* The great transaction error handling refactoring
* Make `decl_error` errors convertible to `&'static str`
* Make srml-executive build again
* Fix `sr-primitives` tests
* More fixes
* Last round of fix ups
* Fix build
* Fix build
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Rename some stuff
* Fixes after master merge
* Adds `CheckBlockGasLimit` signed extension
* Remove debug stuff
* Fix srml-balances test
* Rename `InvalidIndex` to `CannotLookup`
* Remove weird generic parameters
* Rename function again
* Fix import
* Document the signed extension
* Change from `Into` to `From`
* Update srml/contracts/src/lib.rs
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Fix compilation
* Update srml/contracts/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update core/sr-primitives/src/transaction_validity.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Remove unused code
* Fix compilation
* Some cleanups
* Fix compile errors
* Make `TransactionValidity` a `Result`
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <gavin@parity.io>
* Beautify the code a little bit and fix test
* Make `CannotLookup` an inherent error declared by `decl_error!`
* Adds some documentation
* Make `ApplyOutcome` a result
* Up the spec_version
* Apply suggestions from code review
Co-Authored-By: Gavin Wood <gavin@parity.io>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
Bastian Köcher