Substrate runtime interface 2.0 (#4057)

* Adds first version of traits for generating the host functions

* First steps of the procedural macro

* Implements generation of the host extern functions

* Prefix ext host function with snake case trait name

* Implement host functions implementation on the host

* Change `HostFunctions` interface

* Implement `HostFunctions` for tuples

* Make `WasmExecutor` generic over the host functions

* Begin to add a test and make it compile

* Make the test succeed

* Add test to ensure that host functions are not found

* It's alive! Make the `set_storage` test work

* Add test for mutable references

* Code cleanup and documentation etc

* Add marker trait for types that should be passed as SCALE encoded

* Inherit the visibility from the trait and more improvements

* More impls and move them into their own file

* Code simplification by dropping one trait

* Give it a better name

* Implement traits for arrays

* Refactor code to support pass by codec/inner

* Docs

* Implement pass by inner for some crypto types and add a test

* Implement exchangeable function support

* Rewrite sr-io with as runtime interface

* Start reworking after master merge

* Adds `PassByCodec` derive

* Adds `PassByInner` derive

* Fix compilation errors

* More implementations

* Implement runtime interface traits for `str`

* Make `sr-io` compile again

* Fix more compilation errors

* More progress on getting stuff back to compile

* More compilation fixes

* Fix warnings

* Remove le conversions

* Add support for `wasm_only` interfaces

* Implement `Allocator` interface

* Improve error message

* Move `WasmAllocator` to `sr-io` and more clean ups

* Use correct function signature for wasm functions

* Store the host functions with the Wasm runtime

* Docs update

* Fix compilation after master merge

* Remove `sr-io/without_std`

* Make `srml-support` tests run again

* More compilation error fixes

* Use correct doc syntax

* Fix test-runtime

* Fix compilation

* Catch native panics when executing the wasm runtime

As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.

* Fix compilation

* Don't panic in a panic

* Move `sr-sandbox` to new runtime interface

* Fixes tests after sandbox changes

* Make sure we detect invalid utf8

* Fixes after master merge

* Adds pass by enum strategy

* Fix wasmtime integration

* Some macro structure clean up

* Rework and test exchangebale host functions

* PassBy derive macros documentation

* Docs for `runtime_interface` macro

* Support wild card argument names

* Adds ui tests

* Make sure that we are backwards compatible to the old runtime interfaces

* Documentation

* Fixes after latest master merge

* Make `wasmtime` happy

* Make `full_crypto` work

* Make the new interface versionable

* Rename `Sanboxing` to `Sandbox`

* Don't finalize in test while importing

* Fix Performance regression

* Fix test
This commit is contained in:
Bastian Köcher
2019-11-10 21:59:30 +01:00
committed by GitHub
parent f4c1c83667
commit b691cfe093
118 changed files with 4901 additions and 2829 deletions
@@ -0,0 +1,162 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>
//! Util function used by this crate.
use proc_macro2::{TokenStream, Span};
use syn::{
Ident, Error, Signature, Pat, PatType, FnArg, Type, token, TraitItemMethod, ItemTrait,
TraitItem, parse_quote, spanned::Spanned,
};
use proc_macro_crate::crate_name;
use std::env;
use quote::quote;
use inflector::Inflector;
/// Generates the include for the runtime-interface crate.
pub fn generate_runtime_interface_include() -> TokenStream {
if env::var("CARGO_PKG_NAME").unwrap() == "substrate-runtime-interface" {
TokenStream::new()
} else {
match crate_name("substrate-runtime-interface") {
Ok(crate_name) => {
let crate_name = Ident::new(&crate_name, Span::call_site());
quote!(
#[doc(hidden)]
extern crate #crate_name as proc_macro_runtime_interface;
)
},
Err(e) => {
let err = Error::new(Span::call_site(), &e).to_compile_error();
quote!( #err )
}
}
}
}
/// Generates the access to the `substrate-runtime-interface` crate.
pub fn generate_crate_access() -> TokenStream {
if env::var("CARGO_PKG_NAME").unwrap() == "substrate-runtime-interface" {
quote!( substrate_runtime_interface )
} else {
quote!( proc_macro_runtime_interface )
}
}
/// Create the exchangeable host function identifier for the given function name.
pub fn create_exchangeable_host_function_ident(name: &Ident) -> Ident {
Ident::new(&format!("host_{}", name), Span::call_site())
}
/// Create the host function identifier for the given function name.
pub fn create_host_function_ident(name: &Ident, trait_name: &Ident) -> Ident {
Ident::new(
&format!(
"ext_{}_{}_version_1",
trait_name.to_string().to_snake_case(),
name,
),
Span::call_site(),
)
}
/// Returns the function arguments of the given `Signature`, minus any `self` arguments.
pub fn get_function_arguments<'a>(sig: &'a Signature) -> impl Iterator<Item = PatType> + 'a {
sig.inputs
.iter()
.filter_map(|a| match a {
FnArg::Receiver(_) => None,
FnArg::Typed(pat_type) => Some(pat_type),
})
.enumerate()
.map(|(i, arg)| {
let mut res = arg.clone();
if let Pat::Wild(wild) = &*arg.pat {
let ident = Ident::new(
&format!("__runtime_interface_generated_{}_", i),
wild.span(),
);
res.pat = Box::new(parse_quote!( #ident ))
}
res
})
}
/// Returns the function argument names of the given `Signature`, minus any `self`.
pub fn get_function_argument_names<'a>(sig: &'a Signature) -> impl Iterator<Item = Box<Pat>> + 'a {
get_function_arguments(sig).map(|pt| pt.pat)
}
/// Returns the function argument types of the given `Signature`, minus any `Self` type.
pub fn get_function_argument_types<'a>(sig: &'a Signature) -> impl Iterator<Item = Box<Type>> + 'a {
get_function_arguments(sig).map(|pt| pt.ty)
}
/// Returns the function argument types, minus any `Self` type. If any of the arguments
/// is a reference, the underlying type without the ref is returned.
pub fn get_function_argument_types_without_ref<'a>(
sig: &'a Signature,
) -> impl Iterator<Item = Box<Type>> + 'a {
get_function_arguments(sig)
.map(|pt| pt.ty)
.map(|ty| match *ty {
Type::Reference(type_ref) => type_ref.elem,
_ => ty,
})
}
/// Returns the function argument names and types, minus any `self`. If any of the arguments
/// is a reference, the underlying type without the ref is returned.
pub fn get_function_argument_names_and_types_without_ref<'a>(
sig: &'a Signature,
) -> impl Iterator<Item = (Box<Pat>, Box<Type>)> + 'a {
get_function_arguments(sig)
.map(|pt| match *pt.ty {
Type::Reference(type_ref) => (pt.pat, type_ref.elem),
_ => (pt.pat, pt.ty),
})
}
/// Returns the `&`/`&mut` for all function argument types, minus the `self` arg. If a function
/// argument is not a reference, `None` is returned.
pub fn get_function_argument_types_ref_and_mut<'a>(
sig: &'a Signature,
) -> impl Iterator<Item = Option<(token::And, Option<token::Mut>)>> + 'a {
get_function_arguments(sig)
.map(|pt| pt.ty)
.map(|ty| match *ty {
Type::Reference(type_ref) => Some((type_ref.and_token, type_ref.mutability)),
_ => None,
})
}
/// Returns an iterator over all trait methods for the given trait definition.
pub fn get_trait_methods<'a>(trait_def: &'a ItemTrait) -> impl Iterator<Item = &'a TraitItemMethod> {
trait_def
.items
.iter()
.filter_map(|i| match i {
TraitItem::Method(ref method) => Some(method),
_ => None,
})
}