Metadata V15: Expose types for the overarching Call, Event, Error enums (#14143)

* frame-metadata: Point to unreleased branch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Generalize outer enum generation for events and errors

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Remove individual generation of outer enum events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* primitives/traits: Add marker trait for outer runtime enums

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Derive Clone, PartialEq, Eq for RuntimeEvents only

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/pallet: Include `#[pallet::error]` enum into pallet parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata-ir: Include call, event, error types

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata: Include outer enum types in V15 metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Ensure `RuntimeError` includes `#[pallet::error]` parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Document the reserved name for `RuntimeError`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Use self-generated `RuntimeEvent` for `GetRuntimeOuterEnumTypes`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/ui: Fix UI tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Remove unused system path

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/ui: Unexpected field and reintroduce frame_system::Config for RuntimeCall

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Remove `GetRuntimeOuterEnumTypes` marker trait

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Remove `;` from macro

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame-metadata to point to unreleased branch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename error_enum_ty to module_error_enum_ty

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update module_error_ty documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Implement from_dispatch_error

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Adjust test to ModuleErrorType

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Check `from_dispatch_error` impl

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame-metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove the module_error_ty

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert unneeded parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "Revert unneeded parts"

This reverts commit b94bbd16078a025775a48da1095edec1705e6a4d.

Revert "Apply fmt"

This reverts commit 9b1c3e7b4ef27d32e10b35054a99916067e0397b.

Revert "Remove the module_error_ty"

This reverts commit 98de5b24653f9f9ec6ee842b749401b18a01758a.

* Update frame-metadata to origin/master

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add outerEnums to the metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Keep backwards compatibility for explicit pallet parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename tt_error_part to be more generic

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Increase recursion_limit to 1k

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename `fully_expanded` to `expanded`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust UI tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update UI tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update undefined_validate_unsigned_part.stderr UI test

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust yet again

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Optimise macro expansions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use latest frame-metadata and rename `moduleErrorType` to `RuntimeError`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix comment

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame/support/procedural/src/construct_runtime/parse.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/procedural/src/construct_runtime/parse.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame-metadata PR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove `expanded` from error messages and fix typo

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Move docs to the function

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ui: Use the intermed syntax for pallet parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame-metadata with latest release

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Address feedback for `from_dispatch_error`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
This commit is contained in:
Alexandru Vasile
2023-06-28 12:44:05 +03:00
committed by GitHub
parent 4249643df2
commit 73a368c2e4
33 changed files with 1143 additions and 337 deletions
@@ -20,17 +20,69 @@
//! `construct_runtime` implementation is recursive and can generate code which will call itself in
//! order to get all the pallet parts for each pallet.
//!
//! Pallets define their parts (`Call`, `Storage`, ..) either explicitly with the syntax
//! `::{Call, ...}` or implicitly.
//! Pallets can define their parts:
//! - Implicitely: `System: frame_system`
//! - Explicitly: `System: frame_system::{Pallet, Call}`
//!
//! In case a pallet defines its parts implicitly, then the pallet must provide the
//! `tt_default_parts` macro. `construct_runtime` will generate some code which utilizes `tt_call`
//! to call the `tt_default_parts` macro of the pallet. `tt_default_parts` will then return the
//! default pallet parts as input tokens to the `match_and_replace` macro, which ultimately
//! generates a call to `construct_runtime` again, this time with all the pallet parts explicitly
//! defined.
//! The `construct_runtime` transitions from the implicit definition to the explict one.
//! From the explicit state, Substrate expands the pallets with additional information
//! that is to be included in the runtime metadata. This expansion makes visible some extra
//! parts of the pallets, mainly the `Error` if defined. The expanded state looks like
//! `System: frame_system expanded::{Error} ::{Pallet, Call}` and concatenates the extra expanded
//! parts with the user-provided parts. For example, the `Pallet`, `Call` and `Error` parts are
//! collected.
//!
//! Pallets must provide the `tt_extra_parts` and `tt_default_parts` macros for these transitions.
//! These are automatically implemented by the `#[pallet::pallet]` macro.
//!
//! This macro also generates the following enums for ease of decoding:
//! - `enum RuntimeCall`: This type contains the information needed to decode extrinsics.
//! - `enum RuntimeEvent`: This type contains the information needed to decode events.
//! - `enum RuntimeError`: While this cannot be used directly to decode `sp_runtime::DispatchError`
//! from the chain, it contains the information needed to decode the
//! `sp_runtime::DispatchError::Module`.
//!
//! # State Transitions
//!
//! ```ignore
//! +----------+
//! | Implicit | -----------+
//! +----------+ |
//! | |
//! v v
//! +----------+ +------------------+
//! | Explicit | --> | ExplicitExpanded |
//! +----------+ +------------------+
//! ```
//!
//! When all pallet parts are implcit, then the `construct_runtime!` macro expands to its final
//! state, the `ExplicitExpanded`. Otherwise, all implicit parts are converted to an explicit
//! expanded part allow the `construct_runtime!` to expand any remaining explicit parts to an
//! explicit expanded part.
//!
//! # Implicit to Explicit
//!
//! The `construct_runtime` macro transforms the implicit declaration of each pallet
//! `System: frame_system` to an explicit one `System: frame_system::{Pallet, Call}` using the
//! `tt_default_parts` macro.
//!
//! The `tt_default_parts` macro exposes a comma separated list of pallet parts. For example, the
//! `Event` part is exposed only if the pallet implements an event via `#[pallet::event]` macro.
//! The tokens generated by this macro are ` expanded :: { Pallet, Call }` for our example.
//!
//! The `match_and_insert` macro takes in 3 arguments:
//! - target: This is the `TokenStream` that contains the `construct_runtime!` macro.
//! - pattern: The pattern to match against in the target stream.
//! - tokens: The tokens to added after the pattern match.
//!
//! The `construct_runtime` macro uses the `tt_call` to get the default pallet parts via
//! the `tt_default_parts` macro defined by each pallet. The pallet parts are then returned as
//! input to the `match_and_replace` macro.
//! The `match_and_replace` then will modify the the `construct_runtime!` to expand the implicit
//! definition to the explicit one.
//!
//! For example,
//!
//! E.g.
//! ```ignore
//! construct_runtime!(
//! //...
@@ -106,6 +158,7 @@
//! tokens = [{ ::{Pallet, Call} }]
//! }
//! ```
//!
//! Which will then finally expand to the following:
//! ```ignore
//! construct_runtime!(
@@ -116,6 +169,7 @@
//! }
//! )
//! ```
//!
//! This call has no implicit pallet parts, thus it will expand to the runtime construction:
//! ```ignore
//! pub struct Runtime { ... }
@@ -140,6 +194,19 @@
//! | w/ pallet parts |
//! +--------------------+
//! ```
//!
//! # Explicit to Explicit Expanded
//!
//! Users normally do not care about this transition.
//!
//! Similarly to the previous transition, the macro expansion transforms `System:
//! frame_system::{Pallet, Call}` into `System: frame_system expanded::{Error} ::{Pallet, Call}`.
//! The `expanded` section adds extra parts that the Substrate would like to expose for each pallet
//! by default. This is done to expose the approprite types for metadata construction.
//!
//! This time, instead of calling `tt_default_parts` we are using the `tt_extra_parts` macro.
//! This macro returns the ` :: expanded { Error }` list of additional parts we would like to
//! expose.
mod expand;
mod parse;
@@ -171,9 +238,16 @@ pub fn construct_runtime(input: TokenStream) -> TokenStream {
let res = match definition {
RuntimeDeclaration::Implicit(implicit_def) =>
check_pallet_number(input_copy.clone().into(), implicit_def.pallets.len()).and_then(
|_| construct_runtime_intermediary_expansion(input_copy.into(), implicit_def),
|_| construct_runtime_implicit_to_explicit(input_copy.into(), implicit_def),
),
RuntimeDeclaration::Explicit(explicit_decl) =>
RuntimeDeclaration::Explicit(explicit_decl) => check_pallet_number(
input_copy.clone().into(),
explicit_decl.pallets.len(),
)
.and_then(|_| {
construct_runtime_explicit_to_explicit_expanded(input_copy.into(), explicit_decl)
}),
RuntimeDeclaration::ExplicitExpanded(explicit_decl) =>
check_pallet_number(input_copy.into(), explicit_decl.pallets.len())
.and_then(|_| construct_runtime_final_expansion(explicit_decl)),
};
@@ -189,9 +263,14 @@ pub fn construct_runtime(input: TokenStream) -> TokenStream {
res.into()
}
/// When some pallet have implicit parts definition then the macro will expand into a macro call to
/// `construct_runtime_args` of each pallets, see root documentation.
fn construct_runtime_intermediary_expansion(
/// All pallets that have implicit pallet parts (ie `System: frame_system`) are
/// expanded with the default parts defined by the pallet's `tt_default_parts` macro.
///
/// This function transforms the [`RuntimeDeclaration::Implicit`] into
/// [`RuntimeDeclaration::Explicit`] that is not yet fully expanded.
///
/// For more details, please refer to the root documentation.
fn construct_runtime_implicit_to_explicit(
input: TokenStream2,
definition: ImplicitRuntimeDeclaration,
) -> Result<TokenStream2> {
@@ -218,6 +297,42 @@ fn construct_runtime_intermediary_expansion(
Ok(expansion)
}
/// All pallets that have
/// (I): explicit pallet parts (ie `System: frame_system::{Pallet, Call}`) and
/// (II): are not fully expanded (ie do not include the `Error` expansion part)
/// are fully expanded by including the parts from the pallet's `tt_extra_parts` macro.
///
/// This function transforms the [`RuntimeDeclaration::Explicit`] that is not yet fully expanded
/// into [`RuntimeDeclaration::ExplicitExpanded`] fully expanded.
///
/// For more details, please refer to the root documentation.
fn construct_runtime_explicit_to_explicit_expanded(
input: TokenStream2,
definition: ExplicitRuntimeDeclaration,
) -> Result<TokenStream2> {
let frame_support = generate_crate_access_2018("frame-support")?;
let mut expansion = quote::quote!(
#frame_support::construct_runtime! { #input }
);
for pallet in definition.pallets.iter().filter(|pallet| !pallet.is_expanded) {
let pallet_path = &pallet.path;
let pallet_name = &pallet.name;
let pallet_instance = pallet.instance.as_ref().map(|instance| quote::quote!(::<#instance>));
expansion = quote::quote!(
#frame_support::tt_call! {
macro = [{ #pallet_path::tt_extra_parts }]
frame_support = [{ #frame_support }]
~~> #frame_support::match_and_insert! {
target = [{ #expansion }]
pattern = [{ #pallet_name: #pallet_path #pallet_instance }]
}
}
);
}
Ok(expansion)
}
/// All pallets have explicit definition of parts, this will expand to the runtime declaration.
fn construct_runtime_final_expansion(
definition: ExplicitRuntimeDeclaration,
@@ -264,14 +379,23 @@ fn construct_runtime_final_expansion(
let scrate = generate_crate_access(hidden_crate_name, "frame-support");
let scrate_decl = generate_hidden_includes(hidden_crate_name, "frame-support");
let outer_event = expand::expand_outer_event(&name, &pallets, &scrate)?;
let outer_event =
expand::expand_outer_enum(&name, &pallets, &scrate, expand::OuterEnumType::Event)?;
let outer_error =
expand::expand_outer_enum(&name, &pallets, &scrate, expand::OuterEnumType::Error)?;
let outer_origin = expand::expand_outer_origin(&name, system_pallet, &pallets, &scrate)?;
let all_pallets = decl_all_pallets(&name, pallets.iter(), &features);
let pallet_to_index = decl_pallet_runtime_setup(&name, &pallets, &scrate);
let dispatch = expand::expand_outer_dispatch(&name, system_pallet, &pallets, &scrate);
let metadata = expand::expand_runtime_metadata(&name, &pallets, &scrate, &unchecked_extrinsic);
let metadata = expand::expand_runtime_metadata(
&name,
&pallets,
&scrate,
&unchecked_extrinsic,
&system_pallet.path,
);
let outer_config = expand::expand_outer_config(&name, &pallets, &scrate);
let inherent =
expand::expand_outer_inherent(&name, &block, &unchecked_extrinsic, &pallets, &scrate);
@@ -331,6 +455,8 @@ fn construct_runtime_final_expansion(
#outer_event
#outer_error
#outer_origin
#all_pallets