Metadata V15: Expose types for the overarching Call, Event, Error enums (#14143)

* frame-metadata: Point to unreleased branch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Generalize outer enum generation for events and errors

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Remove individual generation of outer enum events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* primitives/traits: Add marker trait for outer runtime enums

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Derive Clone, PartialEq, Eq for RuntimeEvents only

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/pallet: Include `#[pallet::error]` enum into pallet parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata-ir: Include call, event, error types

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata: Include outer enum types in V15 metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Ensure `RuntimeError` includes `#[pallet::error]` parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Document the reserved name for `RuntimeError`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Use self-generated `RuntimeEvent` for `GetRuntimeOuterEnumTypes`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/ui: Fix UI tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Remove unused system path

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/ui: Unexpected field and reintroduce frame_system::Config for RuntimeCall

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Remove `GetRuntimeOuterEnumTypes` marker trait

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Remove `;` from macro

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame-metadata to point to unreleased branch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename error_enum_ty to module_error_enum_ty

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update module_error_ty documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Implement from_dispatch_error

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Adjust test to ModuleErrorType

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Check `from_dispatch_error` impl

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame-metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove the module_error_ty

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert unneeded parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "Revert unneeded parts"

This reverts commit b94bbd16078a025775a48da1095edec1705e6a4d.

Revert "Apply fmt"

This reverts commit 9b1c3e7b4ef27d32e10b35054a99916067e0397b.

Revert "Remove the module_error_ty"

This reverts commit 98de5b24653f9f9ec6ee842b749401b18a01758a.

* Update frame-metadata to origin/master

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add outerEnums to the metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Keep backwards compatibility for explicit pallet parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename tt_error_part to be more generic

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Increase recursion_limit to 1k

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename `fully_expanded` to `expanded`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust UI tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update UI tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update undefined_validate_unsigned_part.stderr UI test

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust yet again

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Optimise macro expansions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use latest frame-metadata and rename `moduleErrorType` to `RuntimeError`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix comment

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame/support/procedural/src/construct_runtime/parse.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/procedural/src/construct_runtime/parse.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame-metadata PR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove `expanded` from error messages and fix typo

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Move docs to the function

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ui: Use the intermed syntax for pallet parts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame-metadata with latest release

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame: Address feedback for `from_dispatch_error`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
This commit is contained in:
Alexandru Vasile
2023-06-28 12:44:05 +03:00
committed by GitHub
parent 4249643df2
commit 73a368c2e4
33 changed files with 1143 additions and 337 deletions
@@ -1,168 +0,0 @@
// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License
use crate::construct_runtime::Pallet;
use proc_macro2::TokenStream;
use quote::quote;
use std::str::FromStr;
use syn::{Generics, Ident};
pub fn expand_outer_event(
runtime: &Ident,
pallet_decls: &[Pallet],
scrate: &TokenStream,
) -> syn::Result<TokenStream> {
let mut event_variants = TokenStream::new();
let mut event_conversions = TokenStream::new();
let mut query_event_part_macros = Vec::new();
for pallet_decl in pallet_decls {
if let Some(pallet_entry) = pallet_decl.find_part("Event") {
let path = &pallet_decl.path;
let pallet_name = &pallet_decl.name;
let index = pallet_decl.index;
let instance = pallet_decl.instance.as_ref();
let generics = &pallet_entry.generics;
if instance.is_some() && generics.params.is_empty() {
let msg = format!(
"Instantiable pallet with no generic `Event` cannot \
be constructed: pallet `{}` must have generic `Event`",
pallet_name,
);
return Err(syn::Error::new(pallet_name.span(), msg))
}
let part_is_generic = !generics.params.is_empty();
let pallet_event = match (instance, part_is_generic) {
(Some(inst), true) => quote!(#path::Event::<#runtime, #path::#inst>),
(Some(inst), false) => quote!(#path::Event::<#path::#inst>),
(None, true) => quote!(#path::Event::<#runtime>),
(None, false) => quote!(#path::Event),
};
event_variants.extend(expand_event_variant(
runtime,
pallet_decl,
index,
instance,
generics,
));
event_conversions.extend(expand_event_conversion(scrate, pallet_decl, &pallet_event));
query_event_part_macros.push(quote! {
#path::__substrate_event_check::is_event_part_defined!(#pallet_name);
});
}
}
Ok(quote! {
#( #query_event_part_macros )*
#[derive(
Clone, PartialEq, Eq,
#scrate::codec::Encode,
#scrate::codec::Decode,
#scrate::scale_info::TypeInfo,
#scrate::RuntimeDebug,
)]
#[allow(non_camel_case_types)]
pub enum RuntimeEvent {
#event_variants
}
#event_conversions
})
}
fn expand_event_variant(
runtime: &Ident,
pallet: &Pallet,
index: u8,
instance: Option<&Ident>,
generics: &Generics,
) -> TokenStream {
let path = &pallet.path;
let variant_name = &pallet.name;
let part_is_generic = !generics.params.is_empty();
let attr = pallet.cfg_pattern.iter().fold(TokenStream::new(), |acc, pattern| {
let attr = TokenStream::from_str(&format!("#[cfg({})]", pattern.original()))
.expect("was successfully parsed before; qed");
quote! {
#acc
#attr
}
});
match instance {
Some(inst) if part_is_generic => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::Event<#runtime, #path::#inst>),
},
Some(inst) => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::Event<#path::#inst>),
},
None if part_is_generic => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::Event<#runtime>),
},
None => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::Event),
},
}
}
fn expand_event_conversion(
scrate: &TokenStream,
pallet: &Pallet,
pallet_event: &TokenStream,
) -> TokenStream {
let variant_name = &pallet.name;
let attr = pallet.cfg_pattern.iter().fold(TokenStream::new(), |acc, pattern| {
let attr = TokenStream::from_str(&format!("#[cfg({})]", pattern.original()))
.expect("was successfully parsed before; qed");
quote! {
#acc
#attr
}
});
quote! {
#attr
impl From<#pallet_event> for RuntimeEvent {
fn from(x: #pallet_event) -> Self {
RuntimeEvent::#variant_name(x)
}
}
#attr
impl TryInto<#pallet_event> for RuntimeEvent {
type Error = ();
fn try_into(self) -> #scrate::sp_std::result::Result<#pallet_event, Self::Error> {
match self {
Self::#variant_name(evt) => Ok(evt),
_ => Err(()),
}
}
}
}
}
@@ -15,7 +15,7 @@
// See the License for the specific language governing permissions and
// limitations under the License
use crate::construct_runtime::Pallet;
use crate::construct_runtime::{parse::PalletPath, Pallet};
use proc_macro2::TokenStream;
use quote::quote;
use std::str::FromStr;
@@ -26,6 +26,7 @@ pub fn expand_runtime_metadata(
pallet_declarations: &[Pallet],
scrate: &TokenStream,
extrinsic: &TypePath,
system_path: &PalletPath,
) -> TokenStream {
let pallets = pallet_declarations
.iter()
@@ -115,6 +116,13 @@ pub fn expand_runtime_metadata(
},
ty: #scrate::scale_info::meta_type::<#runtime>(),
apis: (&rt).runtime_metadata(),
outer_enums: #scrate::metadata_ir::OuterEnumsIR {
call_enum_ty: #scrate::scale_info::meta_type::<
<#runtime as #system_path::Config>::RuntimeCall
>(),
event_enum_ty: #scrate::scale_info::meta_type::<RuntimeEvent>(),
error_enum_ty: #scrate::scale_info::meta_type::<RuntimeError>(),
}
}
}
@@ -17,24 +17,24 @@
mod call;
mod config;
mod event;
mod freeze_reason;
mod hold_reason;
mod inherent;
mod lock_id;
mod metadata;
mod origin;
mod outer_enums;
mod slash_reason;
mod unsigned;
pub use call::expand_outer_dispatch;
pub use config::expand_outer_config;
pub use event::expand_outer_event;
pub use freeze_reason::expand_outer_freeze_reason;
pub use hold_reason::expand_outer_hold_reason;
pub use inherent::expand_outer_inherent;
pub use lock_id::expand_outer_lock_id;
pub use metadata::expand_runtime_metadata;
pub use origin::expand_outer_origin;
pub use outer_enums::{expand_outer_enum, OuterEnumType};
pub use slash_reason::expand_outer_slash_reason;
pub use unsigned::expand_outer_validate_unsigned;
@@ -0,0 +1,283 @@
// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License
use crate::construct_runtime::Pallet;
use proc_macro2::{Span, TokenStream};
use quote::{quote, ToTokens};
use std::str::FromStr;
use syn::{Generics, Ident};
/// Represents the types supported for creating an outer enum.
#[derive(Clone, Copy, PartialEq)]
pub enum OuterEnumType {
/// Collects the Event enums from all pallets.
Event,
/// Collects the Error enums from all pallets.
Error,
}
impl OuterEnumType {
/// The name of the structure this enum represents.
fn struct_name(&self) -> &str {
match self {
OuterEnumType::Event => "RuntimeEvent",
OuterEnumType::Error => "RuntimeError",
}
}
/// The name of the variant (ie `Event` or `Error`).
fn variant_name(&self) -> &str {
match self {
OuterEnumType::Event => "Event",
OuterEnumType::Error => "Error",
}
}
}
impl ToTokens for OuterEnumType {
fn to_tokens(&self, tokens: &mut TokenStream) {
match self {
OuterEnumType::Event => quote!(Event).to_tokens(tokens),
OuterEnumType::Error => quote!(Error).to_tokens(tokens),
}
}
}
/// Create an outer enum that encapsulates all pallets as variants.
///
/// Each variant represents a pallet and contains the corresponding type declared with either:
/// - #[pallet::event] for the [`OuterEnumType::Event`] variant
/// - #[pallet::error] for the [`OuterEnumType::Error`] variant
///
/// The name of the outer enum is prefixed with Runtime, resulting in names like RuntimeEvent
/// or RuntimeError.
///
/// This structure facilitates the decoding process by leveraging the metadata.
///
/// # Example
///
/// The code generate looks like the following for [`OuterEnumType::Event`].
///
/// ```ignore
/// enum RuntimeEvent {
/// #[codec(index = 0)]
/// System(pallet_system::Event),
///
/// #[codec(index = 5)]
/// Balances(pallet_system::Event),
/// }
/// ```
///
/// Notice that the pallet index is preserved using the `#[codec(index = ..)]` attribute.
pub fn expand_outer_enum(
runtime: &Ident,
pallet_decls: &[Pallet],
scrate: &TokenStream,
enum_ty: OuterEnumType,
) -> syn::Result<TokenStream> {
// Stores all pallet variants.
let mut enum_variants = TokenStream::new();
// Generates the enum conversion between the `Runtime` outer enum and the pallet's enum.
let mut enum_conversions = TokenStream::new();
// Specific for events to query via `is_event_part_defined!`.
let mut query_enum_part_macros = Vec::new();
let enum_name_str = enum_ty.variant_name();
let enum_name_ident = Ident::new(enum_ty.struct_name(), Span::call_site());
for pallet_decl in pallet_decls {
let Some(pallet_entry) = pallet_decl.find_part(enum_name_str) else {
continue
};
let path = &pallet_decl.path;
let pallet_name = &pallet_decl.name;
let index = pallet_decl.index;
let instance = pallet_decl.instance.as_ref();
let generics = &pallet_entry.generics;
if instance.is_some() && generics.params.is_empty() {
let msg = format!(
"Instantiable pallet with no generic `{}` cannot \
be constructed: pallet `{}` must have generic `{}`",
enum_name_str, pallet_name, enum_name_str,
);
return Err(syn::Error::new(pallet_name.span(), msg))
}
let part_is_generic = !generics.params.is_empty();
let pallet_enum = match (instance, part_is_generic) {
(Some(inst), true) => quote!(#path::#enum_ty::<#runtime, #path::#inst>),
(Some(inst), false) => quote!(#path::#enum_ty::<#path::#inst>),
(None, true) => quote!(#path::#enum_ty::<#runtime>),
(None, false) => quote!(#path::#enum_ty),
};
enum_variants.extend(expand_enum_variant(
runtime,
pallet_decl,
index,
instance,
generics,
enum_ty,
));
enum_conversions.extend(expand_enum_conversion(
scrate,
pallet_decl,
&pallet_enum,
&enum_name_ident,
));
if enum_ty == OuterEnumType::Event {
query_enum_part_macros.push(quote! {
#path::__substrate_event_check::is_event_part_defined!(#pallet_name);
});
}
}
// Derives specific for the event.
let event_custom_derives =
if enum_ty == OuterEnumType::Event { quote!(Clone, PartialEq, Eq,) } else { quote!() };
// Implementation specific for errors.
let error_custom_impl = generate_error_impl(scrate, enum_ty);
Ok(quote! {
#( #query_enum_part_macros )*
#[derive(
#event_custom_derives
#scrate::codec::Encode,
#scrate::codec::Decode,
#scrate::scale_info::TypeInfo,
#scrate::RuntimeDebug,
)]
#[allow(non_camel_case_types)]
pub enum #enum_name_ident {
#enum_variants
}
#enum_conversions
#error_custom_impl
})
}
fn expand_enum_variant(
runtime: &Ident,
pallet: &Pallet,
index: u8,
instance: Option<&Ident>,
generics: &Generics,
enum_ty: OuterEnumType,
) -> TokenStream {
let path = &pallet.path;
let variant_name = &pallet.name;
let part_is_generic = !generics.params.is_empty();
let attr = pallet.cfg_pattern.iter().fold(TokenStream::new(), |acc, pattern| {
let attr = TokenStream::from_str(&format!("#[cfg({})]", pattern.original()))
.expect("was successfully parsed before; qed");
quote! {
#acc
#attr
}
});
match instance {
Some(inst) if part_is_generic => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::#enum_ty<#runtime, #path::#inst>),
},
Some(inst) => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::#enum_ty<#path::#inst>),
},
None if part_is_generic => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::#enum_ty<#runtime>),
},
None => quote! {
#attr
#[codec(index = #index)]
#variant_name(#path::#enum_ty),
},
}
}
fn expand_enum_conversion(
scrate: &TokenStream,
pallet: &Pallet,
pallet_enum: &TokenStream,
enum_name_ident: &Ident,
) -> TokenStream {
let variant_name = &pallet.name;
let attr = pallet.cfg_pattern.iter().fold(TokenStream::new(), |acc, pattern| {
let attr = TokenStream::from_str(&format!("#[cfg({})]", pattern.original()))
.expect("was successfully parsed before; qed");
quote! {
#acc
#attr
}
});
quote! {
#attr
impl From<#pallet_enum> for #enum_name_ident {
fn from(x: #pallet_enum) -> Self {
#enum_name_ident
::#variant_name(x)
}
}
#attr
impl TryInto<#pallet_enum> for #enum_name_ident {
type Error = ();
fn try_into(self) -> #scrate::sp_std::result::Result<#pallet_enum, Self::Error> {
match self {
Self::#variant_name(evt) => Ok(evt),
_ => Err(()),
}
}
}
}
}
fn generate_error_impl(scrate: &TokenStream, enum_ty: OuterEnumType) -> TokenStream {
// Implementation is specific to `Error`s.
if enum_ty == OuterEnumType::Event {
return quote! {}
}
let enum_name_ident = Ident::new(enum_ty.struct_name(), Span::call_site());
quote! {
impl #enum_name_ident {
/// Optionally convert the `DispatchError` into the `RuntimeError`.
///
/// Returns `Some` if the error matches the `DispatchError::Module` variant, otherwise `None`.
pub fn from_dispatch_error(err: #scrate::sp_runtime::DispatchError) -> Option<Self> {
let #scrate::sp_runtime::DispatchError::Module(module_error) = err else { return None };
let bytes = #scrate::codec::Encode::encode(&module_error);
#scrate::codec::Decode::decode(&mut &bytes[..]).ok()
}
}
}
}
@@ -20,17 +20,69 @@
//! `construct_runtime` implementation is recursive and can generate code which will call itself in
//! order to get all the pallet parts for each pallet.
//!
//! Pallets define their parts (`Call`, `Storage`, ..) either explicitly with the syntax
//! `::{Call, ...}` or implicitly.
//! Pallets can define their parts:
//! - Implicitely: `System: frame_system`
//! - Explicitly: `System: frame_system::{Pallet, Call}`
//!
//! In case a pallet defines its parts implicitly, then the pallet must provide the
//! `tt_default_parts` macro. `construct_runtime` will generate some code which utilizes `tt_call`
//! to call the `tt_default_parts` macro of the pallet. `tt_default_parts` will then return the
//! default pallet parts as input tokens to the `match_and_replace` macro, which ultimately
//! generates a call to `construct_runtime` again, this time with all the pallet parts explicitly
//! defined.
//! The `construct_runtime` transitions from the implicit definition to the explict one.
//! From the explicit state, Substrate expands the pallets with additional information
//! that is to be included in the runtime metadata. This expansion makes visible some extra
//! parts of the pallets, mainly the `Error` if defined. The expanded state looks like
//! `System: frame_system expanded::{Error} ::{Pallet, Call}` and concatenates the extra expanded
//! parts with the user-provided parts. For example, the `Pallet`, `Call` and `Error` parts are
//! collected.
//!
//! Pallets must provide the `tt_extra_parts` and `tt_default_parts` macros for these transitions.
//! These are automatically implemented by the `#[pallet::pallet]` macro.
//!
//! This macro also generates the following enums for ease of decoding:
//! - `enum RuntimeCall`: This type contains the information needed to decode extrinsics.
//! - `enum RuntimeEvent`: This type contains the information needed to decode events.
//! - `enum RuntimeError`: While this cannot be used directly to decode `sp_runtime::DispatchError`
//! from the chain, it contains the information needed to decode the
//! `sp_runtime::DispatchError::Module`.
//!
//! # State Transitions
//!
//! ```ignore
//! +----------+
//! | Implicit | -----------+
//! +----------+ |
//! | |
//! v v
//! +----------+ +------------------+
//! | Explicit | --> | ExplicitExpanded |
//! +----------+ +------------------+
//! ```
//!
//! When all pallet parts are implcit, then the `construct_runtime!` macro expands to its final
//! state, the `ExplicitExpanded`. Otherwise, all implicit parts are converted to an explicit
//! expanded part allow the `construct_runtime!` to expand any remaining explicit parts to an
//! explicit expanded part.
//!
//! # Implicit to Explicit
//!
//! The `construct_runtime` macro transforms the implicit declaration of each pallet
//! `System: frame_system` to an explicit one `System: frame_system::{Pallet, Call}` using the
//! `tt_default_parts` macro.
//!
//! The `tt_default_parts` macro exposes a comma separated list of pallet parts. For example, the
//! `Event` part is exposed only if the pallet implements an event via `#[pallet::event]` macro.
//! The tokens generated by this macro are ` expanded :: { Pallet, Call }` for our example.
//!
//! The `match_and_insert` macro takes in 3 arguments:
//! - target: This is the `TokenStream` that contains the `construct_runtime!` macro.
//! - pattern: The pattern to match against in the target stream.
//! - tokens: The tokens to added after the pattern match.
//!
//! The `construct_runtime` macro uses the `tt_call` to get the default pallet parts via
//! the `tt_default_parts` macro defined by each pallet. The pallet parts are then returned as
//! input to the `match_and_replace` macro.
//! The `match_and_replace` then will modify the the `construct_runtime!` to expand the implicit
//! definition to the explicit one.
//!
//! For example,
//!
//! E.g.
//! ```ignore
//! construct_runtime!(
//! //...
@@ -106,6 +158,7 @@
//! tokens = [{ ::{Pallet, Call} }]
//! }
//! ```
//!
//! Which will then finally expand to the following:
//! ```ignore
//! construct_runtime!(
@@ -116,6 +169,7 @@
//! }
//! )
//! ```
//!
//! This call has no implicit pallet parts, thus it will expand to the runtime construction:
//! ```ignore
//! pub struct Runtime { ... }
@@ -140,6 +194,19 @@
//! | w/ pallet parts |
//! +--------------------+
//! ```
//!
//! # Explicit to Explicit Expanded
//!
//! Users normally do not care about this transition.
//!
//! Similarly to the previous transition, the macro expansion transforms `System:
//! frame_system::{Pallet, Call}` into `System: frame_system expanded::{Error} ::{Pallet, Call}`.
//! The `expanded` section adds extra parts that the Substrate would like to expose for each pallet
//! by default. This is done to expose the approprite types for metadata construction.
//!
//! This time, instead of calling `tt_default_parts` we are using the `tt_extra_parts` macro.
//! This macro returns the ` :: expanded { Error }` list of additional parts we would like to
//! expose.
mod expand;
mod parse;
@@ -171,9 +238,16 @@ pub fn construct_runtime(input: TokenStream) -> TokenStream {
let res = match definition {
RuntimeDeclaration::Implicit(implicit_def) =>
check_pallet_number(input_copy.clone().into(), implicit_def.pallets.len()).and_then(
|_| construct_runtime_intermediary_expansion(input_copy.into(), implicit_def),
|_| construct_runtime_implicit_to_explicit(input_copy.into(), implicit_def),
),
RuntimeDeclaration::Explicit(explicit_decl) =>
RuntimeDeclaration::Explicit(explicit_decl) => check_pallet_number(
input_copy.clone().into(),
explicit_decl.pallets.len(),
)
.and_then(|_| {
construct_runtime_explicit_to_explicit_expanded(input_copy.into(), explicit_decl)
}),
RuntimeDeclaration::ExplicitExpanded(explicit_decl) =>
check_pallet_number(input_copy.into(), explicit_decl.pallets.len())
.and_then(|_| construct_runtime_final_expansion(explicit_decl)),
};
@@ -189,9 +263,14 @@ pub fn construct_runtime(input: TokenStream) -> TokenStream {
res.into()
}
/// When some pallet have implicit parts definition then the macro will expand into a macro call to
/// `construct_runtime_args` of each pallets, see root documentation.
fn construct_runtime_intermediary_expansion(
/// All pallets that have implicit pallet parts (ie `System: frame_system`) are
/// expanded with the default parts defined by the pallet's `tt_default_parts` macro.
///
/// This function transforms the [`RuntimeDeclaration::Implicit`] into
/// [`RuntimeDeclaration::Explicit`] that is not yet fully expanded.
///
/// For more details, please refer to the root documentation.
fn construct_runtime_implicit_to_explicit(
input: TokenStream2,
definition: ImplicitRuntimeDeclaration,
) -> Result<TokenStream2> {
@@ -218,6 +297,42 @@ fn construct_runtime_intermediary_expansion(
Ok(expansion)
}
/// All pallets that have
/// (I): explicit pallet parts (ie `System: frame_system::{Pallet, Call}`) and
/// (II): are not fully expanded (ie do not include the `Error` expansion part)
/// are fully expanded by including the parts from the pallet's `tt_extra_parts` macro.
///
/// This function transforms the [`RuntimeDeclaration::Explicit`] that is not yet fully expanded
/// into [`RuntimeDeclaration::ExplicitExpanded`] fully expanded.
///
/// For more details, please refer to the root documentation.
fn construct_runtime_explicit_to_explicit_expanded(
input: TokenStream2,
definition: ExplicitRuntimeDeclaration,
) -> Result<TokenStream2> {
let frame_support = generate_crate_access_2018("frame-support")?;
let mut expansion = quote::quote!(
#frame_support::construct_runtime! { #input }
);
for pallet in definition.pallets.iter().filter(|pallet| !pallet.is_expanded) {
let pallet_path = &pallet.path;
let pallet_name = &pallet.name;
let pallet_instance = pallet.instance.as_ref().map(|instance| quote::quote!(::<#instance>));
expansion = quote::quote!(
#frame_support::tt_call! {
macro = [{ #pallet_path::tt_extra_parts }]
frame_support = [{ #frame_support }]
~~> #frame_support::match_and_insert! {
target = [{ #expansion }]
pattern = [{ #pallet_name: #pallet_path #pallet_instance }]
}
}
);
}
Ok(expansion)
}
/// All pallets have explicit definition of parts, this will expand to the runtime declaration.
fn construct_runtime_final_expansion(
definition: ExplicitRuntimeDeclaration,
@@ -264,14 +379,23 @@ fn construct_runtime_final_expansion(
let scrate = generate_crate_access(hidden_crate_name, "frame-support");
let scrate_decl = generate_hidden_includes(hidden_crate_name, "frame-support");
let outer_event = expand::expand_outer_event(&name, &pallets, &scrate)?;
let outer_event =
expand::expand_outer_enum(&name, &pallets, &scrate, expand::OuterEnumType::Event)?;
let outer_error =
expand::expand_outer_enum(&name, &pallets, &scrate, expand::OuterEnumType::Error)?;
let outer_origin = expand::expand_outer_origin(&name, system_pallet, &pallets, &scrate)?;
let all_pallets = decl_all_pallets(&name, pallets.iter(), &features);
let pallet_to_index = decl_pallet_runtime_setup(&name, &pallets, &scrate);
let dispatch = expand::expand_outer_dispatch(&name, system_pallet, &pallets, &scrate);
let metadata = expand::expand_runtime_metadata(&name, &pallets, &scrate, &unchecked_extrinsic);
let metadata = expand::expand_runtime_metadata(
&name,
&pallets,
&scrate,
&unchecked_extrinsic,
&system_pallet.path,
);
let outer_config = expand::expand_outer_config(&name, &pallets, &scrate);
let inherent =
expand::expand_outer_inherent(&name, &block, &unchecked_extrinsic, &pallets, &scrate);
@@ -331,6 +455,8 @@ fn construct_runtime_final_expansion(
#outer_event
#outer_error
#outer_origin
#all_pallets
@@ -35,6 +35,7 @@ mod keyword {
syn::custom_keyword!(Call);
syn::custom_keyword!(Storage);
syn::custom_keyword!(Event);
syn::custom_keyword!(Error);
syn::custom_keyword!(Config);
syn::custom_keyword!(Origin);
syn::custom_keyword!(Inherent);
@@ -45,6 +46,7 @@ mod keyword {
syn::custom_keyword!(SlashReason);
syn::custom_keyword!(exclude_parts);
syn::custom_keyword!(use_parts);
syn::custom_keyword!(expanded);
}
/// Declaration of a runtime.
@@ -56,6 +58,7 @@ mod keyword {
pub enum RuntimeDeclaration {
Implicit(ImplicitRuntimeDeclaration),
Explicit(ExplicitRuntimeDeclaration),
ExplicitExpanded(ExplicitRuntimeDeclaration),
}
/// Declaration of a runtime with some pallet with implicit declaration of parts.
@@ -106,6 +109,13 @@ impl Parse for RuntimeDeclaration {
pallets,
pallets_token,
})),
PalletsConversion::ExplicitExpanded(pallets) =>
Ok(RuntimeDeclaration::ExplicitExpanded(ExplicitRuntimeDeclaration {
name,
where_section,
pallets,
pallets_token,
})),
}
}
}
@@ -188,6 +198,8 @@ impl Parse for WhereDefinition {
/// The declaration of a pallet.
#[derive(Debug, Clone)]
pub struct PalletDeclaration {
/// Is this pallet fully expanded?
pub is_expanded: bool,
/// The name of the pallet, e.g.`System` in `System: frame_system`.
pub name: Ident,
/// Optional attributes tagged right above a pallet declaration.
@@ -233,6 +245,7 @@ impl Parse for PalletDeclaration {
let _: Token![>] = input.parse()?;
res
} else if !(input.peek(Token![::]) && input.peek3(token::Brace)) &&
!input.peek(keyword::expanded) &&
!input.peek(keyword::exclude_parts) &&
!input.peek(keyword::use_parts) &&
!input.peek(Token![=]) &&
@@ -246,10 +259,21 @@ impl Parse for PalletDeclaration {
None
};
// Check if the pallet is fully expanded.
let (is_expanded, extra_parts) = if input.peek(keyword::expanded) {
let _: keyword::expanded = input.parse()?;
let _: Token![::] = input.parse()?;
(true, parse_pallet_parts(input)?)
} else {
(false, vec![])
};
// Parse for explicit parts
let pallet_parts = if input.peek(Token![::]) && input.peek3(token::Brace) {
let _: Token![::] = input.parse()?;
Some(parse_pallet_parts(input)?)
let mut parts = parse_pallet_parts(input)?;
parts.extend(extra_parts.into_iter());
Some(parts)
} else if !input.peek(keyword::exclude_parts) &&
!input.peek(keyword::use_parts) &&
!input.peek(Token![=]) &&
@@ -260,7 +284,7 @@ impl Parse for PalletDeclaration {
"Unexpected tokens, expected one of `::{`, `exclude_parts`, `use_parts`, `=`, `,`",
))
} else {
None
is_expanded.then_some(extra_parts)
};
// Parse for specified parts
@@ -288,7 +312,7 @@ impl Parse for PalletDeclaration {
None
};
Ok(Self { attrs, name, path, instance, pallet_parts, specified_parts, index })
Ok(Self { is_expanded, attrs, name, path, instance, pallet_parts, specified_parts, index })
}
}
@@ -371,6 +395,7 @@ pub enum PalletPartKeyword {
Call(keyword::Call),
Storage(keyword::Storage),
Event(keyword::Event),
Error(keyword::Error),
Config(keyword::Config),
Origin(keyword::Origin),
Inherent(keyword::Inherent),
@@ -393,6 +418,8 @@ impl Parse for PalletPartKeyword {
Ok(Self::Storage(input.parse()?))
} else if lookahead.peek(keyword::Event) {
Ok(Self::Event(input.parse()?))
} else if lookahead.peek(keyword::Error) {
Ok(Self::Error(input.parse()?))
} else if lookahead.peek(keyword::Config) {
Ok(Self::Config(input.parse()?))
} else if lookahead.peek(keyword::Origin) {
@@ -423,6 +450,7 @@ impl PalletPartKeyword {
Self::Call(_) => "Call",
Self::Storage(_) => "Storage",
Self::Event(_) => "Event",
Self::Error(_) => "Error",
Self::Config(_) => "Config",
Self::Origin(_) => "Origin",
Self::Inherent(_) => "Inherent",
@@ -441,7 +469,7 @@ impl PalletPartKeyword {
/// Returns the names of all pallet parts that allow to have a generic argument.
fn all_generic_arg() -> &'static [&'static str] {
&["Event", "Origin", "Config"]
&["Event", "Error", "Origin", "Config"]
}
}
@@ -452,6 +480,7 @@ impl ToTokens for PalletPartKeyword {
Self::Call(inner) => inner.to_tokens(tokens),
Self::Storage(inner) => inner.to_tokens(tokens),
Self::Event(inner) => inner.to_tokens(tokens),
Self::Error(inner) => inner.to_tokens(tokens),
Self::Config(inner) => inner.to_tokens(tokens),
Self::Origin(inner) => inner.to_tokens(tokens),
Self::Inherent(inner) => inner.to_tokens(tokens),
@@ -554,6 +583,8 @@ fn parse_pallet_parts_no_generic(input: ParseStream) -> Result<Vec<PalletPartNoG
/// The final definition of a pallet with the resulting fixed index and explicit parts.
#[derive(Debug, Clone)]
pub struct Pallet {
/// Is this pallet fully expanded?
pub is_expanded: bool,
/// The name of the pallet, e.g.`System` in `System: frame_system`.
pub name: Ident,
/// Either automatically infered, or defined (e.g. `MyPallet ... = 3,`).
@@ -586,9 +617,35 @@ impl Pallet {
}
/// Result of a conversion of a declaration of pallets.
///
/// # State Transitions
///
/// ```ignore
/// +----------+ +----------+ +------------------+
/// | Implicit | -> | Explicit | -> | ExplicitExpanded |
/// +----------+ +----------+ +------------------+
/// ```
enum PalletsConversion {
/// Pallets implicitely declare parts.
///
/// `System: frame_system`.
Implicit(Vec<PalletDeclaration>),
/// Pallets explicitly declare parts.
///
/// `System: frame_system::{Pallet, Call}`
///
/// However, for backwards compatibility with Polkadot/Kusama
/// we must propagate some other parts to the pallet by default.
Explicit(Vec<Pallet>),
/// Pallets explicitly declare parts that are fully expanded.
///
/// This is the end state that contains extra parts included by
/// default by Subtrate.
///
/// `System: frame_system expanded::{Error} ::{Pallet, Call}`
///
/// For this example, the `Pallet`, `Call` and `Error` parts are collected.
ExplicitExpanded(Vec<Pallet>),
}
/// Convert from the parsed pallet declaration to their final information.
@@ -604,6 +661,7 @@ fn convert_pallets(pallets: Vec<PalletDeclaration>) -> syn::Result<PalletsConver
let mut indices = HashMap::new();
let mut last_index: Option<u8> = None;
let mut names = HashMap::new();
let mut is_expanded = true;
let pallets = pallets
.into_iter()
@@ -698,7 +756,10 @@ fn convert_pallets(pallets: Vec<PalletDeclaration>) -> syn::Result<PalletsConver
})
.collect::<Result<Vec<_>>>()?;
is_expanded &= pallet.is_expanded;
Ok(Pallet {
is_expanded: pallet.is_expanded,
name: pallet.name,
index: final_index,
path: pallet.path,
@@ -709,5 +770,9 @@ fn convert_pallets(pallets: Vec<PalletDeclaration>) -> syn::Result<PalletsConver
})
.collect::<Result<Vec<_>>>()?;
Ok(PalletsConversion::Explicit(pallets))
if is_expanded {
Ok(PalletsConversion::ExplicitExpanded(pallets))
} else {
Ok(PalletsConversion::Explicit(pallets))
}
}
@@ -26,6 +26,8 @@ pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
let count = COUNTER.with(|counter| counter.borrow_mut().inc());
let default_parts_unique_id =
syn::Ident::new(&format!("__tt_default_parts_{}", count), def.item.span());
let extra_parts_unique_id =
syn::Ident::new(&format!("__tt_extra_parts_{}", count), def.item.span());
let call_part = def.call.as_ref().map(|_| quote::quote!(Call,));
@@ -36,6 +38,8 @@ pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
quote::quote!( Event #gen , )
});
let error_part = def.error.as_ref().map(|_| quote::quote!(Error<T>,));
let origin_part = def.origin.as_ref().map(|origin| {
let gen = origin.is_generic.then(|| quote::quote!( <T> ));
quote::quote!( Origin #gen , )
@@ -95,8 +99,8 @@ pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
$($frame_support)*::tt_return! {
$caller
tokens = [{
::{
Pallet, #call_part #storage_part #event_part #origin_part #config_part
expanded::{
Pallet, #call_part #storage_part #event_part #error_part #origin_part #config_part
#inherent_part #validate_unsigned_part #freeze_reason_part
#hold_reason_part #lock_id_part #slash_reason_part
}
@@ -106,5 +110,33 @@ pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
}
pub use #default_parts_unique_id as tt_default_parts;
// This macro is similar to the `tt_default_parts!`. It expands the pallets thare are declared
// explicitly (`System: frame_system::{Pallet, Call}`) with extra parts.
//
// For example, after expansion an explicit pallet would look like:
// `System: expanded::{Error} ::{Pallet, Call}`.
//
// The `expanded` keyword is a marker of the final state of the `construct_runtime!`.
#[macro_export]
#[doc(hidden)]
macro_rules! #extra_parts_unique_id {
{
$caller:tt
frame_support = [{ $($frame_support:ident)::* }]
} => {
$($frame_support)*::tt_return! {
$caller
tokens = [{
expanded::{
#error_part
}
}]
}
};
}
pub use #extra_parts_unique_id as tt_extra_parts;
)
}