pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-04-27 05:47:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

Automatic pallet parts in construct_runtime (#9681)

Browse Source 
* implement automatic parts

* ui tests

* rename

* remove unnecessary exclude

* better doc

* better doc

* fix genesis config

* fix UI tests

* fix UI test

* Revert "fix UI test"

This reverts commit a910351c0b24cfe42195cfd97d83a416640e3259.

* implemented used_parts

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* doc + fmt

* Update frame/support/procedural/src/construct_runtime/parse.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* add doc in the macro

* remove yet some more parts

* fix ui test

* more determnistic error message + fix ui tests

* fix ui test

* Apply suggestions from code review

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* do refactor + fix ui tests

* fmt

* fix test

* fix test

* fix ui test

* Apply suggestions from code review

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* refactor

* remove even more part in node-runtime

* fix test

* Add flow chart for the construct_runtime! execution flow

* Fix typo

* Ignore snippets that don't contain code

* Refactor some code in expand_after

* Rename expand_after to match_and_insert

* cargo fmt

* Fix rename

* Remove frame_support argument to construct_runtime_parts

* Make use of tt-call to simplify intermediate expansions

* cargo fmt

* Update match_and_insert documentation

* Reset cursor to 0 when no matching patterns are found

* Reorder struct fields on MatchAndInsertDef

* Add test for dependency renames and fix frame-support import

* Add more doc comments

* Update frame/support/test/compile_pass/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
This commit is contained in:
Guillaume Thiolliere
2021-10-31 14:55:10 +01:00
committed by GitHub
parent 0214fde9a6
commit 4292e18e50
31 changed files with 1340 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files
substrate/frame/support/procedural/src/construct_runtime/mod.rs
+181 -91
View File
@@ -15,114 +15,204 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//! Implementation of `construct_runtime`.
//!
//! `construct_runtime` implementation is recursive and can generate code which will call itself in
//! order to get all the pallet parts for each pallet.
//!
//! Pallets define their parts (`Call`, `Storage`, ..) either explicitly with the syntax
//! `::{Call, ...}` or implicitly.
//!
//! In case a pallet defines its parts implicitly, then the pallet must provide the
//! `tt_default_parts` macro. `construct_rutime` will generate some code which utilizes `tt_call`
//! to call the `tt_default_parts` macro of the pallet. `tt_default_parts` will then return the
//! default pallet parts as input tokens to the `match_and_replace` macro, which ultimately
//! generates a call to `construct_runtime` again, this time with all the pallet parts explicitly
//! defined.
//!
//! E.g.
//! ```ignore
//! construct_runtime!(
//! //...
//! {
//! System: frame_system = 0, // Implicit definition of parts
//! Balances: pallet_balances = 1, // Implicit definition of parts
//! }
//! );
//! ```
//! This call has some implicit pallet parts, thus it will expand to:
//! ```ignore
//! frame_support::tt_call! {
//! macro = [{ pallet_balances::tt_default_parts }]
//! ~~> frame_support::match_and_insert! {
//! target = [{
//! frame_support::tt_call! {
//! macro = [{ frame_system::tt_default_parts }]
//! ~~> frame_support::match_and_insert! {
//! target = [{
//! construct_runtime!(
//! //...
//! {
//! System: frame_system = 0,
//! Balances: pallet_balances = 1,
//! }
//! );
//! }]
//! pattern = [{ System: frame_system }]
//! }
//! }
//! }]
//! pattern = [{ Balances: pallet_balances }]
//! }
//! }
//! ```
//! `tt_default_parts` must be defined. It returns the pallet parts inside some tokens, and
//! then `tt_call` will pipe the returned pallet parts into the input of `match_and_insert`.
//! Thus `match_and_insert` will initially receive the following inputs:
//! ```ignore
//! frame_support::match_and_insert! {
//! target = [{
//! frame_support::match_and_insert! {
//! target = [{
//! construct_runtime!(
//! //...
//! {
//! System: frame_system = 0,
//! Balances: pallet_balances = 1,
//! }
//! )
//! }]
//! pattern = [{ System: frame_system }]
//! tokens = [{ ::{Pallet, Call} }]
//! }]
//! pattern = [{ Balances: pallet_balances }]
//! tokens = [{ ::{Pallet, Call} }]
//! }
//! ```
//! After dealing with `pallet_balances`, the inner `match_and_insert` will expand to:
//! ```ignore
//! frame_support::match_and_insert! {
//! target = [{
//! construct_runtime!(
//! //...
//! {
//! System: frame_system = 0, // Implicit definition of parts
//! Balances: pallet_balances::{Pallet, Call} = 1, // Explicit definition of parts
//! }
//! )
//! }]
//! pattern = [{ System: frame_system }]
//! tokens = [{ ::{Pallet, Call} }]
//! }
//! ```
//! Which will then finally expand to the following:
//! ```ignore
//! construct_runtime!(
//! //...
//! {
//! System: frame_system::{Pallet, Call},
//! Balances: pallet_balances::{Pallet, Call},
//! }
//! )
//! ```
//! This call has no implicit pallet parts, thus it will expand to the runtime construction:
//! ```ignore
//! pub struct Runtime { ... }
//! pub struct Call { ... }
//! impl Call ...
//! pub enum Origin { ... }
//! ...
//! ```
//!
//! Visualizing the entire flow of `construct_runtime!`, it would look like the following:
//!
//! ```ignore
//! +--------------------+ +---------------------+ +-------------------+
//! | | | (defined in pallet) | | |
//! | construct_runtime! | --> | tt_default_parts! | --> | match_and_insert! |
//! | w/ no pallet parts | | | | |
//! +--------------------+ +---------------------+ +-------------------+
//!
//! +--------------------+
//! | |
//! --> | construct_runtime! |
//! | w/ pallet parts |
//! +--------------------+
//! ```
mod expand;
mod parse;
use frame_support_procedural_tools::{
generate_crate_access, generate_hidden_includes, syn_ext as ext,
generate_crate_access, generate_crate_access_2018, generate_hidden_includes,
};
use parse::{
ExplicitRuntimeDeclaration, ImplicitRuntimeDeclaration, Pallet, RuntimeDeclaration,
WhereSection,
};
use parse::{PalletDeclaration, PalletPart, PalletPath, RuntimeDefinition, WhereSection};
use proc_macro::TokenStream;
use proc_macro2::TokenStream as TokenStream2;
use quote::quote;
use std::collections::HashMap;
use syn::{Ident, Result};
/// The fixed name of the system pallet.
const SYSTEM_PALLET_NAME: &str = "System";
/// The complete definition of a pallet with the resulting fixed index.
#[derive(Debug, Clone)]
pub struct Pallet {
pub name: Ident,
pub index: u8,
pub path: PalletPath,
pub instance: Option<Ident>,
pub pallet_parts: Vec<PalletPart>,
}
impl Pallet {
/// Get resolved pallet parts
fn pallet_parts(&self) -> &[PalletPart] {
&self.pallet_parts
}
/// Find matching parts
fn find_part(&self, name: &str) -> Option<&PalletPart> {
self.pallet_parts.iter().find(|part| part.name() == name)
}
/// Return whether pallet contains part
fn exists_part(&self, name: &str) -> bool {
self.find_part(name).is_some()
}
}
/// Convert from the parsed pallet to their final information.
/// Assign index to each pallet using same rules as rust for fieldless enum.
/// I.e. implicit are assigned number incrementedly from last explicit or 0.
fn complete_pallets(decl: impl Iterator<Item = PalletDeclaration>) -> syn::Result<Vec<Pallet>> {
let mut indices = HashMap::new();
let mut last_index: Option<u8> = None;
let mut names = HashMap::new();
decl.map(|pallet| {
let final_index = match pallet.index {
Some(i) => i,
None => last_index.map_or(Some(0), |i| i.checked_add(1)).ok_or_else(|| {
let msg = "Pallet index doesn't fit into u8, index is 256";
syn::Error::new(pallet.name.span(), msg)
})?,
};
last_index = Some(final_index);
if let Some(used_pallet) = indices.insert(final_index, pallet.name.clone()) {
let msg = format!(
"Pallet indices are conflicting: Both pallets {} and {} are at index {}",
used_pallet, pallet.name, final_index,
);
let mut err = syn::Error::new(used_pallet.span(), &msg);
err.combine(syn::Error::new(pallet.name.span(), msg));
return Err(err)
}
if let Some(used_pallet) = names.insert(pallet.name.clone(), pallet.name.span()) {
let msg = "Two pallets with the same name!";
let mut err = syn::Error::new(used_pallet, &msg);
err.combine(syn::Error::new(pallet.name.span(), &msg));
return Err(err)
}
Ok(Pallet {
name: pallet.name,
index: final_index,
path: pallet.path,
instance: pallet.instance,
pallet_parts: pallet.pallet_parts,
})
})
.collect()
}
/// Implementation of `construct_runtime` macro. Either expand to some code which will call
/// `construct_runtime` again, or expand to the final runtime definition.
pub fn construct_runtime(input: TokenStream) -> TokenStream {
let definition = syn::parse_macro_input!(input as RuntimeDefinition);
construct_runtime_parsed(definition)
.unwrap_or_else(|e| e.to_compile_error())
.into()
let input_copy = input.clone();
let definition = syn::parse_macro_input!(input as RuntimeDeclaration);
let res = match definition {
RuntimeDeclaration::Implicit(implicit_def) =>
construct_runtime_intermediary_expansion(input_copy.into(), implicit_def),
RuntimeDeclaration::Explicit(explicit_decl) =>
construct_runtime_final_expansion(explicit_decl),
};
res.unwrap_or_else(|e| e.to_compile_error()).into()
}
fn construct_runtime_parsed(definition: RuntimeDefinition) -> Result<TokenStream2> {
let RuntimeDefinition {
name,
where_section: WhereSection { block, node_block, unchecked_extrinsic, .. },
pallets:
ext::Braces { content: ext::Punctuated { inner: pallets, .. }, token: pallets_token },
..
} = definition;
/// When some pallet have implicit parts definition then the macro will expand into a macro call to
/// `construct_runtime_args` of each pallets, see root documentation.
fn construct_runtime_intermediary_expansion(
input: TokenStream2,
definition: ImplicitRuntimeDeclaration,
) -> Result<TokenStream2> {
let frame_support = generate_crate_access_2018("frame-support")?;
let mut expansion = quote::quote!(
#frame_support::construct_runtime! { #input }
);
for pallet in definition.pallets.iter().filter(|pallet| pallet.pallet_parts.is_none()) {
let pallet_path = &pallet.path;
let pallet_name = &pallet.name;
let pallet_instance = pallet.instance.as_ref().map(|instance| quote::quote!(::<#instance>));
expansion = quote::quote!(
#frame_support::tt_call! {
macro = [{ #pallet_path::tt_default_parts }]
frame_support = [{ #frame_support }]
~~> #frame_support::match_and_insert! {
target = [{ #expansion }]
pattern = [{ #pallet_name: #pallet_path #pallet_instance }]
}
}
);
}
let pallets = complete_pallets(pallets.into_iter())?;
Ok(expansion.into())
}
/// All pallets have explicit definition of parts, this will expand to the runtime declaration.
fn construct_runtime_final_expansion(
definition: ExplicitRuntimeDeclaration,
) -> Result<TokenStream2> {
let ExplicitRuntimeDeclaration {
name,
where_section: WhereSection { block, node_block, unchecked_extrinsic },
pallets,
pallets_token,
} = definition;
let hidden_crate_name = "construct_runtime";
let scrate = generate_crate_access(&hidden_crate_name, "frame-support");
substrate/frame/support/procedural/src/construct_runtime/parse.rs
+301 -40
View File
@@ -17,13 +17,13 @@
use frame_support_procedural_tools::syn_ext as ext;
use proc_macro2::{Span, TokenStream};
use std::collections::HashSet;
use std::collections::{HashMap, HashSet};
use syn::{
ext::IdentExt,
parse::{Parse, ParseStream},
punctuated::Punctuated,
spanned::Spanned,
token, Error, Ident, Path, PathArguments, PathSegment, Result, Token,
token, Error, Ident, Path, Result, Token,
};
mod keyword {
@@ -38,26 +38,63 @@ mod keyword {
syn::custom_keyword!(Origin);
syn::custom_keyword!(Inherent);
syn::custom_keyword!(ValidateUnsigned);
syn::custom_keyword!(exclude_parts);
syn::custom_keyword!(use_parts);
}
/// Declaration of a runtime.
///
/// Pallet declare their part either explicitly or implicitly (using no part declaration)
/// If all pallet have explicit parts then the runtime declaration is explicit, otherwise it is
/// implicit.
#[derive(Debug)]
pub struct RuntimeDefinition {
pub visibility_token: Token![pub],
pub enum_token: Token![enum],
pub enum RuntimeDeclaration {
Implicit(ImplicitRuntimeDeclaration),
Explicit(ExplicitRuntimeDeclaration),
}
/// Declaration of a runtime with some pallet with implicit declaration of parts.
#[derive(Debug)]
pub struct ImplicitRuntimeDeclaration {
pub name: Ident,
pub where_section: WhereSection,
pub pallets: ext::Braces<ext::Punctuated<PalletDeclaration, Token![,]>>,
pub pallets: Vec<PalletDeclaration>,
}
impl Parse for RuntimeDefinition {
/// Declaration of a runtime with all pallet having explicit declaration of parts.
#[derive(Debug)]
pub struct ExplicitRuntimeDeclaration {
pub name: Ident,
pub where_section: WhereSection,
pub pallets: Vec<Pallet>,
pub pallets_token: token::Brace,
}
impl Parse for RuntimeDeclaration {
fn parse(input: ParseStream) -> Result<Self> {
Ok(Self {
visibility_token: input.parse()?,
enum_token: input.parse()?,
name: input.parse()?,
where_section: input.parse()?,
pallets: input.parse()?,
})
input.parse::<Token![pub]>()?;
input.parse::<Token![enum]>()?;
let name = input.parse::<syn::Ident>()?;
let where_section = input.parse()?;
let pallets =
input.parse::<ext::Braces<ext::Punctuated<PalletDeclaration, Token![,]>>>()?;
let pallets_token = pallets.token;
match convert_pallets(pallets.content.inner.into_iter().collect())? {
PalletsConversion::Implicit(pallets) =>
Ok(RuntimeDeclaration::Implicit(ImplicitRuntimeDeclaration {
name,
where_section,
pallets,
})),
PalletsConversion::Explicit(pallets) =>
Ok(RuntimeDeclaration::Explicit(ExplicitRuntimeDeclaration {
name,
where_section,
pallets,
pallets_token,
})),
}
}
}
@@ -136,14 +173,34 @@ impl Parse for WhereDefinition {
}
}
/// The declaration of a pallet.
#[derive(Debug, Clone)]
pub struct PalletDeclaration {
/// The name of the pallet, e.g.`System` in `System: frame_system`.
pub name: Ident,
/// Optional fixed index (e.g. `MyPallet ... = 3,`)
/// Optional fixed index, e.g. `MyPallet ... = 3,`.
pub index: Option<u8>,
/// The path of the pallet, e.g. `frame_system` in `System: frame_system`.
pub path: PalletPath,
/// The instance of the pallet, e.g. `Instance1` in `Council: pallet_collective::<Instance1>`.
pub instance: Option<Ident>,
pub pallet_parts: Vec<PalletPart>,
/// The declared pallet parts,
/// e.g. `Some([Pallet, Call])` for `System: system::{Pallet, Call}`
/// or `None` for `System: system`.
pub pallet_parts: Option<Vec<PalletPart>>,
/// The specified parts, either use_parts or exclude_parts.
pub specified_parts: SpecifiedParts,
}
/// The possible declaration of pallet parts to use.
#[derive(Debug, Clone)]
pub enum SpecifiedParts {
/// Use all the pallet parts except those specified.
Exclude(Vec<PalletPartNoGeneric>),
/// Use only the specified pallet parts.
Use(Vec<PalletPartNoGeneric>),
/// Use the all the pallet parts.
All,
}
impl Parse for PalletDeclaration {
@@ -151,38 +208,78 @@ impl Parse for PalletDeclaration {
let name = input.parse()?;
let _: Token![:] = input.parse()?;
let path = input.parse()?;
let instance = if input.peek(Token![<]) {
// Parse for instance.
let instance = if input.peek(Token![::]) && input.peek3(Token![<]) {
let _: Token![::] = input.parse()?;
let _: Token![<] = input.parse()?;
let res = Some(input.parse()?);
let _: Token![>] = input.parse()?;
let _: Token![::] = input.parse()?;
res
} else if !(input.peek(Token![::]) && input.peek3(token::Brace)) &&
!input.peek(keyword::exclude_parts) &&
!input.peek(keyword::use_parts) &&
!input.peek(Token![=]) &&
!input.peek(Token![,]) &&
!input.is_empty()
{
return Err(input.error(
"Unexpected tokens, expected one of `::$ident` `::{`, `exclude_parts`, `use_parts`, `=`, `,`",
))
} else {
None
};
let pallet_parts = parse_pallet_parts(input)?;
// Parse for explicit parts
let pallet_parts = if input.peek(Token![::]) && input.peek3(token::Brace) {
let _: Token![::] = input.parse()?;
Some(parse_pallet_parts(input)?)
} else if !input.peek(keyword::exclude_parts) &&
!input.peek(keyword::use_parts) &&
!input.peek(Token![=]) &&
!input.peek(Token![,]) &&
!input.is_empty()
{
return Err(input.error(
"Unexpected tokens, expected one of `::{`, `exclude_parts`, `use_parts`, `=`, `,`",
))
} else {
None
};
// Parse for specified parts
let specified_parts = if input.peek(keyword::exclude_parts) {
let _: keyword::exclude_parts = input.parse()?;
SpecifiedParts::Exclude(parse_pallet_parts_no_generic(input)?)
} else if input.peek(keyword::use_parts) {
let _: keyword::use_parts = input.parse()?;
SpecifiedParts::Use(parse_pallet_parts_no_generic(input)?)
} else if !input.peek(Token![=]) && !input.peek(Token![,]) && !input.is_empty() {
return Err(input.error("Unexpected tokens, expected one of `exclude_parts`, `=`, `,`"))
} else {
SpecifiedParts::All
};
// Parse for pallet index
let index = if input.peek(Token![=]) {
input.parse::<Token![=]>()?;
let index = input.parse::<syn::LitInt>()?;
let index = index.base10_parse::<u8>()?;
Some(index)
} else if !input.peek(Token![,]) && !input.is_empty() {
return Err(input.error("Unexpected tokens, expected one of `=`, `,`"))
} else {
None
};
let parsed = Self { name, path, instance, pallet_parts, index };
Ok(parsed)
Ok(Self { name, path, instance, pallet_parts, specified_parts, index })
}
}
/// A struct representing a path to a pallet. `PalletPath` is almost identical to the standard
/// Rust path with a few restrictions:
/// - No leading colons allowed
/// - Path segments can only consist of identifers; angle-bracketed or parenthesized segments will
/// result in a parsing error (except when specifying instances)
/// - Path segments can only consist of identifers separated by colons
#[derive(Debug, Clone)]
pub struct PalletPath {
pub inner: Path,
@@ -202,34 +299,27 @@ impl PalletPath {
impl Parse for PalletPath {
fn parse(input: ParseStream) -> Result<Self> {
let mut lookahead = input.lookahead1();
let mut segments = Punctuated::new();
let mut res =
PalletPath { inner: Path { leading_colon: None, segments: Punctuated::new() } };
let lookahead = input.lookahead1();
if lookahead.peek(Token![crate]) ||
lookahead.peek(Token![self]) ||
lookahead.peek(Token![super]) ||
lookahead.peek(Ident)
{
let ident = input.call(Ident::parse_any)?;
segments.push(PathSegment { ident, arguments: PathArguments::None });
let _: Token![::] = input.parse()?;
lookahead = input.lookahead1();
res.inner.segments.push(ident.into());
} else {
return Err(lookahead.error())
}
while lookahead.peek(Ident) {
let ident = input.parse()?;
segments.push(PathSegment { ident, arguments: PathArguments::None });
let _: Token![::] = input.parse()?;
lookahead = input.lookahead1();
while input.peek(Token![::]) && input.peek3(Ident) {
input.parse::<Token![::]>()?;
let ident = input.parse::<Ident>()?;
res.inner.segments.push(ident.into());
}
if !lookahead.peek(token::Brace) && !lookahead.peek(Token![<]) {
return Err(lookahead.error())
}
Ok(Self { inner: Path { leading_colon: None, segments } })
Ok(res)
}
}
@@ -391,3 +481,174 @@ fn remove_kind(
Err(input.error(msg))
}
}
/// The declaration of a part without its generics
#[derive(Debug, Clone)]
pub struct PalletPartNoGeneric {
keyword: PalletPartKeyword,
}
impl Parse for PalletPartNoGeneric {
fn parse(input: ParseStream) -> Result<Self> {
Ok(Self { keyword: input.parse()? })
}
}
/// Parse [`PalletPartNoGeneric`]'s from a braces enclosed list that is split by commas, e.g.
///
/// `{ Call, Event }`
fn parse_pallet_parts_no_generic(input: ParseStream) -> Result<Vec<PalletPartNoGeneric>> {
let pallet_parts: ext::Braces<ext::Punctuated<PalletPartNoGeneric, Token![,]>> =
input.parse()?;
let mut resolved = HashSet::new();
for part in pallet_parts.content.inner.iter() {
if !resolved.insert(part.keyword.name()) {
let msg = format!(
"`{}` was already declared before. Please remove the duplicate declaration",
part.keyword.name(),
);
return Err(Error::new(part.keyword.span(), msg))
}
}
Ok(pallet_parts.content.inner.into_iter().collect())
}
/// The final definition of a pallet with the resulting fixed index and explicit parts.
#[derive(Debug, Clone)]
pub struct Pallet {
/// The name of the pallet, e.g.`System` in `System: frame_system`.
pub name: Ident,
/// Either automatically infered, or defined (e.g. `MyPallet ... = 3,`).
pub index: u8,
/// The path of the pallet, e.g. `frame_system` in `System: frame_system`.
pub path: PalletPath,
/// The instance of the pallet, e.g. `Instance1` in `Council: pallet_collective::<Instance1>`.
pub instance: Option<Ident>,
/// The pallet parts to use for the pallet.
pub pallet_parts: Vec<PalletPart>,
}
impl Pallet {
/// Get resolved pallet parts
pub fn pallet_parts(&self) -> &[PalletPart] {
&self.pallet_parts
}
/// Find matching parts
pub fn find_part(&self, name: &str) -> Option<&PalletPart> {
self.pallet_parts.iter().find(|part| part.name() == name)
}
/// Return whether pallet contains part
pub fn exists_part(&self, name: &str) -> bool {
self.find_part(name).is_some()
}
}
/// Result of a conversion of a declaration of pallets.
enum PalletsConversion {
Implicit(Vec<PalletDeclaration>),
Explicit(Vec<Pallet>),
}
/// Convert from the parsed pallet declaration to their final information.
///
/// Check if all pallet have explicit declaration of their parts, if so then assign index to each
/// pallet using same rules as rust for fieldless enum. I.e. implicit are assigned number
/// incrementedly from last explicit or 0.
fn convert_pallets(pallets: Vec<PalletDeclaration>) -> syn::Result<PalletsConversion> {
if pallets.iter().any(|pallet| pallet.pallet_parts.is_none()) {
return Ok(PalletsConversion::Implicit(pallets))
}
let mut indices = HashMap::new();
let mut last_index: Option<u8> = None;
let mut names = HashMap::new();
let pallets = pallets
.into_iter()
.map(|pallet| {
let final_index = match pallet.index {
Some(i) => i,
None => last_index.map_or(Some(0), |i| i.checked_add(1)).ok_or_else(|| {
let msg = "Pallet index doesn't fit into u8, index is 256";
syn::Error::new(pallet.name.span(), msg)
})?,
};
last_index = Some(final_index);
if let Some(used_pallet) = indices.insert(final_index, pallet.name.clone()) {
let msg = format!(
"Pallet indices are conflicting: Both pallets {} and {} are at index {}",
used_pallet, pallet.name, final_index,
);
let mut err = syn::Error::new(used_pallet.span(), &msg);
err.combine(syn::Error::new(pallet.name.span(), msg));
return Err(err)
}
if let Some(used_pallet) = names.insert(pallet.name.clone(), pallet.name.span()) {
let msg = "Two pallets with the same name!";
let mut err = syn::Error::new(used_pallet, &msg);
err.combine(syn::Error::new(pallet.name.span(), &msg));
return Err(err)
}
let mut pallet_parts = pallet.pallet_parts.expect("Checked above");
let available_parts =
pallet_parts.iter().map(|part| part.keyword.name()).collect::<HashSet<_>>();
// Check parts are correctly specified
match &pallet.specified_parts {
SpecifiedParts::Exclude(parts) | SpecifiedParts::Use(parts) =>
for part in parts {
if !available_parts.contains(part.keyword.name()) {
let msg = format!(
"Invalid pallet part specified, the pallet `{}` doesn't have the \
`{}` part. Available parts are: {}.",
pallet.name,
part.keyword.name(),
pallet_parts.iter().fold(String::new(), |fold, part| {
if fold.is_empty() {
format!("`{}`", part.keyword.name())
} else {
format!("{}, `{}`", fold, part.keyword.name())
}
})
);
return Err(syn::Error::new(part.keyword.span(), msg))
}
},
SpecifiedParts::All => (),
}
// Set only specified parts.
match pallet.specified_parts {
SpecifiedParts::Exclude(excluded_parts) => pallet_parts.retain(|part| {
!excluded_parts
.iter()
.any(|excluded_part| excluded_part.keyword.name() == part.keyword.name())
}),
SpecifiedParts::Use(used_parts) => pallet_parts.retain(|part| {
used_parts.iter().any(|use_part| use_part.keyword.name() == part.keyword.name())
}),
SpecifiedParts::All => (),
}
Ok(Pallet {
name: pallet.name,
index: final_index,
path: pallet.path,
instance: pallet.instance,
pallet_parts,
})
})
.collect::<Result<Vec<_>>>()?;
Ok(PalletsConversion::Explicit(pallets))
}
substrate/frame/support/procedural/src/lib.rs
+96 -32
View File
@@ -26,6 +26,7 @@ mod debug_no_bound;
mod default_no_bound;
mod dummy_part_checker;
mod key_prefix;
mod match_and_insert;
mod pallet;
mod partial_eq_no_bound;
mod storage;
@@ -297,52 +298,91 @@ pub fn decl_storage(input: TokenStream) -> TokenStream {
///
/// # Example:
///
/// ```nocompile
/// ```ignore
/// construct_runtime!(
/// pub enum Runtime where
/// Block = Block,
/// NodeBlock = node::Block,
/// UncheckedExtrinsic = UncheckedExtrinsic
/// {
/// System: system::{Pallet, Call, Event<T>, Config<T>} = 0,
/// Test: test::{Pallet, Call} = 1,
/// Test2: test_with_long_module::{Pallet, Event<T>},
/// System: frame_system::{Pallet, Call, Event<T>, Config<T>} = 0,
/// Test: path::to::test::{Pallet, Call} = 1,
///
/// // Pallets with instances
/// Test3_Instance1: test3::<Instance1>::{Pallet, Call, Storage, Event<T, I>, Config<T, I>, Origin<T, I>},
/// Test3_DefaultInstance: test3::{Pallet, Call, Storage, Event<T>, Config<T>, Origin<T>} = 4,
/// Test2_Instance1: test2::<Instance1>::{Pallet, Call, Storage, Event<T, I>, Config<T, I>, Origin<T, I>},
/// Test2_DefaultInstance: test2::{Pallet, Call, Storage, Event<T>, Config<T>, Origin<T>} = 4,
///
/// // Pallets declared with `pallet` attribute macro: no need to define the parts
/// Test3_Instance1: test3::<Instance1>,
/// Test3_DefaultInstance: test3,
///
/// // with `exclude_parts` keyword some part can be excluded.
/// Test4_Instance1: test4::<Instance1> exclude_parts { Call, Origin },
/// Test4_DefaultInstance: test4 exclude_parts { Storage },
///
/// // with `use_parts` keyword, a subset of the pallet parts can be specified.
/// Test4_Instance1: test4::<Instance1> use_parts { Pallet, Call},
/// Test4_DefaultInstance: test4 use_parts { Pallet },
/// }
/// )
/// ```
///
/// The identifier `System` is the name of the pallet and the lower case identifier `system` is the
/// name of the Rust module/crate for this Substrate pallet. The identifiers between the braces are
/// the pallet parts provided by the pallet. It is important to list these parts here to export
/// them correctly in the metadata or to make the pallet usable in the runtime.
/// Each pallet is declared as such:
/// * `Identifier`: name given to the pallet that uniquely identifies it.
///
/// We provide support for the following module parts in a pallet:
/// * `:`: colon separator
///
/// - `Pallet` - Required for all pallets
/// - `Call` - If the pallet has callable functions
/// - `Storage` - If the pallet uses storage
/// - `Event` or `Event<T>` (if the event is generic) - If the pallet emits events
/// - `Origin` or `Origin<T>` (if the origin is generic) - If the pallet has instanciable origins
/// - `Config` or `Config<T>` (if the config is generic) - If the pallet builds the genesis storage
/// with `GenesisConfig`
/// - `Inherent` - If the pallet provides/can check inherents.
/// - `ValidateUnsigned` - If the pallet validates unsigned extrinsics.
/// * `path::to::pallet`: identifiers separated by colons which declare the path to a pallet
/// definition.
///
/// `= $n` is an optional part allowing to define at which index the pallet variants in
/// `OriginCaller`, `Call` and `Event` are encoded, and to define the ModuleToIndex value.
/// * `::<InstanceN>` optional: specify the instance of the pallet to use. If not specified it will
/// use the default instance (or the only instance in case of non-instantiable pallets).
///
/// if `= $n` is not given, then index is resolved same as fieldless enum in Rust
/// (i.e. incrementedly from previous index):
/// ```nocompile
/// pallet1 .. = 2,
/// pallet2 .., // Here pallet2 is given index 3
/// pallet3 .. = 0,
/// pallet4 .., // Here pallet4 is given index 1
/// ```
/// * `::{ Part1, Part2<T>, .. }` optional if pallet declared with `frame_support::pallet`: Comma
/// separated parts declared with their generic. If a pallet is declared with
/// `frame_support::pallet` macro then the parts can be automatically derived if not explicitly
/// provided. We provide support for the following module parts in a pallet:
///
/// - `Pallet` - Required for all pallets
/// - `Call` - If the pallet has callable functions
/// - `Storage` - If the pallet uses storage
/// - `Event` or `Event<T>` (if the event is generic) - If the pallet emits events
/// - `Origin` or `Origin<T>` (if the origin is generic) - If the pallet has instanciable origins
/// - `Config` or `Config<T>` (if the config is generic) - If the pallet builds the genesis
/// storage with `GenesisConfig`
/// - `Inherent` - If the pallet provides/can check inherents.
/// - `ValidateUnsigned` - If the pallet validates unsigned extrinsics.
///
/// It is important to list these parts here to export them correctly in the metadata or to make
/// the pallet usable in the runtime.
///
/// * `exclude_parts { Part1, Part2 }` optional: comma separated parts without generics. I.e. one of
/// `Pallet`, `Call`, `Storage`, `Event`, `Origin`, `Config`, `Inherent`, `ValidateUnsigned`. It
/// is incompatible with `use_parts`. This specifies the part to exclude. In order to select
/// subset of the pallet parts.
///
/// For example excluding the part `Call` can be useful if the runtime doesn't want to make the
/// pallet calls available.
///
/// * `use_parts { Part1, Part2 }` optional: comma separated parts without generics. I.e. one of
/// `Pallet`, `Call`, `Storage`, `Event`, `Origin`, `Config`, `Inherent`, `ValidateUnsigned`. It
/// is incompatible with `exclude_parts`. This specifies the part to use. In order to select a
/// subset of the pallet parts.
///
/// For example not using the part `Call` can be useful if the runtime doesn't want to make the
/// pallet calls available.
///
/// * `= $n` optional: number to define at which index the pallet variants in `OriginCaller`, `Call`
/// and `Event` are encoded, and to define the ModuleToIndex value.
///
/// if `= $n` is not given, then index is resolved in the same way as fieldless enum in Rust
/// (i.e. incrementedly from previous index):
/// ```nocompile
/// pallet1 .. = 2,
/// pallet2 .., // Here pallet2 is given index 3
/// pallet3 .. = 0,
/// pallet4 .., // Here pallet4 is given index 1
/// ```
///
/// # Note
///
@@ -352,8 +392,8 @@ pub fn decl_storage(input: TokenStream) -> TokenStream {
///
/// # Type definitions
///
/// * The macro generates a type alias for each pallet to their `Module` (or `Pallet`). E.g. `type
/// System = frame_system::Pallet<Runtime>`
/// * The macro generates a type alias for each pallet to their `Pallet`. E.g. `type System =
/// frame_system::Pallet<Runtime>`
#[proc_macro]
pub fn construct_runtime(input: TokenStream) -> TokenStream {
construct_runtime::construct_runtime(input)
@@ -498,3 +538,27 @@ pub fn impl_key_prefix_for_tuples(input: TokenStream) -> TokenStream {
pub fn __generate_dummy_part_checker(input: TokenStream) -> TokenStream {
dummy_part_checker::generate_dummy_part_checker(input)
}
/// Macro that inserts some tokens after the first match of some pattern.
///
/// # Example:
///
/// ```nocompile
/// match_and_insert!(
/// target = [{ Some content with { at some point match pattern } other match pattern are ignored }]
/// pattern = [{ match pattern }] // the match pattern cannot contain any group: `[]`, `()`, `{}`
/// // can relax this constraint, but will require modifying the match logic in code
/// tokens = [{ expansion tokens }] // content inside braces can be anything including groups
/// );
/// ```
///
/// will generate:
///
/// ```nocompile
/// Some content with { at some point match pattern expansion tokens } other match patterns are
/// ignored
/// ```
#[proc_macro]
pub fn match_and_insert(input: TokenStream) -> TokenStream {
match_and_insert::match_and_insert(input)
}
substrate/frame/support/procedural/src/match_and_insert.rs
+159
View File
@@ -0,0 +1,159 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Implementation of the `match_and_insert` macro.
use proc_macro2::{Group, Span, TokenStream, TokenTree};
use std::iter::once;
use syn::spanned::Spanned;
mod keyword {
syn::custom_keyword!(target);
syn::custom_keyword!(pattern);
syn::custom_keyword!(tokens);
}
pub fn match_and_insert(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let MatchAndInsertDef { pattern, tokens, target } =
syn::parse_macro_input!(input as MatchAndInsertDef);
match expand_in_stream(&pattern, &mut Some(tokens), target) {
Ok(stream) => stream.into(),
Err(err) => err.to_compile_error().into(),
}
}
struct MatchAndInsertDef {
// Token stream to search and insert tokens into.
target: TokenStream,
// Pattern to match against, this is ensured to have no TokenTree::Group nor TokenTree::Literal
// (i.e. contains only Punct or Ident), and not being empty.
pattern: Vec<TokenTree>,
// Token stream to insert after the match pattern.
tokens: TokenStream,
}
impl syn::parse::Parse for MatchAndInsertDef {
fn parse(input: syn::parse::ParseStream) -> syn::Result<Self> {
let mut target;
let _ = input.parse::<keyword::target>()?;
let _ = input.parse::<syn::Token![=]>()?;
let _replace_with_bracket: syn::token::Bracket = syn::bracketed!(target in input);
let _replace_with_brace: syn::token::Brace = syn::braced!(target in target);
let target = target.parse()?;
let mut pattern;
let _ = input.parse::<keyword::pattern>()?;
let _ = input.parse::<syn::Token![=]>()?;
let _replace_with_bracket: syn::token::Bracket = syn::bracketed!(pattern in input);
let _replace_with_brace: syn::token::Brace = syn::braced!(pattern in pattern);
let pattern = pattern.parse::<TokenStream>()?.into_iter().collect::<Vec<TokenTree>>();
if let Some(t) = pattern.iter().find(|t| matches!(t, TokenTree::Group(_))) {
return Err(syn::Error::new(t.span(), "Unexpected group token tree"))
}
if let Some(t) = pattern.iter().find(|t| matches!(t, TokenTree::Literal(_))) {
return Err(syn::Error::new(t.span(), "Unexpected literal token tree"))
}
if pattern.is_empty() {
return Err(syn::Error::new(Span::call_site(), "empty match pattern is invalid"))
}
let mut tokens;
let _ = input.parse::<keyword::tokens>()?;
let _ = input.parse::<syn::Token![=]>()?;
let _replace_with_bracket: syn::token::Bracket = syn::bracketed!(tokens in input);
let _replace_with_brace: syn::token::Brace = syn::braced!(tokens in tokens);
let tokens = tokens.parse()?;
Ok(Self { tokens, pattern, target })
}
}
// Insert `tokens` after the first matching `pattern`.
// `tokens` must be some (Option is used for internal simplification).
// `pattern` must not be empty and should only contain Ident or Punct.
fn expand_in_stream(
pattern: &[TokenTree],
tokens: &mut Option<TokenStream>,
stream: TokenStream,
) -> syn::Result<TokenStream> {
assert!(
tokens.is_some(),
"`tokens` must be some, Option is used because `tokens` is used only once"
);
assert!(
!pattern.is_empty(),
"`pattern` must not be empty, otherwise there is nothing to match against"
);
let stream_span = stream.span();
let mut stream = stream.into_iter();
let mut extended = TokenStream::new();
let mut match_cursor = 0;
while let Some(token) = stream.next() {
match token {
TokenTree::Group(group) => {
match_cursor = 0;
let group_stream = group.stream();
match expand_in_stream(pattern, tokens, group_stream) {
Ok(s) => {
extended.extend(once(TokenTree::Group(Group::new(group.delimiter(), s))));
extended.extend(stream);
return Ok(extended)
},
Err(_) => {
extended.extend(once(TokenTree::Group(group)));
},
}
},
other => {
advance_match_cursor(&other, pattern, &mut match_cursor);
extended.extend(once(other));
if match_cursor == pattern.len() {
extended
.extend(once(tokens.take().expect("tokens is used to replace only once")));
extended.extend(stream);
return Ok(extended)
}
},
}
}
// if we reach this point, it means the stream is empty and we haven't found a matching pattern
let msg = format!("Cannot find pattern `{:?}` in given token stream", pattern);
Err(syn::Error::new(stream_span, msg))
}
fn advance_match_cursor(other: &TokenTree, pattern: &[TokenTree], match_cursor: &mut usize) {
use TokenTree::{Ident, Punct};
let does_match_other_pattern = match (other, &pattern[*match_cursor]) {
(Ident(i1), Ident(i2)) => i1 == i2,
(Punct(p1), Punct(p2)) => p1.as_char() == p2.as_char(),
_ => false,
};
if does_match_other_pattern {
*match_cursor += 1;
} else {
*match_cursor = 0;
}
}
substrate/frame/support/procedural/src/pallet/expand/mod.rs
+3
View File
@@ -29,6 +29,7 @@ mod origin;
mod pallet_struct;
mod storage;
mod store_trait;
mod tt_default_parts;
mod type_value;
mod validate_unsigned;
@@ -67,6 +68,7 @@ pub fn expand(mut def: Def) -> proc_macro2::TokenStream {
let type_values = type_value::expand_type_values(&mut def);
let origins = origin::expand_origins(&mut def);
let validate_unsigned = validate_unsigned::expand_validate_unsigned(&mut def);
let tt_default_parts = tt_default_parts::expand_tt_default_parts(&mut def);
if get_doc_literals(&def.item.attrs).is_empty() {
def.item.attrs.push(syn::parse_quote!(
@@ -96,6 +98,7 @@ pub fn expand(mut def: Def) -> proc_macro2::TokenStream {
#type_values
#origins
#validate_unsigned
#tt_default_parts
);
def.item
substrate/frame/support/procedural/src/pallet/expand/tt_default_parts.rs
+82
View File
@@ -0,0 +1,82 @@
// This file is part of Substrate.
// Copyright (C) 2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::{pallet::Def, COUNTER};
use syn::spanned::Spanned;
/// Generate the `tt_default_parts` macro.
pub fn expand_tt_default_parts(def: &mut Def) -> proc_macro2::TokenStream {
let count = COUNTER.with(|counter| counter.borrow_mut().inc());
let default_parts_unique_id =
syn::Ident::new(&format!("__tt_default_parts_{}", count), def.item.span());
let call_part = def.call.as_ref().map(|_| quote::quote!(Call,));
let storage_part = (!def.storages.is_empty()).then(|| quote::quote!(Storage,));
let event_part = def.event.as_ref().map(|event| {
let gen = event.gen_kind.is_generic().then(|| quote::quote!( <T> ));
quote::quote!( Event #gen , )
});
let origin_part = def.origin.as_ref().map(|origin| {
let gen = origin.is_generic.then(|| quote::quote!( <T> ));
quote::quote!( Origin #gen , )
});
let config_part = def.genesis_config.as_ref().map(|genesis_config| {
let gen = genesis_config.gen_kind.is_generic().then(|| quote::quote!( <T> ));
quote::quote!( Config #gen , )
});
let inherent_part = def.inherent.as_ref().map(|_| quote::quote!(Inherent,));
let validate_unsigned_part =
def.validate_unsigned.as_ref().map(|_| quote::quote!(ValidateUnsigned,));
quote::quote!(
// This macro follows the conventions as laid out by the `tt-call` crate. It does not
// accept any arguments and simply returns the pallet parts, separated by commas, then
// wrapped inside of braces and finally prepended with double colons, to the caller inside
// of a key named `tokens`.
//
// We need to accept a frame_support argument here, because this macro gets expanded on the
// crate that called the `construct_runtime!` macro, and said crate may have renamed
// frame-support, and so we need to pass in the frame-support path that said crate
// recognizes.
#[macro_export]
#[doc(hidden)]
macro_rules! #default_parts_unique_id {
{
$caller:tt
frame_support = [{ $($frame_support:ident)::* }]
} => {
$($frame_support)*::tt_return! {
$caller
tokens = [{
::{
Pallet, #call_part #storage_part #event_part #origin_part #config_part
#inherent_part #validate_unsigned_part
}
}]
}
};
}
pub use #default_parts_unique_id as tt_default_parts;
)
}