Default Pallet Config Trait / derive_impl (#13454)

* first draft, probably won't work

* first draft, probably won't work

* good progress..

* good milestone, still a lot to do.

* EVERYTHING WORKS

* Update frame/support/procedural/src/derive_impl.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/procedural/src/derive_impl.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clean up + cargo fmt

* import tokens WIP

* export_tokens working with impl Trait

* WIP / notes

* use macro_magic 0.2.0's export_tokens to access foreign items

* token importing working properly using macro_magic 0.2.5

* combine_impls almost working

* successfully get foreign path via macro_magic 0.2.6

* combine_impls using implementing_type generics

* working + clean up

* more clean up

* decrease rightwards drift and add docs to combine_impls

* add support for macros to impl_item_ident in case we hit that

* add docs for impl_item_ident method

* fix no_std issues

* re-export of macro_magic working in pallets 🎉

* clean up + fully resolve no_std issue with macro_magic with v0.2.11

* remove trait item code for different trait item types since this
is now handled directly by combine_impls

* clean up

* remove dev comments

* only generate default trait if #[pallet::default_trait] is attached

* authorship and most other pallets now compiling

* compiling 🎉

* add check for more than two pallet attributes on Config trait

* remove unused import in nomination-pool

* clean up debug code

* upgrade to macro_magic v0.2.12

* add neater #[register_default_config(SomeIdent)] macro

* really just a thin wrapper around #[export_tokens]

* upgrade to macro_magic 0.3.1

* rewrite parsing to be compatible with syn 2.x, compiling 🎉

* remove unused keywords

* macro stubs for the new pallet:: macros, preliminary docs

* upgrade to macro_magic v0.3.2

* rename register_default_config => register_default_impl

* bump to macro_magic v0.3.3

* custom disambiguation_path working as 2nd arg to derive_impl

* overhaul docs

* fixes, ident-style paths shortcut working

* remove ident-style shortcut because it makes testing difficult

* add passing UI tests for derive_impl

* switch to `ForeignPath as DisambiguationPath` syntax + update docs

* add UI test for bad foreign path

* add UI test for bad disambiguation path

* add UI test for missing disambiguation path

* add UI test for attached to non impl

* fix derive_impl_attr_args_parsing test

* move tests to bottom

* fix nightly issue

* add doc notes on importing/re-exporting

* remove explicit use of macro_magic::use_attr

Co-authored-by: Bastian Köcher <git@kchr.de>

* use explicit macro_magic::use_attr

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove unneeded {}

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove unneeded collect

Co-authored-by: Bastian Köcher <git@kchr.de>

* add docs for TestDefaultConfig

* remove unneeded `#[export_tokens]` on `DefaultConfig`

* add docs for auto-generated `DefaultConfig`

* no need to clone

Co-authored-by: Bastian Köcher <git@kchr.de>

* clean up combine_impls + compiling again

* remove unused dependency

* simplify struct definition

Co-authored-by: Bastian Köcher <git@kchr.de>

* fix register_default_impl docs

* reduce rightward drift / refactor

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix derive_impl after keith's changes

* simplify disambiguation_path calculation

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* compiling again

* simplify parsing of trait item

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* rename preludes => prelude

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix other places where we used preludes instead of prelude

* fix indents

* simplify PalletAttr parsing

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* go back to having no_default and constant as keywords

* make it more clear that disambiguation_path is optional

* make default_trait_items just a Vec instead of Option<Vec>

* rename foreign_path => default_impl_path within substrate

* fix docs

* Change {} to ;

Co-authored-by: Bastian Köcher <git@kchr.de>

* highlight full end-to-end example with link

* add pallet-default-config-example, start by copying dev mode code

* update dev-mode specific docs

* use Person and Points instead of Dummy and Bar

* add docs to example pallet

* revert changes to pallets other than the default config example

* fix outdated references to basic example pallet

* re-order docs to be a bit more clear

* better errors for extra attributes

* add UI tests for duplicate/extra attributes on trait items

* change `#[pallet::default_config]` to option on `#[pallet::config()]`

* update UI tests
* add UI test covering missing `#[pallet::config(with_default)]` when
  `#[pallet::no_default]` is used

* add note about new optional conventions

* improve docs about `DefaultConfig` and link to these from a few places

* fix doc comment

* fix old comment referencing `pallet::default_config`

* use u32 instead of u64 for block number

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* use () instead of u32 for `AccountData`

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* use ConstU32<10> for BlockHashCount instead of ConstU64<10>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* people are not dummies

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* fix wording

Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>

* Person => People and compiling again

* add docs for `prelude` module in frame_system

* update Cargo.lock

* cleaner example

* tweaks

* update docs more

* update docs more

* update docs more

* update docs more

* fix ui tests

* err

* Update frame/support/test/tests/pallet_ui.rs

* update ui tests

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
This commit is contained in:
Kian Paimani
2023-05-30 10:06:40 +02:00
committed by GitHub
parent a8edaf47d5
commit 263a5d6c1e
35 changed files with 1445 additions and 80 deletions
@@ -0,0 +1,175 @@
// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Implementation of the `derive_impl` attribute macro.
use derive_syn_parse::Parse;
use macro_magic::mm_core::ForeignPath;
use proc_macro2::TokenStream as TokenStream2;
use quote::{quote, ToTokens};
use std::collections::HashSet;
use syn::{parse2, parse_quote, spanned::Spanned, Ident, ImplItem, ItemImpl, Path, Result, Token};
#[derive(Parse)]
pub struct DeriveImplAttrArgs {
pub default_impl_path: Path,
_as: Option<Token![as]>,
#[parse_if(_as.is_some())]
pub disambiguation_path: Option<Path>,
}
impl ForeignPath for DeriveImplAttrArgs {
fn foreign_path(&self) -> &Path {
&self.default_impl_path
}
}
impl ToTokens for DeriveImplAttrArgs {
fn to_tokens(&self, tokens: &mut TokenStream2) {
tokens.extend(self.default_impl_path.to_token_stream());
tokens.extend(self._as.to_token_stream());
tokens.extend(self.disambiguation_path.to_token_stream());
}
}
/// Gets the [`Ident`] representation of the given [`ImplItem`], if one exists. Otherwise
/// returns [`None`].
///
/// Used by [`combine_impls`] to determine whether we can compare [`ImplItem`]s by [`Ident`]
/// or not.
fn impl_item_ident(impl_item: &ImplItem) -> Option<&Ident> {
match impl_item {
ImplItem::Const(item) => Some(&item.ident),
ImplItem::Fn(item) => Some(&item.sig.ident),
ImplItem::Type(item) => Some(&item.ident),
ImplItem::Macro(item) => item.mac.path.get_ident(),
_ => None,
}
}
/// The real meat behind `derive_impl`. Takes in a `local_impl`, which is the impl for which we
/// want to implement defaults (i.e. the one the attribute macro is attached to), and a
/// `foreign_impl`, which is the impl containing the defaults we want to use, and returns an
/// [`ItemImpl`] containing the final generated impl.
///
/// This process has the following caveats:
/// * Colliding items that have an ident are not copied into `local_impl`
/// * Uncolliding items that have an ident are copied into `local_impl` but are qualified as `type
/// #ident = <#default_impl_path as #disambiguation_path>::#ident;`
/// * Items that lack an ident are de-duplicated so only unique items that lack an ident are copied
/// into `local_impl`. Items that lack an ident and also exist verbatim in `local_impl` are not
/// copied over.
fn combine_impls(
local_impl: ItemImpl,
foreign_impl: ItemImpl,
default_impl_path: Path,
disambiguation_path: Path,
) -> ItemImpl {
let (existing_local_keys, existing_unsupported_items): (HashSet<ImplItem>, HashSet<ImplItem>) =
local_impl
.items
.iter()
.cloned()
.partition(|impl_item| impl_item_ident(impl_item).is_some());
let existing_local_keys: HashSet<Ident> = existing_local_keys
.into_iter()
.filter_map(|item| impl_item_ident(&item).cloned())
.collect();
let mut final_impl = local_impl;
let extended_items = foreign_impl.items.into_iter().filter_map(|item| {
if let Some(ident) = impl_item_ident(&item) {
if existing_local_keys.contains(&ident) {
// do not copy colliding items that have an ident
return None
}
if matches!(item, ImplItem::Type(_)) {
// modify and insert uncolliding type items
let modified_item: ImplItem = parse_quote! {
type #ident = <#default_impl_path as #disambiguation_path>::#ident;
};
return Some(modified_item)
}
// copy uncolliding non-type items that have an ident
Some(item)
} else {
// do not copy colliding items that lack an ident
(!existing_unsupported_items.contains(&item))
// copy uncolliding items without an ident verbatim
.then_some(item)
}
});
final_impl.items.extend(extended_items);
final_impl
}
/// Internal implementation behind [`#[derive_impl(..)]`](`macro@crate::derive_impl`).
///
/// `default_impl_path`: the module path of the external `impl` statement whose tokens we are
/// importing via `macro_magic`
///
/// `foreign_tokens`: the tokens for the external `impl` statement
///
/// `local_tokens`: the tokens for the local `impl` statement this attribute is attached to
///
/// `disambiguation_path`: the module path of the external trait we will use to qualify
/// defaults imported from the external `impl` statement
pub fn derive_impl(
default_impl_path: TokenStream2,
foreign_tokens: TokenStream2,
local_tokens: TokenStream2,
disambiguation_path: Option<Path>,
) -> Result<TokenStream2> {
let local_impl = parse2::<ItemImpl>(local_tokens)?;
let foreign_impl = parse2::<ItemImpl>(foreign_tokens)?;
let default_impl_path = parse2::<Path>(default_impl_path)?;
// have disambiguation_path default to the item being impl'd in the foreign impl if we
// don't specify an `as [disambiguation_path]` in the macro attr
let disambiguation_path = match (disambiguation_path, foreign_impl.clone().trait_) {
(Some(disambiguation_path), _) => disambiguation_path,
(None, Some((_, foreign_impl_path, _))) => foreign_impl_path,
_ =>
return Err(syn::Error::new(
foreign_impl.span(),
"Impl statement must have a defined type being implemented \
for a defined type such as `impl A for B`",
)),
};
// generate the combined impl
let combined_impl =
combine_impls(local_impl, foreign_impl, default_impl_path, disambiguation_path);
Ok(quote!(#combined_impl))
}
#[test]
fn test_derive_impl_attr_args_parsing() {
parse2::<DeriveImplAttrArgs>(quote!(
some::path::TestDefaultConfig as some::path::DefaultConfig
))
.unwrap();
parse2::<DeriveImplAttrArgs>(quote!(
frame_system::prelude::testing::TestDefaultConfig as DefaultConfig
))
.unwrap();
parse2::<DeriveImplAttrArgs>(quote!(Something as some::path::DefaultConfig)).unwrap();
parse2::<DeriveImplAttrArgs>(quote!(Something as DefaultConfig)).unwrap();
parse2::<DeriveImplAttrArgs>(quote!(DefaultConfig)).unwrap();
assert!(parse2::<DeriveImplAttrArgs>(quote!()).is_err());
assert!(parse2::<DeriveImplAttrArgs>(quote!(Config Config)).is_err());
}