Default Pallet Config Trait / derive_impl (#13454)

* first draft, probably won't work

* first draft, probably won't work

* good progress..

* good milestone, still a lot to do.

* EVERYTHING WORKS

* Update frame/support/procedural/src/derive_impl.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/procedural/src/derive_impl.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clean up + cargo fmt

* import tokens WIP

* export_tokens working with impl Trait

* WIP / notes

* use macro_magic 0.2.0's export_tokens to access foreign items

* token importing working properly using macro_magic 0.2.5

* combine_impls almost working

* successfully get foreign path via macro_magic 0.2.6

* combine_impls using implementing_type generics

* working + clean up

* more clean up

* decrease rightwards drift and add docs to combine_impls

* add support for macros to impl_item_ident in case we hit that

* add docs for impl_item_ident method

* fix no_std issues

* re-export of macro_magic working in pallets 🎉

* clean up + fully resolve no_std issue with macro_magic with v0.2.11

* remove trait item code for different trait item types since this
is now handled directly by combine_impls

* clean up

* remove dev comments

* only generate default trait if #[pallet::default_trait] is attached

* authorship and most other pallets now compiling

* compiling 🎉

* add check for more than two pallet attributes on Config trait

* remove unused import in nomination-pool

* clean up debug code

* upgrade to macro_magic v0.2.12

* add neater #[register_default_config(SomeIdent)] macro

* really just a thin wrapper around #[export_tokens]

* upgrade to macro_magic 0.3.1

* rewrite parsing to be compatible with syn 2.x, compiling 🎉

* remove unused keywords

* macro stubs for the new pallet:: macros, preliminary docs

* upgrade to macro_magic v0.3.2

* rename register_default_config => register_default_impl

* bump to macro_magic v0.3.3

* custom disambiguation_path working as 2nd arg to derive_impl

* overhaul docs

* fixes, ident-style paths shortcut working

* remove ident-style shortcut because it makes testing difficult

* add passing UI tests for derive_impl

* switch to `ForeignPath as DisambiguationPath` syntax + update docs

* add UI test for bad foreign path

* add UI test for bad disambiguation path

* add UI test for missing disambiguation path

* add UI test for attached to non impl

* fix derive_impl_attr_args_parsing test

* move tests to bottom

* fix nightly issue

* add doc notes on importing/re-exporting

* remove explicit use of macro_magic::use_attr

Co-authored-by: Bastian Köcher <git@kchr.de>

* use explicit macro_magic::use_attr

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove unneeded {}

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove unneeded collect

Co-authored-by: Bastian Köcher <git@kchr.de>

* add docs for TestDefaultConfig

* remove unneeded `#[export_tokens]` on `DefaultConfig`

* add docs for auto-generated `DefaultConfig`

* no need to clone

Co-authored-by: Bastian Köcher <git@kchr.de>

* clean up combine_impls + compiling again

* remove unused dependency

* simplify struct definition

Co-authored-by: Bastian Köcher <git@kchr.de>

* fix register_default_impl docs

* reduce rightward drift / refactor

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix derive_impl after keith's changes

* simplify disambiguation_path calculation

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* compiling again

* simplify parsing of trait item

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* rename preludes => prelude

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix other places where we used preludes instead of prelude

* fix indents

* simplify PalletAttr parsing

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* go back to having no_default and constant as keywords

* make it more clear that disambiguation_path is optional

* make default_trait_items just a Vec instead of Option<Vec>

* rename foreign_path => default_impl_path within substrate

* fix docs

* Change {} to ;

Co-authored-by: Bastian Köcher <git@kchr.de>

* highlight full end-to-end example with link

* add pallet-default-config-example, start by copying dev mode code

* update dev-mode specific docs

* use Person and Points instead of Dummy and Bar

* add docs to example pallet

* revert changes to pallets other than the default config example

* fix outdated references to basic example pallet

* re-order docs to be a bit more clear

* better errors for extra attributes

* add UI tests for duplicate/extra attributes on trait items

* change `#[pallet::default_config]` to option on `#[pallet::config()]`

* update UI tests
* add UI test covering missing `#[pallet::config(with_default)]` when
  `#[pallet::no_default]` is used

* add note about new optional conventions

* improve docs about `DefaultConfig` and link to these from a few places

* fix doc comment

* fix old comment referencing `pallet::default_config`

* use u32 instead of u64 for block number

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* use () instead of u32 for `AccountData`

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* use ConstU32<10> for BlockHashCount instead of ConstU64<10>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* people are not dummies

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* fix wording

Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>

* Person => People and compiling again

* add docs for `prelude` module in frame_system

* update Cargo.lock

* cleaner example

* tweaks

* update docs more

* update docs more

* update docs more

* update docs more

* fix ui tests

* err

* Update frame/support/test/tests/pallet_ui.rs

* update ui tests

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
This commit is contained in:
Kian Paimani
2023-05-30 10:06:40 +02:00
committed by GitHub
parent a8edaf47d5
commit 263a5d6c1e
35 changed files with 1445 additions and 80 deletions
+318 -1
View File
@@ -25,6 +25,7 @@ mod construct_runtime;
mod crate_version;
mod debug_no_bound;
mod default_no_bound;
mod derive_impl;
mod dummy_part_checker;
mod key_prefix;
mod match_and_insert;
@@ -36,10 +37,12 @@ mod storage_alias;
mod transactional;
mod tt_macro;
use macro_magic::import_tokens_attr;
use proc_macro::TokenStream;
use quote::quote;
use quote::{quote, ToTokens};
use std::{cell::RefCell, str::FromStr};
pub(crate) use storage::INHERENT_INSTANCE_NAME;
use syn::{parse_macro_input, ItemImpl};
thread_local! {
/// A global counter, can be used to generate a relatively unique identifier.
@@ -777,6 +780,274 @@ pub fn storage_alias(_: TokenStream, input: TokenStream) -> TokenStream {
.into()
}
/// This attribute can be used to derive a full implementation of a trait based on a local partial
/// impl and an external impl containing defaults that can be overriden in the local impl.
///
/// For a full end-to-end example, see [below](#use-case-auto-derive-test-pallet-config-traits).
///
/// # Usage
///
/// The attribute should be attached to an impl block (strictly speaking a `syn::ItemImpl`) for
/// which we want to inject defaults in the event of missing trait items in the block.
///
/// The attribute minimally takes a single `default_impl_path` argument, which should be the module
/// path to an impl registered via [`#[register_default_impl]`](`macro@register_default_impl`) that
/// contains the default trait items we want to potentially inject, with the general form:
///
/// ```ignore
/// #[derive_impl(default_impl_path)]
/// impl SomeTrait for SomeStruct {
/// ...
/// }
/// ```
///
/// Optionally, a `disambiguation_path` can be specified as follows by providing `as path::here`
/// after the `default_impl_path`:
///
/// ```ignore
/// #[derive_impl(default_impl_path as disambiguation_path)]
/// impl SomeTrait for SomeStruct {
/// ...
/// }
/// ```
///
/// The `disambiguation_path`, if specified, should be the path to a trait that will be used to
/// qualify all default entries that are injected into the local impl. For example if your
/// `default_impl_path` is `some::path::TestTraitImpl` and your `disambiguation_path` is
/// `another::path::DefaultTrait`, any items injected into the local impl will be qualified as
/// `<some::path::TestTraitImpl as another::path::DefaultTrait>::specific_trait_item`.
///
/// If you omit the `as disambiguation_path` portion, the `disambiguation_path` will internally
/// default to `A` from the `impl A for B` part of the default impl. This is useful for scenarios
/// where all of the relevant types are already in scope via `use` statements.
///
/// Conversely, the `default_impl_path` argument is required and cannot be omitted.
///
/// You can also make use of `#[pallet::no_default]` on specific items in your default impl that you
/// want to ensure will not be copied over but that you nonetheless want to use locally in the
/// context of the foreign impl and the pallet (or context) in which it is defined.
///
/// ## Use-Case Example: Auto-Derive Test Pallet Config Traits
///
/// The `#[derive_imp(..)]` attribute can be used to derive a test pallet `Config` based on an
/// existing pallet `Config` that has been marked with
/// [`#[pallet::config(with_default)]`](`macro@config`) (which under the hood, generates a
/// `DefaultConfig` trait in the pallet in which the macro was invoked).
///
/// In this case, the `#[derive_impl(..)]` attribute should be attached to an `impl` block that
/// implements a compatible `Config` such as `frame_system::Config` for a test/mock runtime, and
/// should receive as its first argument the path to a `DefaultConfig` impl that has been registered
/// via [`#[register_default_impl]`](`macro@register_default_impl`), and as its second argument, the
/// path to the auto-generated `DefaultConfig` for the existing pallet `Config` we want to base our
/// test config off of.
///
/// The following is what the `basic` example pallet would look like with a default testing config:
///
/// ```ignore
/// #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::pallet::DefaultConfig)]
/// impl frame_system::Config for Test {
/// // These are all defined by system as mandatory.
/// type BaseCallFilter = frame_support::traits::Everything;
/// type RuntimeEvent = RuntimeEvent;
/// type RuntimeCall = RuntimeCall;
/// type RuntimeOrigin = RuntimeOrigin;
/// type OnSetCode = ();
/// type PalletInfo = PalletInfo;
/// type Header = Header;
/// // We decide to override this one.
/// type AccountData = pallet_balances::AccountData<u64>;
/// }
/// ```
///
/// where `TestDefaultConfig` was defined and registered as follows:
///
/// ```ignore
/// pub struct TestDefaultConfig;
///
/// #[register_default_impl(TestDefaultConfig)]
/// impl DefaultConfig for TestDefaultConfig {
/// type Version = ();
/// type BlockWeights = ();
/// type BlockLength = ();
/// type DbWeight = ();
/// type Index = u64;
/// type BlockNumber = u64;
/// type Hash = sp_core::hash::H256;
/// type Hashing = sp_runtime::traits::BlakeTwo256;
/// type AccountId = AccountId;
/// type Lookup = IdentityLookup<AccountId>;
/// type BlockHashCount = frame_support::traits::ConstU64<10>;
/// type AccountData = u32;
/// type OnNewAccount = ();
/// type OnKilledAccount = ();
/// type SystemWeightInfo = ();
/// type SS58Prefix = ();
/// type MaxConsumers = frame_support::traits::ConstU32<16>;
/// }
/// ```
///
/// The above call to `derive_impl` would expand to roughly the following:
///
/// ```ignore
/// impl frame_system::Config for Test {
/// use frame_system::config_preludes::TestDefaultConfig;
/// use frame_system::pallet::DefaultConfig;
///
/// type BaseCallFilter = frame_support::traits::Everything;
/// type RuntimeEvent = RuntimeEvent;
/// type RuntimeCall = RuntimeCall;
/// type RuntimeOrigin = RuntimeOrigin;
/// type OnSetCode = ();
/// type PalletInfo = PalletInfo;
/// type Header = Header;
/// type AccountData = pallet_balances::AccountData<u64>;
/// type Version = <TestDefaultConfig as DefaultConfig>::Version;
/// type BlockWeights = <TestDefaultConfig as DefaultConfig>::BlockWeights;
/// type BlockLength = <TestDefaultConfig as DefaultConfig>::BlockLength;
/// type DbWeight = <TestDefaultConfig as DefaultConfig>::DbWeight;
/// type Index = <TestDefaultConfig as DefaultConfig>::Index;
/// type BlockNumber = <TestDefaultConfig as DefaultConfig>::BlockNumber;
/// type Hash = <TestDefaultConfig as DefaultConfig>::Hash;
/// type Hashing = <TestDefaultConfig as DefaultConfig>::Hashing;
/// type AccountId = <TestDefaultConfig as DefaultConfig>::AccountId;
/// type Lookup = <TestDefaultConfig as DefaultConfig>::Lookup;
/// type BlockHashCount = <TestDefaultConfig as DefaultConfig>::BlockHashCount;
/// type OnNewAccount = <TestDefaultConfig as DefaultConfig>::OnNewAccount;
/// type OnKilledAccount = <TestDefaultConfig as DefaultConfig>::OnKilledAccount;
/// type SystemWeightInfo = <TestDefaultConfig as DefaultConfig>::SystemWeightInfo;
/// type SS58Prefix = <TestDefaultConfig as DefaultConfig>::SS58Prefix;
/// type MaxConsumers = <TestDefaultConfig as DefaultConfig>::MaxConsumers;
/// }
/// ```
///
/// You can then use the resulting `Test` config in test scenarios.
///
/// Note that items that are _not_ present in our local `DefaultConfig` are automatically copied
/// from the foreign trait (in this case `TestDefaultConfig`) into the local trait impl (in this
/// case `Test`), unless the trait item in the local trait impl is marked with
/// [`#[pallet::no_default]`](`macro@no_default`), in which case it cannot be overridden, and any
/// attempts to do so will result in a compiler error.
///
/// See `frame/examples/default-config/tests.rs` for a runnable end-to-end example pallet that makes
/// use of `derive_impl` to derive its testing config.
///
/// See [here](`macro@config`) for more information and caveats about the auto-generated
/// `DefaultConfig` trait.
///
/// ## Optional Conventions
///
/// Note that as an optional convention, we encourage creating a `config_preludes` module inside of
/// your pallet. This is the convention we follow for `frame_system`'s `TestDefaultConfig` which, as
/// shown above, is located at `frame_system::config_preludes::TestDefaultConfig`. This is just a
/// suggested convention -- there is nothing in the code that expects modules with these names to be
/// in place, so there is no imperative to follow this pattern unless desired.
///
/// In `config_preludes`, you can place types named like:
///
/// * `TestDefaultConfig`
/// * `ParachainDefaultConfig`
/// * `SolochainDefaultConfig`
///
/// Signifying in which context they can be used.
///
/// # Advanced Usage
///
/// ## Importing & Re-Exporting
///
/// Since `#[derive_impl(..)]` is a
/// [`macro_magic`](https://docs.rs/macro_magic/latest/macro_magic/)-based attribute macro, special
/// care must be taken when importing and re-exporting it. Glob imports will work properly, such as
/// `use frame_support::*` to bring `derive_impl` into scope, however any other use statements
/// involving `derive_impl` should have
/// [`#[macro_magic::use_attr]`](https://docs.rs/macro_magic/latest/macro_magic/attr.use_attr.html)
/// attached or your use statement will fail to fully bring the macro into scope.
///
/// This brings `derive_impl` into scope in the current context:
/// ```ignore
/// #[use_attr]
/// use frame_support::derive_impl;
/// ```
///
/// This brings `derive_impl` into scope and publicly re-exports it from the current context:
/// ```ignore
/// #[use_attr]
/// pub use frame_support::derive_impl;
/// ```
///
/// ## Expansion
///
/// The `#[derive_impl(default_impl_path as disambiguation_path)]` attribute will expand to the
/// local impl, with any extra items from the foreign impl that aren't present in the local impl
/// also included. In the case of a colliding trait item, the version of the item that exists in the
/// local impl will be retained. All imported items are qualified by the `disambiguation_path`, as
/// discussed above.
///
/// ## Handling of Unnamed Trait Items
///
/// Items that lack a `syn::Ident` for whatever reason are first checked to see if they exist,
/// verbatim, in the local/destination trait before they are copied over, so you should not need to
/// worry about collisions between identical unnamed items.
#[import_tokens_attr(frame_support::macro_magic)]
#[with_custom_parsing(derive_impl::DeriveImplAttrArgs)]
#[proc_macro_attribute]
pub fn derive_impl(attrs: TokenStream, input: TokenStream) -> TokenStream {
let custom_attrs = parse_macro_input!(__custom_tokens as derive_impl::DeriveImplAttrArgs);
derive_impl::derive_impl(
__source_path.into(),
attrs.into(),
input.into(),
custom_attrs.disambiguation_path,
)
.unwrap_or_else(|r| r.into_compile_error())
.into()
}
/// The optional attribute `#[pallet::no_default]` can be attached to trait items within a
/// `Config` trait impl that has [`#[pallet::config(with_default)]`](`macro@config`) attached.
///
/// Attaching this attribute to a trait item ensures that that trait item will not be used as a
/// default with the [`#[derive_impl(..)]`](`macro@derive_impl`) attribute macro.
#[proc_macro_attribute]
pub fn no_default(_: TokenStream, _: TokenStream) -> TokenStream {
pallet_macro_stub()
}
/// Attach this attribute to an impl statement that you want to use with
/// [`#[derive_impl(..)]`](`macro@derive_impl`).
///
/// You must also provide an identifier/name as the attribute's argument. This is the name you
/// must provide to [`#[derive_impl(..)]`](`macro@derive_impl`) when you import this impl via
/// the `default_impl_path` argument. This name should be unique at the crate-level.
///
/// ## Example
///
/// ```ignore
/// pub struct ExampleTestDefaultConfig;
///
/// #[register_default_impl(ExampleTestDefaultConfig)]
/// impl DefaultConfig for ExampleTestDefaultConfig {
/// type Version = ();
/// type BlockWeights = ();
/// type BlockLength = ();
/// ...
/// type SS58Prefix = ();
/// type MaxConsumers = frame_support::traits::ConstU32<16>;
/// }
/// ```
/// This macro acts as a thin wrapper around macro_magic's `#[export_tokens]`. See the docs
/// [here](https://docs.rs/macro_magic/latest/macro_magic/attr.export_tokens.html) for more info.
#[proc_macro_attribute]
pub fn register_default_impl(attrs: TokenStream, tokens: TokenStream) -> TokenStream {
// ensure this is a impl statement
let item_impl = syn::parse_macro_input!(tokens as ItemImpl);
// internally wrap macro_magic's `#[export_tokens]` macro
match macro_magic::mm_core::export_tokens_internal(attrs, item_impl.to_token_stream(), true) {
Ok(tokens) => tokens.into(),
Err(err) => err.to_compile_error().into(),
}
}
/// Used internally to decorate pallet attribute macro stubs when they are erroneously used
/// outside of a pallet module
fn pallet_macro_stub() -> TokenStream {
@@ -809,6 +1080,52 @@ fn pallet_macro_stub() -> TokenStream {
///
/// [`pallet::event`](`macro@event`) must be present if `RuntimeEvent` exists as a config item
/// in your `#[pallet::config]`.
///
/// ## Optional: `with_default`
///
/// An optional `with_default` argument may also be specified. Doing so will automatically
/// generate a `DefaultConfig` trait inside your pallet which is suitable for use with
/// [`[#[derive_impl(..)]`](`macro@derive_impl`) to derive a default testing config:
///
/// ```ignore
/// #[pallet::config(with_default)]
/// pub trait Config: frame_system::Config {
/// type RuntimeEvent: Parameter
/// + Member
/// + From<Event<Self>>
/// + Debug
/// + IsType<<Self as frame_system::Config>::RuntimeEvent>;
///
/// #[pallet::no_default]
/// type BaseCallFilter: Contains<Self::RuntimeCall>;
/// // ...
/// }
/// ```
///
/// As shown above, you may also attach the [`#[pallet::no_default]`](`macro@no_default`)
/// attribute to specify that a particular trait item _cannot_ be used as a default when a test
/// `Config` is derived using the [`#[derive_impl(..)]`](`macro@derive_impl`) attribute macro.
/// This will cause that particular trait item to simply not appear in default testing configs
/// based on this config (the trait item will not be included in `DefaultConfig`).
///
/// ### `DefaultConfig` Caveats
///
/// The auto-generated `DefaultConfig` trait:
/// - is always a _subset_ of your pallet's `Config` trait.
/// - can only contain items that don't rely on externalities, such as `frame_system::Config`.
///
/// Trait items that _do_ rely on externalities should be marked with
/// [`#[pallet::no_default]`](`macro@no_default`)
///
/// Consequently:
/// - Any items that rely on externalities _must_ be marked with
/// [`#[pallet::no_default]`](`macro@no_default`) or your trait will fail to compile when used
/// with [`derive_impl`](`macro@derive_impl`).
/// - Items marked with [`#[pallet::no_default]`](`macro@no_default`) are entirely excluded from the
/// `DefaultConfig` trait, and therefore any impl of `DefaultConfig` doesn't need to implement
/// such items.
///
/// For more information, see [`macro@derive_impl`].
#[proc_macro_attribute]
pub fn config(_: TokenStream, _: TokenStream) -> TokenStream {
pallet_macro_stub()