pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-04-22 12:37:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use scale-typegen as a backend for the codegen (#1260)

Browse Source 
* integrate scale-typegen, remove types mod

* reintroduce default substitutes and derives

* support runtime_types only again

* generating polkadot.rs ok

* update scale-typegen to discrete error types

* scale-typegen-api-changes

* add note about UncheckedExtrinsic in default substitutes

* add resursive attributes and derives

* adjust example where Clone bound recursive

* move scale-typegen dependency to workspace

* expose default typegen settings

* lightclient: Fix wasm socket closure called after being dropped (#1289)

* lightclient: Close wasm socket while dropping from connecting state

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Construct one time only closures

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Enable console logs for lightclient WASM testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Separate wakes and check connectivity on poll_read

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Close the socket depending on internal state

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "lightclient: Separate wakes and check connectivity on poll_read"

This reverts commit 866094001d4c0b119a80ed681a74b323f74eae1b.

* lightclient: Return pending if socket is opening from poll_read

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Close the socket on `poll_close`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Reset closures on Drop to avoid recursive invokation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Close the socket if not already closing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* workflows: Install rustup component for building substrate (#1295)

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Command to fetch chainSpec and optimise its size (#1278)

* cli: Add chainSpec command

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli/chainSpec: Move to dedicated module

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Compute the state root hash

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Remove code substitutes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Update polkadot.json

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* scripts: Generate the chain spec

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Remove testing artifacts

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Apply rustfmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Introduce feature flag for smoldot dependency

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Rename chain-spec to chain-spec-pruning

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* scripts: Update chain-spec command

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* remove comments and unused args

* Update substrate- and signer-related dependencies (#1297)

* update crypto dependencies, adjust keypair

* add scale_info::TypeInfo derive in some places

* add multi signature derive

* fix lock file

* fix lock file again :|

* adjust to new interface in scale-typegen

* use released scale typegen

* reintroduce type aliases

* introduce type aliases again using scale-typegen

* cargo fmt and clippy

* reconcile changes with master branch

* update polkadot.rs

* bump scale-typgen to fix substitution

* implemented Alex suggestions, regenerated polkadot.rs (did not change)

* resolve conflicts in Cargo.lock

* make expect messages more clear

* correct typos

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
This commit is contained in:
Tadeo Hepperle
2024-01-11 16:42:51 +01:00
committed by GitHub
parent d004789b04
commit fc5a18aaa0
30 changed files with 1822 additions and 4874 deletions
Download Patch File Download Diff File Expand all files Collapse all files
codegen/src/api/calls.rs
+31 -45
View File
@@ -3,10 +3,10 @@
// see LICENSE for license details.
use super::CodegenError;
use crate::types::{CompositeDefFields, TypeGenerator};
use heck::{ToSnakeCase as _, ToUpperCamelCase as _};
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use scale_typegen::{typegen::ir::type_ir::CompositeIRKind, TypeGenerator};
use subxt_metadata::PalletMetadata;
/// Generate calls from the provided pallet's metadata. Each call returns a `StaticTxPayload`
@@ -14,81 +14,69 @@ use subxt_metadata::PalletMetadata;
///
/// # Arguments
///
/// - `metadata` - Runtime metadata from which the calls are generated.
/// - `type_gen` - The type generator containing all types defined by metadata.
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the calls are generated.
/// - `types_mod_ident` - The ident of the base module that we can use to access the generated types from.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
pub fn generate_calls(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
types_mod_ident: &syn::Ident,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
// Early return if the pallet has no calls.
let Some(call_ty) = pallet.call_ty_id() else {
return Ok(quote!());
};
let mut struct_defs = super::generate_structs_from_variants(
let variant_names_and_struct_defs = super::generate_structs_from_variants(
type_gen,
types_mod_ident,
call_ty,
|name| name.to_upper_camel_case().into(),
"Call",
crate_path,
should_gen_docs,
)?;
let result = struct_defs
.iter_mut()
.map(|(variant_name, struct_def, aliases)| {
let fn_name = format_ident!("{}", variant_name.to_snake_case());
let result: Vec<_> = match struct_def.fields {
CompositeDefFields::Named(ref named_fields) => named_fields
let (call_structs, call_fns): (Vec<_>, Vec<_>) = variant_names_and_struct_defs
.into_iter()
.map(|var| {
let (call_fn_args, call_args): (Vec<_>, Vec<_>) = match &var.composite.kind {
CompositeIRKind::Named(named_fields) => named_fields
.iter()
.map(|(name, field)| {
let call_arg = if field.is_boxed() {
// Note: fn_arg_type this is relative the type path of the type alias when prefixed with `types::`, e.g. `set_max_code_size::New`
let fn_arg_type = &field.type_path;
let call_arg = if field.is_boxed {
quote! { #name: ::std::boxed::Box::new(#name) }
} else {
quote! { #name }
};
let alias_name =
format_ident!("{}", name.to_string().to_upper_camel_case());
(quote!( #name: types::#fn_name::#alias_name ), call_arg)
(quote!( #name: types::#fn_arg_type ), call_arg)
})
.collect(),
CompositeDefFields::NoFields => Default::default(),
CompositeDefFields::Unnamed(_) => {
.unzip(),
CompositeIRKind::NoFields => Default::default(),
CompositeIRKind::Unnamed(_) => {
return Err(CodegenError::InvalidCallVariant(call_ty))
}
};
let call_fn_args = result.iter().map(|(call_fn_arg, _)| call_fn_arg);
let call_args = result.iter().map(|(_, call_arg)| call_arg);
let pallet_name = pallet.name();
let call_name = &variant_name;
let struct_name = &struct_def.name;
let call_name = &var.variant_name;
let struct_name = &var.composite.name;
let Some(call_hash) = pallet.call_hash(call_name) else {
return Err(CodegenError::MissingCallMetadata(
pallet_name.into(),
call_name.to_string(),
));
};
let fn_name = format_ident!("{}", var.variant_name.to_snake_case());
// Propagate the documentation just to `TransactionApi` methods, while
// draining the documentation of inner call structures.
let docs = should_gen_docs.then_some(struct_def.docs.take()).flatten();
let docs = &var.composite.docs;
// this converts the composite into a full struct type. No Type Parameters needed here.
let struct_def = type_gen.upcast_composite(&var.composite);
let alias_mod = var.type_alias_mod;
// The call structure's documentation was stripped above.
let call_struct = quote! {
#struct_def
#aliases
#alias_mod
impl #crate_path::blocks::StaticExtrinsic for #struct_name {
const PALLET: &'static str = #pallet_name;
@@ -113,17 +101,15 @@ pub fn generate_calls(
Ok((call_struct, client_fn))
})
.collect::<Result<Vec<_>, _>>()?;
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.unzip();
let call_structs = result.iter().map(|(call_struct, _)| call_struct);
let call_fns = result.iter().map(|(_, client_fn)| client_fn);
let call_type = type_gen.resolve_type_path(call_ty)?;
let call_ty = type_gen.resolve_type(call_ty)?;
let docs = type_gen.docs_from_scale_info(&call_ty.docs);
let call_type = type_gen.resolve_type_path(call_ty);
let call_ty = type_gen.resolve_type(call_ty);
let docs = &call_ty.docs;
let docs = should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
let types_mod_ident = type_gen.types_mod_ident();
Ok(quote! {
#docs
codegen/src/api/constants.rs
+10 -9
View File
@@ -2,10 +2,10 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::types::TypeGenerator;
use heck::ToSnakeCase as _;
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use scale_typegen::TypeGenerator;
use subxt_metadata::PalletMetadata;
use super::CodegenError;
@@ -29,16 +29,13 @@ use super::CodegenError;
///
/// # Arguments
///
/// - `metadata` - Runtime metadata from which the calls are generated.
/// - `type_gen` - The type generator containing all types defined by metadata
/// - `pallet` - Pallet metadata from which the calls are generated.
/// - `types_mod_ident` - The ident of the base module that we can use to access the generated types from.
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the constants are generated.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
pub fn generate_constants(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
types_mod_ident: &syn::Ident,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
// Early return if the pallet has no constants.
if pallet.constants().len() == 0 {
@@ -58,9 +55,11 @@ pub fn generate_constants(
));
};
let return_ty = type_gen.resolve_type_path(constant.ty());
let return_ty = type_gen.resolve_type_path(constant.ty())?;
let docs = constant.docs();
let docs = should_gen_docs
let docs = type_gen
.settings()
.should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
@@ -77,6 +76,8 @@ pub fn generate_constants(
})
.collect::<Result<Vec<_>, _>>()?;
let types_mod_ident = type_gen.types_mod_ident();
Ok(quote! {
pub mod constants {
use super::#types_mod_ident;
codegen/src/api/custom_values.rs
+3 -3
View File
@@ -2,10 +2,9 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use std::collections::HashSet;
use crate::types::TypeGenerator;
use heck::ToSnakeCase as _;
use scale_typegen::TypeGenerator;
use std::collections::HashSet;
use subxt_metadata::{CustomValueMetadata, Metadata};
use proc_macro2::TokenStream as TokenStream2;
@@ -60,6 +59,7 @@ fn generate_custom_value_fn(
let (return_ty, decodable) = if type_is_valid {
let return_ty = type_gen
.resolve_type_path(custom_value.type_id())
.expect("type is in metadata; qed")
.to_token_stream();
let decodable = quote!(#crate_path::custom_values::Yes);
(return_ty, decodable)
codegen/src/api/errors.rs
+6 -6
View File
@@ -4,26 +4,26 @@
use proc_macro2::TokenStream as TokenStream2;
use quote::quote;
use scale_typegen::TypeGenerator;
use subxt_metadata::PalletMetadata;
use crate::types::TypeGenerator;
use super::CodegenError;
/// Generate error type alias from the provided pallet metadata.
pub fn generate_error_type_alias(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let Some(error_ty) = pallet.error_ty_id() else {
return Ok(quote!());
};
let error_type = type_gen.resolve_type_path(error_ty);
let error_ty = type_gen.resolve_type(error_ty);
let error_type = type_gen.resolve_type_path(error_ty)?;
let error_ty = type_gen.resolve_type(error_ty)?;
let docs = &error_ty.docs;
let docs = should_gen_docs
let docs = type_gen
.settings()
.should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
Ok(quote! {
codegen/src/api/events.rs
+26 -33
View File
@@ -2,9 +2,9 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::types::TypeGenerator;
use proc_macro2::TokenStream as TokenStream2;
use quote::quote;
use scale_typegen::TypeGenerator;
use subxt_metadata::PalletMetadata;
use super::CodegenError;
@@ -35,55 +35,48 @@ use super::CodegenError;
///
/// # Arguments
///
/// - `type_gen` - The type generator containing all types defined by metadata.
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the events are generated.
/// - `types_mod_ident` - The ident of the base module that we can use to access the generated types from.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
pub fn generate_events(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
types_mod_ident: &syn::Ident,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
// Early return if the pallet has no events.
let Some(event_ty) = pallet.event_ty_id() else {
return Ok(quote!());
};
let struct_defs = super::generate_structs_from_variants(
type_gen,
types_mod_ident,
event_ty,
|name| name.into(),
"Event",
crate_path,
should_gen_docs,
)?;
let variant_names_and_struct_defs =
super::generate_structs_from_variants(type_gen, event_ty, |name| name.into(), "Event")?;
let event_structs = struct_defs
.iter()
.map(|(variant_name, struct_def, aliases)| {
let pallet_name = pallet.name();
let event_struct = &struct_def.name;
let event_name = variant_name;
let event_structs = variant_names_and_struct_defs.into_iter().map(|var| {
let pallet_name = pallet.name();
let event_struct_name = &var.composite.name;
let event_name = var.variant_name;
let alias_mod = var.type_alias_mod;
let struct_def = type_gen.upcast_composite(&var.composite);
quote! {
#struct_def
#alias_mod
quote! {
#struct_def
#aliases
impl #crate_path::events::StaticEvent for #event_struct {
const PALLET: &'static str = #pallet_name;
const EVENT: &'static str = #event_name;
}
impl #crate_path::events::StaticEvent for #event_struct_name {
const PALLET: &'static str = #pallet_name;
const EVENT: &'static str = #event_name;
}
});
let event_type = type_gen.resolve_type_path(event_ty);
let event_ty = type_gen.resolve_type(event_ty);
}
});
let event_type = type_gen.resolve_type_path(event_ty)?;
let event_ty = type_gen.resolve_type(event_ty)?;
let docs = &event_ty.docs;
let docs = should_gen_docs
let docs = type_gen
.settings()
.should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
let types_mod_ident = type_gen.types_mod_ident();
Ok(quote! {
#docs
codegen/src/api/mod.rs
+111 -140
View File
@@ -12,20 +12,20 @@ mod events;
mod runtime_apis;
mod storage;
use scale_typegen::typegen::ir::type_ir::{CompositeFieldIR, CompositeIR, CompositeIRKind};
use scale_typegen::typegen::type_params::TypeParameters;
use scale_typegen::typegen::type_path::TypePath;
use scale_typegen::TypeGenerator;
use subxt_metadata::Metadata;
use syn::{parse_quote, Ident};
use crate::api::custom_values::generate_custom_values;
use crate::error::CodegenError;
use crate::types::DerivesRegistry;
use crate::{
ir,
types::{CompositeDef, CompositeDefFields, TypeGenerator, TypeSubstitutes},
};
use crate::subxt_type_gen_settings;
use crate::{api::custom_values::generate_custom_values, ir};
use heck::ToSnakeCase as _;
use heck::{ToSnakeCase as _, ToUpperCamelCase};
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use syn::parse_quote;
/// Create the API for interacting with a Substrate runtime.
pub struct RuntimeGenerator {
@@ -60,27 +60,22 @@ impl RuntimeGenerator {
pub fn generate_runtime_types(
&self,
item_mod: syn::ItemMod,
derives: DerivesRegistry,
type_substitutes: TypeSubstitutes,
derives: scale_typegen::DerivesRegistry,
type_substitutes: scale_typegen::TypeSubstitutes,
crate_path: syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let item_mod_attrs = item_mod.attrs.clone();
let item_mod_ir = ir::ItemMod::try_from(item_mod)?;
let settings =
subxt_type_gen_settings(derives, type_substitutes, &crate_path, should_gen_docs);
let type_gen = TypeGenerator::new(self.metadata.types(), &settings);
let types_mod = type_gen.generate_types_mod()?;
let mod_ident = &item_mod_ir.ident;
let rust_items = item_mod_ir.rust_items();
let type_gen = TypeGenerator::new(
self.metadata.types(),
"runtime_types",
type_substitutes,
derives,
crate_path,
should_gen_docs,
);
let types_mod = type_gen.generate_types_mod()?;
Ok(quote! {
#( #item_mod_attrs )*
#[allow(dead_code, unused_imports, non_camel_case_types)]
@@ -114,24 +109,20 @@ impl RuntimeGenerator {
pub fn generate_runtime(
&self,
item_mod: syn::ItemMod,
derives: DerivesRegistry,
type_substitutes: TypeSubstitutes,
derives: scale_typegen::DerivesRegistry,
type_substitutes: scale_typegen::TypeSubstitutes,
crate_path: syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let item_mod_attrs = item_mod.attrs.clone();
let item_mod_ir = ir::ItemMod::try_from(item_mod)?;
let type_gen = TypeGenerator::new(
self.metadata.types(),
"runtime_types",
type_substitutes,
derives,
crate_path.clone(),
should_gen_docs,
);
let settings =
subxt_type_gen_settings(derives, type_substitutes, &crate_path, should_gen_docs);
let type_gen = TypeGenerator::new(self.metadata.types(), &settings);
let types_mod = type_gen.generate_types_mod()?;
let types_mod_ident = types_mod.ident();
let types_mod_ident = type_gen.types_mod_ident();
let pallets_with_mod_names = self
.metadata
.pallets()
@@ -165,39 +156,15 @@ impl RuntimeGenerator {
let modules = pallets_with_mod_names
.iter()
.map(|(pallet, mod_name)| {
let calls = calls::generate_calls(
&type_gen,
pallet,
types_mod_ident,
&crate_path,
should_gen_docs,
)?;
let calls = calls::generate_calls(&type_gen, pallet, &crate_path)?;
let event = events::generate_events(
&type_gen,
pallet,
types_mod_ident,
&crate_path,
should_gen_docs,
)?;
let event = events::generate_events(&type_gen, pallet, &crate_path)?;
let storage_mod = storage::generate_storage(
&type_gen,
pallet,
types_mod_ident,
&crate_path,
should_gen_docs,
)?;
let storage_mod = storage::generate_storage(&type_gen, pallet, &crate_path)?;
let constants_mod = constants::generate_constants(
&type_gen,
pallet,
types_mod_ident,
&crate_path,
should_gen_docs,
)?;
let constants_mod = constants::generate_constants(&type_gen, pallet, &crate_path)?;
let errors = errors::generate_error_type_alias(&type_gen, pallet, should_gen_docs)?;
let errors = errors::generate_error_type_alias(&type_gen, pallet)?;
Ok(quote! {
pub mod #mod_name {
@@ -242,14 +209,14 @@ impl RuntimeGenerator {
&type_gen,
types_mod_ident,
&crate_path,
should_gen_docs,
)?;
// Fetch the paths of the outer enums.
// Substrate exposes those under `kitchensink_runtime`, while Polkadot under `polkadot_runtime`.
let call_path = type_gen.resolve_type_path(self.metadata.outer_enums().call_enum_ty());
let event_path = type_gen.resolve_type_path(self.metadata.outer_enums().event_enum_ty());
let error_path = type_gen.resolve_type_path(self.metadata.outer_enums().error_enum_ty());
let call_path = type_gen.resolve_type_path(self.metadata.outer_enums().call_enum_ty())?;
let event_path = type_gen.resolve_type_path(self.metadata.outer_enums().event_enum_ty())?;
let error_path = type_gen.resolve_type_path(self.metadata.outer_enums().error_enum_ty())?;
let custom_values = generate_custom_values(&self.metadata, &type_gen, &crate_path);
@@ -358,17 +325,14 @@ impl RuntimeGenerator {
/// Return a vector of tuples of variant names and corresponding struct definitions.
pub fn generate_structs_from_variants<F>(
type_gen: &TypeGenerator,
types_mod_ident: &syn::Ident,
type_id: u32,
variant_to_struct_name: F,
error_message_type_name: &str,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<Vec<(String, CompositeDef, TypeAliases)>, CodegenError>
) -> Result<Vec<StructFromVariant>, CodegenError>
where
F: Fn(&str) -> std::borrow::Cow<str>,
{
let ty = type_gen.resolve_type(type_id);
let ty = type_gen.resolve_type(type_id)?;
let scale_info::TypeDef::Variant(variant) = &ty.type_def else {
return Err(CodegenError::InvalidType(error_message_type_name.into()));
@@ -378,84 +342,91 @@ where
.variants
.iter()
.map(|var| {
let mut type_params = TypeParameters::from_scale_info(&[]);
let composite_ir_kind =
type_gen.create_composite_ir_kind(&var.fields, &mut type_params)?;
let struct_name = variant_to_struct_name(&var.name);
let mut composite = CompositeIR::new(
syn::parse_str(&struct_name).expect("enum variant is a valid ident; qed"),
composite_ir_kind,
type_gen.docs_from_scale_info(&var.docs),
);
let fields = CompositeDefFields::from_scale_info_fields(
struct_name.as_ref(),
&var.fields,
&[],
type_gen,
)?;
let alias_module_name = format_ident!("{}", var.name.to_snake_case());
let docs = should_gen_docs.then_some(&*var.docs).unwrap_or_default();
let struct_def = CompositeDef::struct_def(
&ty,
struct_name.as_ref(),
Default::default(),
fields.clone(),
Some(parse_quote!(pub)),
type_gen,
docs,
crate_path,
Some(alias_module_name.clone()),
)?;
let type_aliases = TypeAliases::new(fields, types_mod_ident.clone(), alias_module_name);
Ok((var.name.to_string(), struct_def, type_aliases))
let type_alias_mod = generate_type_alias_mod(&mut composite, type_gen);
Ok(StructFromVariant {
variant_name: var.name.to_string(),
composite,
type_alias_mod,
})
})
.collect()
}
/// Generate the type aliases from a set of enum / struct definitions.
pub struct StructFromVariant {
variant_name: String,
composite: CompositeIR,
type_alias_mod: TokenStream2,
}
/// Modifies the composite, by replacing its types with references to the generated type alias module.
/// Returns the TokenStream of the type alias module.
///
/// The type aliases are used to make the generated code more readable.
#[derive(Debug)]
pub struct TypeAliases {
fields: CompositeDefFields,
types_mod_ident: syn::Ident,
mod_name: syn::Ident,
}
/// E.g a struct like this:
/// ```ignore
/// pub struct SetMaxCodeSize {
/// pub new: ::core::primitive::u32,
/// }
/// ```
/// will be made into this:
/// ```ignore
/// pub struct SetMaxCodeSize {
/// pub new: set_max_code_size::New,
/// }
/// ```
/// And the type alias module will look like this:
/// ```ignore
/// pub mod set_max_code_size {
/// use super::runtime_types;
/// pub type New = ::core::primitive::u32;
/// }
/// ```
pub fn generate_type_alias_mod(
composite: &mut CompositeIR,
type_gen: &TypeGenerator,
) -> TokenStream2 {
let mut aliases: Vec<TokenStream2> = vec![];
let alias_mod_name: Ident = syn::parse_str(&composite.name.to_string().to_snake_case())
.expect("composite name in snake_case should be a valid identifier");
impl TypeAliases {
pub fn new(
fields: CompositeDefFields,
types_mod_ident: syn::Ident,
mod_name: syn::Ident,
) -> Self {
TypeAliases {
fields,
types_mod_ident,
mod_name,
let mut modify_field_to_be_type_alias = |field: &mut CompositeFieldIR, alias_name: Ident| {
let type_path = &field.type_path;
aliases.push(quote!(pub type #alias_name = #type_path;));
let type_alias_path: syn::Path = parse_quote!(#alias_mod_name::#alias_name);
field.type_path = TypePath::from_syn_path(type_alias_path);
};
match &mut composite.kind {
CompositeIRKind::NoFields => {
return quote!(); // no types mod generated for unit structs.
}
}
}
impl quote::ToTokens for TypeAliases {
fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
let has_fields = matches!(&self.fields, CompositeDefFields::Named(fields) if !fields.is_empty())
|| matches!(&self.fields, CompositeDefFields::Unnamed(fields) if !fields.is_empty());
if !has_fields {
return;
}
let visibility: syn::Visibility = parse_quote!(pub);
let aliases = self
.fields
.to_type_aliases_tokens(Some(visibility).as_ref());
let mod_name = &self.mod_name;
let types_mod_ident = &self.types_mod_ident;
tokens.extend(quote! {
pub mod #mod_name {
use super::#types_mod_ident;
#aliases
CompositeIRKind::Named(named) => {
for (name, field) in named.iter_mut() {
let alias_name = format_ident!("{}", name.to_string().to_upper_camel_case());
modify_field_to_be_type_alias(field, alias_name);
}
})
}
}
CompositeIRKind::Unnamed(unnamed) => {
for (i, field) in unnamed.iter_mut().enumerate() {
let alias_name = format_ident!("Field{}", i);
modify_field_to_be_type_alias(field, alias_name);
}
}
};
let types_mod_ident = type_gen.types_mod_ident();
quote!(pub mod #alias_mod_name {
use super::#types_mod_ident;
#( #aliases )*
})
}
codegen/src/api/runtime_apis.rs
+107 -95
View File
@@ -4,140 +4,155 @@
use std::collections::HashSet;
use crate::{types::TypeGenerator, CodegenError};
use heck::ToSnakeCase as _;
use heck::ToUpperCamelCase as _;
use scale_typegen::TypeGenerator;
use subxt_metadata::{Metadata, RuntimeApiMetadata};
use proc_macro2::TokenStream as TokenStream2;
use quote::{format_ident, quote};
use crate::CodegenError;
/// Generates runtime functions for the given API metadata.
fn generate_runtime_api(
api: RuntimeApiMetadata,
type_gen: &TypeGenerator,
types_mod_ident: &syn::Ident,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<(TokenStream2, TokenStream2), CodegenError> {
// Trait name must remain as is (upper case) to identity the runtime call.
let trait_name_str = api.name();
// The snake case for the trait name.
let trait_name_snake = format_ident!("{}", api.name().to_snake_case());
let docs = api.docs();
let docs: TokenStream2 = should_gen_docs
let docs: TokenStream2 = type_gen
.settings()
.should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
let structs_and_methods: Vec<_> = api.methods().map(|method| {
let method_name = format_ident!("{}", method.name());
let method_name_str = method.name();
let structs_and_methods: Vec<_> = api
.methods()
.map(|method| {
let method_name = format_ident!("{}", method.name());
let method_name_str = method.name();
let docs = method.docs();
let docs: TokenStream2 = should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
let docs = method.docs();
let docs: TokenStream2 = type_gen
.settings()
.should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
let mut unique_names = HashSet::new();
let mut unique_aliases = HashSet::new();
let mut unique_names = HashSet::new();
let mut unique_aliases = HashSet::new();
let inputs: Vec<_> = method.inputs().enumerate().map(|(idx, input)| {
// These are method names, which can just be '_', but struct field names can't
// just be an underscore, so fix any such names we find to work in structs.
let inputs: Vec<_> = method
.inputs()
.enumerate()
.map(|(idx, input)| {
// These are method names, which can just be '_', but struct field names can't
// just be an underscore, so fix any such names we find to work in structs.
let mut name = input.name.trim_start_matches('_').to_string();
if name.is_empty() {
name = format!("_{}", idx);
}
while !unique_names.insert(name.clone()) {
// Name is already used, append the index until it is unique.
name = format!("{}_param{}", name, idx);
}
let mut name = input.name.trim_start_matches('_').to_string();
if name.is_empty() {
name = format!("_{}", idx);
}
while !unique_names.insert(name.clone()) {
// Name is already used, append the index until it is unique.
name = format!("{}_param{}", name, idx);
}
let mut alias = name.to_upper_camel_case();
// Note: name is not empty.
if alias.as_bytes()[0].is_ascii_digit() {
alias = format!("Param{}", alias);
}
while !unique_aliases.insert(alias.clone()) {
alias = format!("{}Param{}", alias, idx);
}
let mut alias = name.to_upper_camel_case();
// Note: name is not empty.
if alias.as_bytes()[0].is_ascii_digit() {
alias = format!("Param{}", alias);
}
while !unique_aliases.insert(alias.clone()) {
alias = format!("{}Param{}", alias, idx);
}
let (alias_name, name) = (format_ident!("{alias}"), format_ident!("{name}"));
let (alias_name, name) = (format_ident!("{alias}"), format_ident!("{name}"));
// Generate alias for runtime type.
let ty = type_gen
.resolve_type_path(input.ty)
.expect("runtime api input type is in metadata; qed");
let aliased_param = quote!( pub type #alias_name = #ty; );
// Generate alias for runtime type.
let ty = type_gen.resolve_type_path(input.ty);
let aliased_param = quote!( pub type #alias_name = #ty; );
// Structures are placed on the same level as the alias module.
let struct_ty_path = quote!( #method_name::#alias_name );
let struct_param = quote!(#name: #struct_ty_path);
// Structures are placed on the same level as the alias module.
let struct_ty_path = quote!( #method_name::#alias_name );
let struct_param = quote!(#name: #struct_ty_path);
// Function parameters must be indented by `types`.
let fn_param = quote!(#name: types::#struct_ty_path);
(fn_param, struct_param, name, aliased_param)
})
.collect();
// Function parameters must be indented by `types`.
let fn_param = quote!(#name: types::#struct_ty_path);
(fn_param, struct_param, name, aliased_param)
}).collect();
let fn_params = inputs.iter().map(|(fn_param, _, _, _)| fn_param);
let struct_params = inputs.iter().map(|(_, struct_param, _, _)| struct_param);
let param_names = inputs.iter().map(|(_, _, name, _)| name);
let type_aliases = inputs.iter().map(|(_, _, _, aliased_param)| aliased_param);
let types_mod_ident = type_gen.types_mod_ident();
let fn_params = inputs.iter().map(|(fn_param, _, _, _)| fn_param);
let struct_params = inputs.iter().map(|(_, struct_param, _, _)| struct_param);
let param_names = inputs.iter().map(|(_, _, name, _,)| name);
let type_aliases = inputs.iter().map(|(_, _, _, aliased_param)| aliased_param);
let output = type_gen.resolve_type_path(method.output_ty());
let aliased_module = quote!(
pub mod #method_name {
use super::#types_mod_ident;
#( #type_aliases )*
// Guard the `Output` name against collisions by placing it in a dedicated module.
pub mod output {
let output = type_gen.resolve_type_path(method.output_ty())?;
let aliased_module = quote!(
pub mod #method_name {
use super::#types_mod_ident;
pub type Output = #output;
#( #type_aliases )*
// Guard the `Output` name against collisions by placing it in a dedicated module.
pub mod output {
use super::#types_mod_ident;
pub type Output = #output;
}
}
}
);
);
// From the method metadata generate a structure that holds
// all parameter types. This structure is used with metadata
// to encode parameters to the call via `encode_as_fields_to`.
let derives = type_gen.default_derives();
let struct_name = format_ident!("{}", method.name().to_upper_camel_case());
let struct_input = quote!(
#aliased_module
// From the method metadata generate a structure that holds
// all parameter types. This structure is used with metadata
// to encode parameters to the call via `encode_as_fields_to`.
let derives = type_gen.settings().derives.default_derives();
let struct_name = format_ident!("{}", method.name().to_upper_camel_case());
let struct_input = quote!(
#aliased_module
#derives
pub struct #struct_name {
#( pub #struct_params, )*
}
);
#derives
pub struct #struct_name {
#( pub #struct_params, )*
}
);
let Some(call_hash) = api.method_hash(method.name()) else {
return Err(CodegenError::MissingRuntimeApiMetadata(
trait_name_str.to_owned(),
method_name_str.to_owned(),
))
};
let Some(call_hash) = api.method_hash(method.name()) else {
return Err(CodegenError::MissingRuntimeApiMetadata(
trait_name_str.to_owned(),
method_name_str.to_owned(),
))
};
let method = quote!(
#docs
pub fn #method_name(&self, #( #fn_params, )* ) -> #crate_path::runtime_api::Payload<types::#struct_name, types::#method_name::output::Output> {
#crate_path::runtime_api::Payload::new_static(
#trait_name_str,
#method_name_str,
types::#struct_name { #( #param_names, )* },
[#(#call_hash,)*],
)
}
);
let method = quote!(
#docs
pub fn #method_name(&self, #( #fn_params, )* ) -> #crate_path::runtime_api::Payload<types::#struct_name, types::#method_name::output::Output> {
#crate_path::runtime_api::Payload::new_static(
#trait_name_str,
#method_name_str,
types::#struct_name { #( #param_names, )* },
[#(#call_hash,)*],
)
}
);
Ok((struct_input, method))
}).collect::<Result<_, _>>()?;
Ok((struct_input, method))
})
.collect::<Result<_, _>>()?;
let trait_name = format_ident!("{}", trait_name_str);
let structs = structs_and_methods.iter().map(|(struct_, _)| struct_);
let methods = structs_and_methods.iter().map(|(_, method)| method);
let types_mod_ident = type_gen.types_mod_ident();
let runtime_api = quote!(
pub mod #trait_name_snake {
@@ -175,13 +190,10 @@ pub fn generate_runtime_apis(
type_gen: &TypeGenerator,
types_mod_ident: &syn::Ident,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let runtime_fns: Vec<_> = metadata
.runtime_api_traits()
.map(|api| {
generate_runtime_api(api, type_gen, types_mod_ident, crate_path, should_gen_docs)
})
.map(|api| generate_runtime_api(api, type_gen, crate_path))
.collect::<Result<_, _>>()?;
let runtime_apis_def = runtime_fns.iter().map(|(apis, _)| apis);
codegen/src/api/storage.rs
+28 -28
View File
@@ -2,13 +2,11 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::types::TypeGenerator;
use crate::types::TypePath;
use heck::ToSnakeCase as _;
use heck::ToUpperCamelCase as _;
use heck::{ToSnakeCase as _, ToUpperCamelCase};
use proc_macro2::{Ident, TokenStream as TokenStream2, TokenStream};
use quote::{format_ident, quote};
use scale_info::TypeDef;
use scale_typegen::{typegen::type_path::TypePath, TypeGenerator};
use subxt_metadata::{
PalletMetadata, StorageEntryMetadata, StorageEntryModifier, StorageEntryType,
};
@@ -20,37 +18,26 @@ use super::CodegenError;
///
/// # Arguments
///
/// - `metadata` - Runtime metadata from which the storages are generated.
/// - `type_gen` - The type generator containing all types defined by metadata.
/// - `pallet` - Pallet metadata from which the storages are generated.
/// - `types_mod_ident` - The ident of the base module that we can use to access the generated types from.
/// - `type_gen` - [`scale_typegen::TypeGenerator`] that contains settings and all types from the runtime metadata.
/// - `pallet` - Pallet metadata from which the storage items are generated.
/// - `crate_path` - The crate path under which subxt is located, e.g. `::subxt` when using subxt as a dependency.
pub fn generate_storage(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
types_mod_ident: &syn::Ident,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let Some(storage) = pallet.storage() else {
return Ok(quote!());
};
let (storage_fns, alias_modules): (Vec<_>, Vec<_>) = storage
let (storage_fns, alias_modules): (Vec<TokenStream2>, Vec<TokenStream2>) = storage
.entries()
.iter()
.map(|entry| {
generate_storage_entry_fns(
type_gen,
pallet,
entry,
crate_path,
should_gen_docs,
types_mod_ident,
)
})
.map(|entry| generate_storage_entry_fns(type_gen, pallet, entry, crate_path))
.collect::<Result<Vec<_>, CodegenError>>()?
.into_iter()
.unzip();
let types_mod_ident = type_gen.types_mod_ident();
Ok(quote! {
pub mod storage {
@@ -71,17 +58,18 @@ pub fn generate_storage(
})
}
/// Returns storage entry functions and alias modules.
fn generate_storage_entry_fns(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
storage_entry: &StorageEntryMetadata,
crate_path: &syn::Path,
should_gen_docs: bool,
types_mod_ident: &syn::Ident,
) -> Result<(TokenStream2, TokenStream2), CodegenError> {
let snake_case_name = storage_entry.name().to_snake_case();
let storage_entry_ty = storage_entry.entry_type().value_ty();
let storage_entry_value_ty = type_gen.resolve_type_path(storage_entry_ty);
let storage_entry_value_ty = type_gen
.resolve_type_path(storage_entry_ty)
.expect("storage type is in metadata; qed");
let alias_name = format_ident!("{}", storage_entry.name().to_upper_camel_case());
let alias_module_name = format_ident!("{snake_case_name}");
@@ -89,7 +77,9 @@ fn generate_storage_entry_fns(
let storage_entry_map = |idx, id| {
let ident: Ident = format_ident!("_{}", idx);
let ty_path = type_gen.resolve_type_path(id);
let ty_path = type_gen
.resolve_type_path(id)
.expect("type is in metadata; qed");
let alias_name = format_ident!("Param{}", idx);
let alias_type = primitive_type_alias(&ty_path);
@@ -103,7 +93,11 @@ fn generate_storage_entry_fns(
let keys: Vec<(Ident, TokenStream, TokenStream)> = match storage_entry.entry_type() {
StorageEntryType::Plain(_) => vec![],
StorageEntryType::Map { key_ty, .. } => {
match &type_gen.resolve_type(*key_ty).type_def {
match &type_gen
.resolve_type(*key_ty)
.expect("key type should be present")
.type_def
{
// An N-map; return each of the keys separately.
TypeDef::Tuple(tuple) => tuple
.fields
@@ -128,7 +122,9 @@ fn generate_storage_entry_fns(
};
let docs = storage_entry.docs();
let docs = should_gen_docs
let docs = type_gen
.settings()
.should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
@@ -181,6 +177,7 @@ fn generate_storage_entry_fns(
let alias_types = keys.iter().map(|(_, alias_type, _)| alias_type);
let types_mod_ident = type_gen.types_mod_ident();
// Generate type alias for the return type only, since
// the keys of the storage entry are not explicitly named.
let alias_module = quote! {
@@ -217,9 +214,10 @@ fn primitive_type_alias(type_path: &TypePath) -> TokenStream {
mod tests {
use crate::RuntimeGenerator;
use frame_metadata::v15;
use heck::ToUpperCamelCase as _;
use heck::ToUpperCamelCase;
use quote::{format_ident, quote};
use scale_info::{meta_type, MetaType};
use std::borrow::Cow;
use subxt_metadata::Metadata;
@@ -330,6 +328,8 @@ mod tests {
_0: impl ::std::borrow::Borrow<types::#name_ident::Param0>,
)
);
dbg!(&generated_str);
dbg!(&expected_storage_constructor.to_string());
assert!(generated_str.contains(&expected_storage_constructor.to_string()));
let alias_name = format_ident!("{}", name.to_upper_camel_case());
codegen/src/error.rs
+5 -50
View File
@@ -5,14 +5,12 @@
//! Errors that can be emitted from codegen.
use proc_macro2::{Span, TokenStream as TokenStream2};
use scale_typegen::TypegenError;
/// Error returned when the Codegen cannot generate the runtime API.
#[derive(Debug, thiserror::Error)]
#[non_exhaustive]
pub enum CodegenError {
/// The given metadata type could not be found.
#[error("Could not find type with ID {0} in the type registry; please raise a support issue.")]
TypeNotFound(u32),
/// Cannot fetch the metadata bytes.
#[error("Failed to fetch metadata, make sure that you're pointing at a node which is providing substrate-based metadata: {0}")]
Fetch(#[from] FetchMetadataError),
@@ -22,12 +20,6 @@ pub enum CodegenError {
/// Out of line modules are not supported.
#[error("Out-of-line subxt modules are not supported, make sure you are providing a body to your module: pub mod polkadot {{ ... }}")]
InvalidModule(Span),
/// Expected named or unnamed fields.
#[error("Fields should either be all named or all unnamed, make sure you are providing a valid metadata: {0}")]
InvalidFields(String),
/// Substitute types must have a valid path.
#[error("Type substitution error: {0}")]
TypeSubstitutionError(#[from] TypeSubstitutionError),
/// Invalid type path.
#[error("Invalid type path {0}: {1}")]
InvalidTypePath(String, syn::Error),
@@ -56,6 +48,9 @@ pub enum CodegenError {
"Extrinsic call type could not be found. Make sure you are providing a valid substrate-based metadata"
)]
MissingCallType,
/// Cannot generate types.
#[error("Type Generation failed: {0}")]
TypeGeneration(#[from] TypegenError),
}
impl CodegenError {
@@ -65,7 +60,7 @@ impl CodegenError {
fn get_location(&self) -> Span {
match self {
Self::InvalidModule(span) => *span,
Self::TypeSubstitutionError(err) => err.get_location(),
Self::TypeGeneration(TypegenError::InvalidSubstitute(err)) => err.span,
Self::InvalidTypePath(_, err) => err.span(),
_ => proc_macro2::Span::call_site(),
}
@@ -102,43 +97,3 @@ pub enum FetchMetadataError {
#[error("Other error: {0}")]
Other(String),
}
/// Error attempting to do type substitution.
#[derive(Debug, thiserror::Error)]
#[non_exhaustive]
pub enum TypeSubstitutionError {
/// Substitute "to" type must be an absolute path.
#[error("`substitute_type(with = <path>)` must be a path prefixed with 'crate::' or '::'")]
ExpectedAbsolutePath(Span),
/// Substitute types must have a valid path.
#[error("Substitute types must have a valid path.")]
EmptySubstitutePath(Span),
/// From/To substitution types should use angle bracket generics.
#[error("Expected the from/to type generics to have the form 'Foo<A,B,C..>'.")]
ExpectedAngleBracketGenerics(Span),
/// Source substitute type must be an ident.
#[error("Expected an ident like 'Foo' or 'A' to mark a type to be substituted.")]
InvalidFromType(Span),
/// Target type is invalid.
#[error("Expected an ident like 'Foo' or an absolute concrete path like '::path::to::Bar' for the substitute type.")]
InvalidToType(Span),
/// Target ident doesn't correspond to any source type.
#[error("Cannot find matching param on 'from' type.")]
NoMatchingFromType(Span),
}
impl TypeSubstitutionError {
/// Fetch the location for this error.
// Todo: Probably worth storing location outside of the variant,
// so that there's a common way to set a location for some error.
fn get_location(&self) -> Span {
match self {
TypeSubstitutionError::ExpectedAbsolutePath(span) => *span,
TypeSubstitutionError::EmptySubstitutePath(span) => *span,
TypeSubstitutionError::ExpectedAngleBracketGenerics(span) => *span,
TypeSubstitutionError::InvalidFromType(span) => *span,
TypeSubstitutionError::InvalidToType(span) => *span,
TypeSubstitutionError::NoMatchingFromType(span) => *span,
}
}
}
codegen/src/lib.rs
+177 -34
View File
@@ -9,10 +9,8 @@
#![deny(unused_crate_dependencies, missing_docs)]
mod api;
mod ir;
mod types;
pub mod error;
mod ir;
// These should probably be in a separate crate; they are used by the
// macro and CLI tool, so they only live here because this is a common
@@ -25,14 +23,12 @@ use getrandom as _;
use api::RuntimeGenerator;
use proc_macro2::TokenStream as TokenStream2;
use scale_typegen::{
typegen::settings::substitutes::absolute_path, DerivesRegistry, TypeGeneratorSettings,
TypeSubstitutes, TypegenError,
};
use std::collections::HashMap;
// We expose these only because they are currently needed in subxt-explorer.
// Eventually we'll move the type generation stuff out into a separate crate.
#[doc(hidden)]
pub mod __private {
pub use crate::types::{DerivesRegistry, TypeDefGen, TypeGenerator, TypeSubstitutes};
}
use syn::parse_quote;
// Part of the public interface, so expose:
pub use error::CodegenError;
@@ -72,6 +68,8 @@ pub struct CodegenBuilder {
type_substitutes: HashMap<syn::Path, syn::Path>,
derives_for_type: HashMap<syn::TypePath, Vec<syn::Path>>,
attributes_for_type: HashMap<syn::TypePath, Vec<syn::Attribute>>,
derives_for_type_recursive: HashMap<syn::TypePath, Vec<syn::Path>>,
attributes_for_type_recursive: HashMap<syn::TypePath, Vec<syn::Attribute>>,
}
impl Default for CodegenBuilder {
@@ -90,6 +88,8 @@ impl Default for CodegenBuilder {
type_substitutes: HashMap::new(),
derives_for_type: HashMap::new(),
attributes_for_type: HashMap::new(),
derives_for_type_recursive: HashMap::new(),
attributes_for_type_recursive: HashMap::new(),
}
}
}
@@ -159,33 +159,47 @@ impl CodegenBuilder {
/// Set additional derives for a specific type at the path given.
///
/// # Warning
///
/// For composite types, you may also need to set the same additional derives on all of
/// the contained types as well to avoid compile errors in the generated code.
/// If you want to set the additional derives on all contained types recursively as well,
/// you can set the `recursive` argument to `true`. If you don't do that,
/// there might be compile errors in the generated code, if the derived trait
/// relies on the fact that contained types also implement that trait.
pub fn add_derives_for_type(
&mut self,
ty: syn::TypePath,
derives: impl IntoIterator<Item = syn::Path>,
recursive: bool,
) {
self.derives_for_type.entry(ty).or_default().extend(derives);
if recursive {
self.derives_for_type_recursive
.entry(ty)
.or_default()
.extend(derives);
} else {
self.derives_for_type.entry(ty).or_default().extend(derives);
}
}
/// Set additional attributes for a specific type at the path given.
///
/// # Warning
///
/// For composite types, you may also need to consider contained types and whether they need
/// similar attributes setting.
/// Setting the `recursive` argument to `true` will additionally add the specified
/// attributes to all contained types recursively.
pub fn add_attributes_for_type(
&mut self,
ty: syn::TypePath,
attributes: impl IntoIterator<Item = syn::Attribute>,
recursive: bool,
) {
self.attributes_for_type
.entry(ty)
.or_default()
.extend(attributes);
if recursive {
self.attributes_for_type_recursive
.entry(ty)
.or_default()
.extend(attributes);
} else {
self.attributes_for_type
.entry(ty)
.or_default()
.extend(attributes);
}
}
/// Substitute a type at the given path with some type at the second path. During codegen,
@@ -217,30 +231,39 @@ impl CodegenBuilder {
pub fn generate(self, metadata: Metadata) -> Result<TokenStream2, CodegenError> {
let crate_path = self.crate_path;
let mut derives_registry = if self.use_default_derives {
types::DerivesRegistry::with_default_derives(&crate_path)
let mut derives_registry: DerivesRegistry = if self.use_default_derives {
default_derives(&crate_path)
} else {
types::DerivesRegistry::new()
DerivesRegistry::new()
};
derives_registry.extend_for_all(self.extra_global_derives, self.extra_global_attributes);
derives_registry.add_derives_for_all(self.extra_global_derives);
derives_registry.add_attributes_for_all(self.extra_global_attributes);
for (ty, derives) in self.derives_for_type {
derives_registry.extend_for_type(ty, derives, vec![]);
derives_registry.add_derives_for(ty, derives, false);
}
for (ty, derives) in self.derives_for_type_recursive {
derives_registry.add_derives_for(ty, derives, true);
}
for (ty, attributes) in self.attributes_for_type {
derives_registry.extend_for_type(ty, vec![], attributes);
derives_registry.add_attributes_for(ty, attributes, false);
}
for (ty, attributes) in self.attributes_for_type_recursive {
derives_registry.add_attributes_for(ty, attributes, true);
}
let mut type_substitutes = if self.use_default_substitutions {
types::TypeSubstitutes::with_default_substitutes(&crate_path)
let mut type_substitutes: TypeSubstitutes = if self.use_default_substitutions {
default_substitutes(&crate_path)
} else {
types::TypeSubstitutes::new()
TypeSubstitutes::new()
};
for (from, with) in self.type_substitutes {
let abs_path = with.try_into()?;
type_substitutes.insert(from, abs_path)?;
let abs_path = absolute_path(with).map_err(TypegenError::from)?;
type_substitutes
.insert(from, abs_path)
.map_err(TypegenError::from)?;
}
let item_mod = self.item_mod;
@@ -266,3 +289,123 @@ impl CodegenBuilder {
}
}
}
/// The default [`scale_typegen::TypeGeneratorSettings`], subxt is using for generating code.
/// Useful for emulating subxt's code generation settings from e.g. subxt-explorer.
pub fn default_subxt_type_gen_settings() -> TypeGeneratorSettings {
let crate_path: syn::Path = parse_quote!(::subxt);
let derives = default_derives(&crate_path);
let substitutes = default_substitutes(&crate_path);
subxt_type_gen_settings(derives, substitutes, &crate_path, true)
}
fn subxt_type_gen_settings(
derives: scale_typegen::DerivesRegistry,
substitutes: scale_typegen::TypeSubstitutes,
crate_path: &syn::Path,
should_gen_docs: bool,
) -> TypeGeneratorSettings {
TypeGeneratorSettings {
types_mod_ident: parse_quote!(runtime_types),
should_gen_docs,
derives,
substitutes,
decoded_bits_type_path: Some(parse_quote!(#crate_path::utils::bits::DecodedBits)),
compact_as_type_path: Some(parse_quote!(#crate_path::ext::codec::CompactAs)),
compact_type_path: Some(parse_quote!(#crate_path::ext::codec::Compact)),
insert_codec_attributes: true,
}
}
fn default_derives(crate_path: &syn::Path) -> DerivesRegistry {
let encode_crate_path = quote::quote! { #crate_path::ext::scale_encode }.to_string();
let decode_crate_path = quote::quote! { #crate_path::ext::scale_decode }.to_string();
let derives: [syn::Path; 5] = [
parse_quote!(#crate_path::ext::scale_encode::EncodeAsType),
parse_quote!(#crate_path::ext::scale_decode::DecodeAsType),
parse_quote!(#crate_path::ext::codec::Encode),
parse_quote!(#crate_path::ext::codec::Decode),
parse_quote!(Debug),
];
let attributes: [syn::Attribute; 3] = [
parse_quote!(#[encode_as_type(crate_path = #encode_crate_path)]),
parse_quote!(#[decode_as_type(crate_path = #decode_crate_path)]),
parse_quote!(#[codec(crate = #crate_path::ext::codec)]),
];
let mut derives_registry = DerivesRegistry::new();
derives_registry.add_derives_for_all(derives);
derives_registry.add_attributes_for_all(attributes);
derives_registry
}
fn default_substitutes(crate_path: &syn::Path) -> TypeSubstitutes {
let mut type_substitutes = TypeSubstitutes::new();
let defaults: [(syn::Path, syn::Path); 11] = [
(
parse_quote!(bitvec::order::Lsb0),
parse_quote!(#crate_path::utils::bits::Lsb0),
),
(
parse_quote!(bitvec::order::Msb0),
parse_quote!(#crate_path::utils::bits::Msb0),
),
(
parse_quote!(sp_core::crypto::AccountId32),
parse_quote!(#crate_path::utils::AccountId32),
),
(
parse_quote!(sp_runtime::multiaddress::MultiAddress),
parse_quote!(#crate_path::utils::MultiAddress),
),
(
parse_quote!(primitive_types::H160),
parse_quote!(#crate_path::utils::H160),
),
(
parse_quote!(primitive_types::H256),
parse_quote!(#crate_path::utils::H256),
),
(
parse_quote!(primitive_types::H512),
parse_quote!(#crate_path::utils::H512),
),
(
parse_quote!(frame_support::traits::misc::WrapperKeepOpaque),
parse_quote!(#crate_path::utils::WrapperKeepOpaque),
),
// BTreeMap and BTreeSet impose an `Ord` constraint on their key types. This
// can cause an issue with generated code that doesn't impl `Ord` by default.
// Decoding them to Vec by default (KeyedVec is just an alias for Vec with
// suitable type params) avoids these issues.
(
parse_quote!(BTreeMap),
parse_quote!(#crate_path::utils::KeyedVec),
),
(parse_quote!(BTreeSet), parse_quote!(::std::vec::Vec)),
// The `UncheckedExtrinsic(pub Vec<u8>)` is part of the runtime API calls.
// The inner bytes represent the encoded extrinsic, however when deriving the
// `EncodeAsType` the bytes would be re-encoded. This leads to the bytes
// being altered by adding the length prefix in front of them.
// Note: Not sure if this is appropriate or not. The most recent polkadot.rs file does not have these.
(
parse_quote!(sp_runtime::generic::unchecked_extrinsic::UncheckedExtrinsic),
parse_quote!(#crate_path::utils::UncheckedExtrinsic),
),
];
let defaults = defaults.into_iter().map(|(from, to)| {
(
from,
absolute_path(to).expect("default substitutes above are absolute paths; qed"),
)
});
type_substitutes
.extend(defaults)
.expect("default substitutes can always be parsed; qed");
type_substitutes
}
codegen/src/types/composite_def.rs
-395
View File
@@ -1,395 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::error::CodegenError;
use super::{Derives, Field, TypeDefParameters, TypeGenerator, TypeParameter, TypePath};
use heck::ToUpperCamelCase as _;
use proc_macro2::TokenStream;
use quote::{format_ident, quote};
use scale_info::{form::PortableForm, Type, TypeDef, TypeDefPrimitive};
/// Representation of a type which consists of a set of fields. Used to generate Rust code for
/// either a standalone `struct` definition, or an `enum` variant.
///
/// Fields can either be named or unnamed in either case.
#[derive(Debug)]
pub struct CompositeDef {
/// The name of the `struct`, or the name of the `enum` variant.
pub name: syn::Ident,
/// Generate either a standalone `struct` or an `enum` variant.
pub kind: CompositeDefKind,
/// The fields of the type, which are either all named or all unnamed.
pub fields: CompositeDefFields,
/// Documentation of the composite type as presented in metadata.
pub docs: Option<TokenStream>,
}
impl CompositeDef {
/// Construct a definition which will generate code for a standalone `struct`.
///
/// This is useful for generating structures from call and enum metadata variants;
/// and from all the runtime types of the metadata.
#[allow(clippy::too_many_arguments)]
pub fn struct_def(
ty: &Type<PortableForm>,
ident: &str,
type_params: TypeDefParameters,
fields_def: CompositeDefFields,
field_visibility: Option<syn::Visibility>,
type_gen: &TypeGenerator,
docs: &[String],
crate_path: &syn::Path,
alias_module_name: Option<syn::Ident>,
) -> Result<Self, CodegenError> {
let mut derives = type_gen.type_derives(ty)?;
let fields: Vec<_> = fields_def.field_types().collect();
if fields.len() == 1 {
// any single field wrapper struct with a concrete unsigned int type can derive
// CompactAs.
let field = &fields[0];
if !type_params
.params()
.iter()
.any(|tp| Some(tp.original_name.to_string()) == field.type_name)
{
let ty = type_gen.resolve_type(field.type_id);
if matches!(
ty.type_def,
TypeDef::Primitive(
TypeDefPrimitive::U8
| TypeDefPrimitive::U16
| TypeDefPrimitive::U32
| TypeDefPrimitive::U64
| TypeDefPrimitive::U128
)
) {
derives.insert_codec_compact_as(crate_path)
}
}
}
let name = format_ident!("{}", ident);
let docs_token = Some(quote! { #( #[doc = #docs ] )* });
Ok(Self {
name,
kind: CompositeDefKind::Struct {
derives,
type_params,
field_visibility,
alias_module_name,
},
fields: fields_def,
docs: docs_token,
})
}
/// Construct a definition which will generate code for an `enum` variant.
pub fn enum_variant_def(ident: &str, fields: CompositeDefFields, docs: &[String]) -> Self {
let name = format_ident!("{}", ident);
let docs_token = Some(quote! { #( #[doc = #docs ] )* });
Self {
name,
kind: CompositeDefKind::EnumVariant,
fields,
docs: docs_token,
}
}
}
impl quote::ToTokens for CompositeDef {
fn to_tokens(&self, tokens: &mut TokenStream) {
let name = &self.name;
let docs = &self.docs;
let decl = match &self.kind {
CompositeDefKind::Struct {
derives,
type_params,
field_visibility,
alias_module_name,
} => {
let phantom_data = type_params.unused_params_phantom_data();
let fields: TokenStream = self.fields.to_struct_field_tokens(
phantom_data,
field_visibility.as_ref(),
alias_module_name.as_ref(),
);
let trailing_semicolon = matches!(
self.fields,
CompositeDefFields::NoFields | CompositeDefFields::Unnamed(_)
)
.then(|| quote!(;));
quote! {
#derives
#docs
pub struct #name #type_params #fields #trailing_semicolon
}
}
CompositeDefKind::EnumVariant => {
let fields = self.fields.to_enum_variant_field_tokens();
quote! {
#docs
#name #fields
}
}
};
tokens.extend(decl)
}
}
/// Which kind of composite type are we generating, either a standalone `struct` or an `enum`
/// variant.
#[derive(Debug)]
pub enum CompositeDefKind {
/// Composite type comprising a Rust `struct`.
Struct {
derives: Derives,
type_params: TypeDefParameters,
field_visibility: Option<syn::Visibility>,
alias_module_name: Option<syn::Ident>,
},
/// Comprises a variant of a Rust `enum`.
EnumVariant,
}
/// Encapsulates the composite fields, keeping the invariant that all fields are either named or
/// unnamed.
#[derive(Debug, Clone)]
pub enum CompositeDefFields {
NoFields,
Named(Vec<(syn::Ident, CompositeDefFieldType)>),
Unnamed(Vec<CompositeDefFieldType>),
}
impl CompositeDefFields {
/// Construct a new set of composite fields from the supplied [`::scale_info::Field`]s.
pub fn from_scale_info_fields(
name: &str,
fields: &[Field],
parent_type_params: &[TypeParameter],
type_gen: &TypeGenerator,
) -> Result<Self, CodegenError> {
if fields.is_empty() {
return Ok(Self::NoFields);
}
let mut named_fields = Vec::new();
let mut unnamed_fields = Vec::new();
for field in fields {
let type_path = type_gen.resolve_field_type_path(
field.ty.id,
parent_type_params,
field.type_name.as_deref(),
);
let field_type =
CompositeDefFieldType::new(field.ty.id, type_path, field.type_name.clone());
if let Some(name) = &field.name {
let field_name = format_ident!("{}", name);
named_fields.push((field_name, field_type))
} else {
unnamed_fields.push(field_type)
}
}
if !named_fields.is_empty() && !unnamed_fields.is_empty() {
return Err(CodegenError::InvalidFields(name.into()));
}
let res = if !named_fields.is_empty() {
Self::Named(named_fields)
} else {
Self::Unnamed(unnamed_fields)
};
Ok(res)
}
/// Returns the set of composite fields.
pub fn field_types(&self) -> Box<dyn Iterator<Item = &CompositeDefFieldType> + '_> {
match self {
Self::NoFields => Box::new([].iter()),
Self::Named(named_fields) => Box::new(named_fields.iter().map(|(_, f)| f)),
Self::Unnamed(unnamed_fields) => Box::new(unnamed_fields.iter()),
}
}
/// Generate the code for type aliases which will be used to construct the `struct` or `enum`.
pub fn to_type_aliases_tokens(&self, visibility: Option<&syn::Visibility>) -> TokenStream {
match self {
Self::NoFields => {
quote!()
}
Self::Named(ref fields) => {
let fields = fields.iter().map(|(name, ty)| {
let alias_name = format_ident!("{}", name.to_string().to_upper_camel_case());
// Alias without boxing to have a cleaner call interface.
let ty_path = &ty.type_path;
quote! ( #visibility type #alias_name = #ty_path; )
});
quote!( #( #fields )* )
}
Self::Unnamed(ref fields) => {
let fields = fields.iter().enumerate().map(|(idx, ty)| {
let alias_name = format_ident!("Field{}", idx);
quote! ( #visibility type #alias_name = #ty; )
});
quote!( #( #fields )* )
}
}
}
/// Generate the code for fields which will compose a `struct`.
pub fn to_struct_field_tokens(
&self,
phantom_data: Option<syn::TypePath>,
visibility: Option<&syn::Visibility>,
alias_module_name: Option<&syn::Ident>,
) -> TokenStream {
match self {
Self::NoFields => {
if let Some(phantom_data) = phantom_data {
quote! { ( #phantom_data ) }
} else {
quote! {}
}
}
Self::Named(ref fields) => {
let fields = fields.iter().map(|(name, ty)| {
let compact_attr = ty.compact_attr();
if let Some(alias_module_name) = alias_module_name {
let alias_name =
format_ident!("{}", name.to_string().to_upper_camel_case());
let mut path = quote!( #alias_module_name::#alias_name);
if ty.is_boxed() {
path = quote!( ::std::boxed::Box<#path> );
}
quote! { #compact_attr #visibility #name: #path }
} else {
quote! { #compact_attr #visibility #name: #ty }
}
});
let marker = phantom_data.map(|phantom_data| {
quote!(
#[codec(skip)]
#visibility __subxt_unused_type_params: #phantom_data
)
});
quote!(
{
#( #fields, )*
#marker
}
)
}
Self::Unnamed(ref fields) => {
let fields = fields.iter().enumerate().map(|(idx, ty)| {
let compact_attr = ty.compact_attr();
if let Some(alias_module_name) = alias_module_name {
let alias_name = format_ident!("Field{}", idx);
let mut path = quote!( #alias_module_name::#alias_name);
if ty.is_boxed() {
path = quote!( ::std::boxed::Box<#path> );
}
quote! { #compact_attr #visibility #path }
} else {
quote! { #compact_attr #visibility #ty }
}
});
let marker = phantom_data.map(|phantom_data| {
quote!(
#[codec(skip)]
#visibility #phantom_data
)
});
quote! {
(
#( #fields, )*
#marker
)
}
}
}
}
/// Generate the code for fields which will compose an `enum` variant.
pub fn to_enum_variant_field_tokens(&self) -> TokenStream {
match self {
Self::NoFields => quote! {},
Self::Named(ref fields) => {
let fields = fields.iter().map(|(name, ty)| {
let compact_attr = ty.compact_attr();
quote! { #compact_attr #name: #ty }
});
quote!( { #( #fields, )* } )
}
Self::Unnamed(ref fields) => {
let fields = fields.iter().map(|ty| {
let compact_attr = ty.compact_attr();
quote! { #compact_attr #ty }
});
quote! { ( #( #fields, )* ) }
}
}
}
}
/// Represents a field of a composite type to be generated.
#[derive(Debug, Clone)]
pub struct CompositeDefFieldType {
pub type_id: u32,
pub type_path: TypePath,
pub type_name: Option<String>,
}
impl CompositeDefFieldType {
/// Construct a new [`CompositeDefFieldType`].
pub fn new(type_id: u32, type_path: TypePath, type_name: Option<String>) -> Self {
CompositeDefFieldType {
type_id,
type_path,
type_name,
}
}
/// Returns `true` if the field is a [`::std::boxed::Box`].
pub fn is_boxed(&self) -> bool {
// Use the type name to detect a `Box` field.
// Should be updated once `Box` types are no longer erased:
// https://github.com/paritytech/scale-info/pull/82
matches!(&self.type_name, Some(ty_name) if ty_name.contains("Box<"))
}
/// Returns the `#[codec(compact)]` attribute if the type is compact.
fn compact_attr(&self) -> Option<TokenStream> {
self.type_path
.is_compact()
.then(|| quote!( #[codec(compact)] ))
}
}
impl quote::ToTokens for CompositeDefFieldType {
fn to_tokens(&self, tokens: &mut TokenStream) {
let ty_path = &self.type_path;
if self.is_boxed() {
tokens.extend(quote! { ::std::boxed::Box<#ty_path> })
} else {
tokens.extend(quote! { #ty_path })
};
}
}
codegen/src/types/derives.rs
-199
View File
@@ -1,199 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use syn::{parse_quote, Path};
use std::collections::{HashMap, HashSet};
/// A struct containing the derives that we'll be applying to types;
/// a combination of some common derives for all types, plus type
/// specific derives.
#[derive(Debug, Clone)]
pub struct DerivesRegistry {
default_derives: Derives,
specific_type_derives: HashMap<syn::TypePath, Derives>,
}
impl Default for DerivesRegistry {
fn default() -> Self {
Self::new()
}
}
impl DerivesRegistry {
/// Creates a new `DerivesRegistry` with no default derives.
pub fn new() -> Self {
Self {
default_derives: Derives::new(),
specific_type_derives: Default::default(),
}
}
/// Creates a new `DerivesRegistry` with default derives.
///
/// The `crate_path` denotes the `subxt` crate access path in the
/// generated code.
pub fn with_default_derives(crate_path: &syn::Path) -> Self {
Self {
default_derives: Derives::with_defaults(crate_path),
specific_type_derives: Default::default(),
}
}
/// Insert derives to be applied to all generated types.
pub fn extend_for_all(
&mut self,
derives: impl IntoIterator<Item = syn::Path>,
attributes: impl IntoIterator<Item = syn::Attribute>,
) {
self.default_derives.derives.extend(derives);
self.default_derives.attributes.extend(attributes);
}
/// Insert derives to be applied to a specific generated type.
pub fn extend_for_type(
&mut self,
ty: syn::TypePath,
derives: impl IntoIterator<Item = syn::Path>,
attributes: impl IntoIterator<Item = syn::Attribute>,
) {
let type_derives = self.specific_type_derives.entry(ty).or_default();
type_derives.derives.extend(derives);
type_derives.attributes.extend(attributes);
}
/// Returns the derives to be applied to all generated types.
pub fn default_derives(&self) -> &Derives {
&self.default_derives
}
/// Resolve the derives for a generated type. Includes:
/// - The default derives for all types e.g. `scale::Encode, scale::Decode`
/// - Any user-defined derives for all types via `generated_type_derives`
/// - Any user-defined derives for this specific type
pub fn resolve(&self, ty: &syn::TypePath) -> Derives {
let mut resolved_derives = self.default_derives.clone();
if let Some(specific) = self.specific_type_derives.get(ty) {
resolved_derives.extend_from(specific.clone());
}
resolved_derives
}
}
/// A struct storing the set of derives and derive attributes that we'll apply
/// to generated types.
#[derive(Debug, Clone)]
pub struct Derives {
derives: HashSet<syn::Path>,
attributes: HashSet<syn::Attribute>,
}
impl Default for Derives {
fn default() -> Self {
Self::new()
}
}
impl FromIterator<syn::Path> for Derives {
fn from_iter<T: IntoIterator<Item = Path>>(iter: T) -> Self {
let derives = iter.into_iter().collect();
Self {
derives,
attributes: HashSet::new(),
}
}
}
impl Derives {
/// Creates an empty instance of `Derives` (with no default derives).
pub fn new() -> Self {
Self {
derives: HashSet::new(),
attributes: HashSet::new(),
}
}
/// Creates a new instance of `Derives` with the `crate_path` prepended
/// to the set of default derives that reside in `subxt`.
pub fn with_defaults(crate_path: &syn::Path) -> Self {
let mut derives = HashSet::new();
let mut attributes = HashSet::new();
derives.insert(syn::parse_quote!(#crate_path::ext::scale_encode::EncodeAsType));
let encode_crate_path = quote::quote! { #crate_path::ext::scale_encode }.to_string();
attributes.insert(syn::parse_quote!(#[encode_as_type(crate_path = #encode_crate_path)]));
derives.insert(syn::parse_quote!(#crate_path::ext::scale_decode::DecodeAsType));
let decode_crate_path = quote::quote! { #crate_path::ext::scale_decode }.to_string();
attributes.insert(syn::parse_quote!(#[decode_as_type(crate_path = #decode_crate_path)]));
derives.insert(syn::parse_quote!(#crate_path::ext::codec::Encode));
derives.insert(syn::parse_quote!(#crate_path::ext::codec::Decode));
attributes.insert(syn::parse_quote!(#[codec(crate = #crate_path::ext::codec)]));
derives.insert(syn::parse_quote!(Debug));
Self {
derives,
attributes,
}
}
/// Extend this set of `Derives` from another.
pub fn extend_from(&mut self, other: Derives) {
self.derives.extend(other.derives);
self.attributes.extend(other.attributes);
}
/// Add `#crate_path::ext::codec::CompactAs` to the derives.
pub fn insert_codec_compact_as(&mut self, crate_path: &syn::Path) {
self.insert_derive(parse_quote!(#crate_path::ext::codec::CompactAs));
}
/// Extend the set of derives by providing an iterator of paths to derive macros.
pub fn extend(&mut self, derives: impl Iterator<Item = syn::Path>) {
for derive in derives {
self.insert_derive(derive)
}
}
/// Insert a single derive.
pub fn insert_derive(&mut self, derive: syn::Path) {
self.derives.insert(derive);
}
/// Insert a single attribute to be applied to types.
pub fn insert_attribute(&mut self, attribute: syn::Attribute) {
self.attributes.insert(attribute);
}
}
impl quote::ToTokens for Derives {
fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
if !self.derives.is_empty() {
let mut sorted = self.derives.iter().cloned().collect::<Vec<_>>();
sorted.sort_by(|a, b| {
quote::quote!(#a)
.to_string()
.cmp(&quote::quote!(#b).to_string())
});
tokens.extend(quote::quote! {
#[derive(#( #sorted ),*)]
})
}
if !self.attributes.is_empty() {
let mut sorted = self.attributes.iter().cloned().collect::<Vec<_>>();
sorted.sort_by(|a, b| {
quote::quote!(#a)
.to_string()
.cmp(&quote::quote!(#b).to_string())
});
tokens.extend(quote::quote! {
#( #sorted )*
})
}
}
}
codegen/src/types/mod.rs
-350
View File
@@ -1,350 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
mod composite_def;
mod derives;
mod substitutes;
#[cfg(test)]
mod tests;
mod type_def;
mod type_def_params;
mod type_path;
use proc_macro2::{Ident, Span, TokenStream};
use quote::{quote, ToTokens};
use scale_info::{form::PortableForm, PortableRegistry, Type, TypeDef};
use std::collections::BTreeMap;
use crate::error::CodegenError;
pub use self::{
composite_def::{CompositeDef, CompositeDefFieldType, CompositeDefFields},
derives::{Derives, DerivesRegistry},
substitutes::TypeSubstitutes,
type_def::TypeDefGen,
type_def_params::TypeDefParameters,
type_path::{TypeParameter, TypePath, TypePathType},
};
pub type Field = scale_info::Field<PortableForm>;
/// Generate a Rust module containing all types defined in the supplied [`PortableRegistry`].
#[derive(Debug)]
pub struct TypeGenerator<'a> {
/// The name of the module which will contain the generated types.
types_mod_ident: Ident,
/// Registry of type definitions to be transformed into Rust type definitions.
type_registry: &'a PortableRegistry,
/// User defined overrides for generated types.
type_substitutes: TypeSubstitutes,
/// Set of derives with which to annotate generated types.
derives: DerivesRegistry,
/// The `subxt` crate access path in the generated code.
crate_path: syn::Path,
/// True if codegen should generate the documentation for the API.
should_gen_docs: bool,
}
impl<'a> TypeGenerator<'a> {
/// Construct a new [`TypeGenerator`].
pub fn new(
type_registry: &'a PortableRegistry,
root_mod: &'static str,
type_substitutes: TypeSubstitutes,
derives: DerivesRegistry,
crate_path: syn::Path,
should_gen_docs: bool,
) -> Self {
let root_mod_ident = Ident::new(root_mod, Span::call_site());
Self {
types_mod_ident: root_mod_ident,
type_registry,
type_substitutes,
derives,
crate_path,
should_gen_docs,
}
}
/// Generate a module containing all types defined in the supplied type registry.
pub fn generate_types_mod(&self) -> Result<Module, CodegenError> {
let root_mod_ident = &self.types_mod_ident;
let mut root_mod = Module::new(root_mod_ident.clone(), root_mod_ident.clone());
for ty in &self.type_registry.types {
let path = &ty.ty.path;
// Don't generate a type if it was substituted - the target type might
// not be in the type registry + our resolution already performs the substitution.
if self.type_substitutes.contains(path) {
continue;
}
let namespace = path.namespace();
// prelude types e.g. Option/Result have no namespace, so we don't generate them
if namespace.is_empty() {
continue;
}
// Lazily create submodules for the encountered namespace path, if they don't exist
let innermost_module = namespace
.iter()
.map(|segment| Ident::new(segment, Span::call_site()))
.fold(&mut root_mod, |module, ident| {
module
.children
.entry(ident.clone())
.or_insert_with(|| Module::new(ident, root_mod_ident.clone()))
});
innermost_module
.types
.insert(path.clone(), TypeDefGen::from_type(&ty.ty, self)?);
}
Ok(root_mod)
}
/// # Panics
///
/// If no type with the given id found in the type registry.
pub fn resolve_type(&self, id: u32) -> Type<PortableForm> {
self.type_registry
.resolve(id)
.unwrap_or_else(|| panic!("No type with id {id} found"))
.clone()
}
/// Get the type path for a field of a struct or an enum variant, providing any generic
/// type parameters from the containing type. This is for identifying where a generic type
/// parameter is used in a field type e.g.
///
/// ```rust
/// struct S<T> {
/// a: T, // `T` is the "parent" type param from the containing type.
/// b: Vec<Option<T>>, // nested use of generic type param `T`.
/// }
/// ```
///
/// This allows generating the correct generic field type paths.
///
/// # Panics
///
/// If no type with the given id found in the type registry.
pub fn resolve_field_type_path(
&self,
id: u32,
parent_type_params: &[TypeParameter],
original_name: Option<&str>,
) -> TypePath {
self.resolve_type_path_recurse(id, true, parent_type_params, original_name)
}
/// Get the type path for the given type identifier.
///
/// # Panics
///
/// If no type with the given id found in the type registry.
pub fn resolve_type_path(&self, id: u32) -> TypePath {
self.resolve_type_path_recurse(id, false, &[], None)
}
/// Visit each node in a possibly nested type definition to produce a type path.
///
/// e.g `Result<GenericStruct<NestedGenericStruct<T>>, String>`
///
/// if `original_name` is `Some(original_name)`, the resolved type needs to have the same `original_name`.
fn resolve_type_path_recurse(
&self,
id: u32,
is_field: bool,
parent_type_params: &[TypeParameter],
original_name: Option<&str>,
) -> TypePath {
if let Some(parent_type_param) = parent_type_params.iter().find(|tp| {
tp.concrete_type_id == id
&& original_name.map_or(true, |original_name| tp.original_name == original_name)
}) {
return TypePath::from_parameter(parent_type_param.clone());
}
let mut ty = self.resolve_type(id);
if ty.path.ident() == Some("Cow".to_string()) {
ty = self.resolve_type(
ty.type_params[0]
.ty
.expect("type parameters to Cow are not expected to be skipped; qed")
.id,
)
}
let params: Vec<_> = ty
.type_params
.iter()
.filter_map(|f| {
f.ty.map(|f| self.resolve_type_path_recurse(f.id, false, parent_type_params, None))
})
.collect();
let ty = match &ty.type_def {
TypeDef::Composite(_) | TypeDef::Variant(_) => {
if let Some(ty) = self
.type_substitutes
.for_path_with_params(&ty.path, &params)
{
ty
} else {
TypePathType::from_type_def_path(&ty.path, self.types_mod_ident.clone(), params)
}
}
TypeDef::Primitive(primitive) => TypePathType::Primitive {
def: primitive.clone(),
},
TypeDef::Array(arr) => TypePathType::Array {
len: arr.len as usize,
of: Box::new(self.resolve_type_path_recurse(
arr.type_param.id,
false,
parent_type_params,
None,
)),
},
TypeDef::Sequence(seq) => TypePathType::Vec {
of: Box::new(self.resolve_type_path_recurse(
seq.type_param.id,
false,
parent_type_params,
None,
)),
},
TypeDef::Tuple(tuple) => TypePathType::Tuple {
elements: tuple
.fields
.iter()
.map(|f| self.resolve_type_path_recurse(f.id, false, parent_type_params, None))
.collect(),
},
TypeDef::Compact(compact) => TypePathType::Compact {
inner: Box::new(self.resolve_type_path_recurse(
compact.type_param.id,
false,
parent_type_params,
None,
)),
is_field,
crate_path: self.crate_path.clone(),
},
TypeDef::BitSequence(bitseq) => TypePathType::BitVec {
bit_order_type: Box::new(self.resolve_type_path_recurse(
bitseq.bit_order_type.id,
false,
parent_type_params,
None,
)),
bit_store_type: Box::new(self.resolve_type_path_recurse(
bitseq.bit_store_type.id,
false,
parent_type_params,
None,
)),
crate_path: self.crate_path.clone(),
},
};
TypePath::from_type(ty)
}
/// Returns the derives to be applied to all generated types.
pub fn default_derives(&self) -> &Derives {
self.derives.default_derives()
}
/// Returns the type registry.
pub fn types(&self) -> &PortableRegistry {
self.type_registry
}
/// Returns the derives to be applied to a generated type.
pub fn type_derives(&self, ty: &Type<PortableForm>) -> Result<Derives, CodegenError> {
let joined_path = ty.path.segments.join("::");
let ty_path: syn::TypePath = syn::parse_str(&joined_path)
.map_err(|e| CodegenError::InvalidTypePath(joined_path, e))?;
Ok(self.derives.resolve(&ty_path))
}
/// The name of the module which will contain the generated types.
pub fn types_mod_ident(&self) -> &Ident {
&self.types_mod_ident
}
/// The `subxt` crate access path in the generated code.
pub fn crate_path(&self) -> &syn::Path {
&self.crate_path
}
/// True if codegen should generate the documentation for the API.
pub fn should_gen_docs(&self) -> bool {
self.should_gen_docs
}
}
/// Represents a Rust `mod`, containing generated types and child `mod`s.
#[derive(Debug)]
pub struct Module {
name: Ident,
root_mod: Ident,
children: BTreeMap<Ident, Module>,
types: BTreeMap<scale_info::Path<PortableForm>, TypeDefGen>,
}
impl ToTokens for Module {
fn to_tokens(&self, tokens: &mut TokenStream) {
let name = &self.name;
let root_mod = &self.root_mod;
let modules = self.children.values();
let types = self.types.values().clone();
tokens.extend(quote! {
pub mod #name {
use super::#root_mod;
#( #modules )*
#( #types )*
}
})
}
}
impl Module {
/// Create a new [`Module`], with a reference to the root `mod` for resolving type paths.
pub(crate) fn new(name: Ident, root_mod: Ident) -> Self {
Self {
name,
root_mod,
children: BTreeMap::new(),
types: BTreeMap::new(),
}
}
/// Returns the module ident.
pub fn ident(&self) -> &Ident {
&self.name
}
/// Returns this `Module`s child `mod`s.
pub fn children(&self) -> impl Iterator<Item = (&Ident, &Module)> {
self.children.iter()
}
/// Returns the generated types.
pub fn types(&self) -> impl Iterator<Item = (&scale_info::Path<PortableForm>, &TypeDefGen)> {
self.types.iter()
}
/// Returns the root `mod` used for resolving type paths.
pub fn root_mod(&self) -> &Ident {
&self.root_mod
}
}
codegen/src/types/substitutes.rs
-529
View File
@@ -1,529 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::error::TypeSubstitutionError;
use std::{borrow::Borrow, collections::HashMap};
use syn::{parse_quote, spanned::Spanned as _};
use super::{TypePath, TypePathType};
/// A map of type substitutes. We match on the paths to generated types in order
/// to figure out when to swap said type with some provided substitute.
#[derive(Debug)]
pub struct TypeSubstitutes {
substitutes: HashMap<PathSegments, Substitute>,
}
#[derive(Debug)]
struct Substitute {
path: syn::Path,
param_mapping: TypeParamMapping,
}
#[derive(Debug)]
enum TypeParamMapping {
// Pass any generics from source to target type
PassThrough,
// Replace any ident seen in the path with the input generic type at this index
Specified(Vec<(syn::Ident, usize)>),
}
macro_rules! path_segments {
($($ident: ident)::*) => {
PathSegments(
[$(stringify!($ident)),*].into_iter().map(String::from).collect::<Vec<_>>()
)
}
}
impl Default for TypeSubstitutes {
fn default() -> Self {
Self::new()
}
}
impl TypeSubstitutes {
/// Creates a new `TypeSubstitutes` with no default derives.
pub fn new() -> Self {
Self {
substitutes: HashMap::new(),
}
}
/// Creates a new `TypeSubstitutes` with some default substitutions in place.
///
/// The `crate_path` denotes the `subxt` crate access path in the
/// generated code.
pub fn with_default_substitutes(crate_path: &syn::Path) -> Self {
// Some hardcoded default type substitutes, can be overridden by user
let defaults = [
(
path_segments!(bitvec::order::Lsb0),
parse_quote!(#crate_path::utils::bits::Lsb0),
),
(
path_segments!(bitvec::order::Msb0),
parse_quote!(#crate_path::utils::bits::Msb0),
),
(
path_segments!(sp_core::crypto::AccountId32),
parse_quote!(#crate_path::utils::AccountId32),
),
(
path_segments!(sp_runtime::multiaddress::MultiAddress),
parse_quote!(#crate_path::utils::MultiAddress),
),
(
path_segments!(primitive_types::H160),
parse_quote!(#crate_path::utils::H160),
),
(
path_segments!(primitive_types::H256),
parse_quote!(#crate_path::utils::H256),
),
(
path_segments!(primitive_types::H512),
parse_quote!(#crate_path::utils::H512),
),
(
path_segments!(frame_support::traits::misc::WrapperKeepOpaque),
parse_quote!(#crate_path::utils::WrapperKeepOpaque),
),
// BTreeMap and BTreeSet impose an `Ord` constraint on their key types. This
// can cause an issue with generated code that doesn't impl `Ord` by default.
// Decoding them to Vec by default (KeyedVec is just an alias for Vec with
// suitable type params) avoids these issues.
(
path_segments!(BTreeMap),
parse_quote!(#crate_path::utils::KeyedVec),
),
(path_segments!(BTreeSet), parse_quote!(::std::vec::Vec)),
// The `UncheckedExtrinsic(pub Vec<u8>)` is part of the runtime API calls.
// The inner bytes represent the encoded extrinsic, however when deriving the
// `EncodeAsType` the bytes would be re-encoded. This leads to the bytes
// being altered by adding the length prefix in front of them.
(
path_segments!(sp_runtime::generic::unchecked_extrinsic::UncheckedExtrinsic),
parse_quote!(#crate_path::utils::UncheckedExtrinsic),
),
];
let default_substitutes = defaults
.into_iter()
.map(|(k, v)| {
(
k,
Substitute {
path: v,
param_mapping: TypeParamMapping::PassThrough,
},
)
})
.collect();
Self {
substitutes: default_substitutes,
}
}
/// Insert the given substitution, overwriting any other with the same path.
pub fn insert(
&mut self,
source: syn::Path,
target: AbsolutePath,
) -> Result<(), TypeSubstitutionError> {
let (key, val) = TypeSubstitutes::parse_path_substitution(source, target.0)?;
self.substitutes.insert(key, val);
Ok(())
}
/// Only insert the given substitution if a substitution at that path doesn't
/// already exist.
pub fn insert_if_not_exists(
&mut self,
source: syn::Path,
target: AbsolutePath,
) -> Result<(), TypeSubstitutionError> {
let (key, val) = TypeSubstitutes::parse_path_substitution(source, target.0)?;
self.substitutes.entry(key).or_insert(val);
Ok(())
}
/// Add a bunch of source to target type substitutions.
pub fn extend(
&mut self,
elems: impl IntoIterator<Item = (syn::Path, AbsolutePath)>,
) -> Result<(), TypeSubstitutionError> {
for (source, target) in elems.into_iter() {
let (key, val) = TypeSubstitutes::parse_path_substitution(source, target.0)?;
self.substitutes.insert(key, val);
}
Ok(())
}
/// Given a source and target path, parse the type params to work out the mapping from
/// source to target, and output the source => substitution mapping that we work out from this.
fn parse_path_substitution(
src_path: syn::Path,
target_path: syn::Path,
) -> Result<(PathSegments, Substitute), TypeSubstitutionError> {
let param_mapping = Self::parse_path_param_mapping(&src_path, &target_path)?;
Ok((
PathSegments::from(&src_path),
Substitute {
// Note; at this point, target_path might have some generics still. These
// might be hardcoded types that we want to keep, so leave them here for now.
path: target_path,
param_mapping,
},
))
}
/// Given a source and target path, parse the type params to work out the mapping from
/// source to target, and return it.
fn parse_path_param_mapping(
src_path: &syn::Path,
target_path: &syn::Path,
) -> Result<TypeParamMapping, TypeSubstitutionError> {
let Some(syn::PathSegment {
arguments: src_path_args,
..
}) = src_path.segments.last()
else {
return Err(TypeSubstitutionError::EmptySubstitutePath(src_path.span()));
};
let Some(syn::PathSegment {
arguments: target_path_args,
..
}) = target_path.segments.last()
else {
return Err(TypeSubstitutionError::EmptySubstitutePath(
target_path.span(),
));
};
// Get hold of the generic args for the "from" type, erroring if they aren't valid.
let source_args = match src_path_args {
syn::PathArguments::None => {
// No generics defined on the source type:
Vec::new()
}
syn::PathArguments::AngleBracketed(args) => {
// We have generics like <A,B> defined on the source type (error for any non-ident type):
args.args
.iter()
.map(|arg| match get_valid_from_substitution_type(arg) {
Some(ident) => Ok(ident),
None => Err(TypeSubstitutionError::InvalidFromType(arg.span())),
})
.collect::<Result<Vec<_>, _>>()?
}
syn::PathArguments::Parenthesized(args) => {
// Generics like (A,B) -> defined; not allowed:
return Err(TypeSubstitutionError::ExpectedAngleBracketGenerics(
args.span(),
));
}
};
// Get hold of the generic args for the "to" type, erroring if they aren't valid.
let target_args = match target_path_args {
syn::PathArguments::None => {
// No generics on target.
Vec::new()
}
syn::PathArguments::AngleBracketed(args) => {
// We have generics like <A,B> defined on the target type.
args.args
.iter()
.map(|arg| match get_valid_to_substitution_type(arg) {
Some(arg) => Ok(arg),
None => Err(TypeSubstitutionError::InvalidToType(arg.span())),
})
.collect::<Result<Vec<_>, _>>()?
}
syn::PathArguments::Parenthesized(args) => {
// Generics like (A,B) -> defined; not allowed:
return Err(TypeSubstitutionError::ExpectedAngleBracketGenerics(
args.span(),
));
}
};
// If no generics defined on source or target, we just apply any concrete generics
// to the substitute type.
if source_args.is_empty() && target_args.is_empty() {
return Ok(TypeParamMapping::PassThrough);
}
// Make a note of the idents in the source args and their indexes.
let mapping = source_args
.into_iter()
.enumerate()
.map(|(idx, ident)| (ident.clone(), idx))
.collect();
Ok(TypeParamMapping::Specified(mapping))
}
/// Given a source type path, return whether a substitute exists for it.
pub fn contains(&self, path: impl Into<PathSegments>) -> bool {
self.substitutes.contains_key(&path.into())
}
/// Given a source type path and the resolved, supplied type parameters,
/// return a new path and optionally overwritten type parameters.
pub fn for_path_with_params(
&self,
path: impl Into<PathSegments>,
params: &[TypePath],
) -> Option<TypePathType> {
// If we find a substitute type, we'll take the substitute path, and
// swap any idents with their new concrete types.
fn replace_params(
mut substitute_path: syn::Path,
params: &[TypePath],
mapping: &TypeParamMapping,
) -> TypePathType {
match mapping {
// We need to map the input params to the output params somehow:
TypeParamMapping::Specified(mapping) => {
// A map from ident name to replacement path.
let replacement_map: Vec<(&syn::Ident, &TypePath)> = mapping
.iter()
.filter_map(|(ident, idx)| params.get(*idx).map(|param| (ident, param)))
.collect();
// Replace params in our substitute path with the incoming ones as needed.
// No need if no replacements given.
if !replacement_map.is_empty() {
replace_path_params_recursively(&mut substitute_path, &replacement_map);
}
TypePathType::Path {
path: substitute_path,
params: Vec::new(),
}
}
// No mapping; just hand back the substitute path and input params.
TypeParamMapping::PassThrough => TypePathType::Path {
path: substitute_path,
params: params.to_vec(),
},
}
}
let path = path.into();
self.substitutes
.get(&path)
.map(|sub| replace_params(sub.path.clone(), params, &sub.param_mapping))
}
}
/// Identifiers joined by the `::` separator.
///
/// We use this as a common denominator, since we need a consistent keys for both
/// `syn::TypePath` and `scale_info::ty::path::Path` types.
#[derive(Debug, Hash, PartialEq, Eq)]
pub struct PathSegments(Vec<String>);
impl From<&syn::Path> for PathSegments {
fn from(path: &syn::Path) -> Self {
PathSegments(path.segments.iter().map(|x| x.ident.to_string()).collect())
}
}
impl<T: scale_info::form::Form> From<&scale_info::Path<T>> for PathSegments {
fn from(path: &scale_info::Path<T>) -> Self {
PathSegments(
path.segments
.iter()
.map(|x| x.as_ref().to_owned())
.collect(),
)
}
}
/// Dig through a `syn::TypePath` (this is provided by the user in a type substitution definition as the "to" type) and
/// swap out any type params which match the idents given in the "from" type with the corresponding concrete types.
///
/// eg if we have:
///
/// ```text
/// from = sp_runtime::MultiAddress<A, B>,
/// to = ::subxt::utils::Static<::sp_runtime::MultiAddress<A, B>>
/// ```
///
/// And we encounter a `sp_runtime::MultiAddress<Foo, Bar>`, then we will pass the `::sp_runtime::MultiAddress<A, B>`
/// type param value into this call to turn it into `::sp_runtime::MultiAddress<Foo, Bar>`.
fn replace_path_params_recursively<I: Borrow<syn::Ident>, P: Borrow<TypePath>>(
path: &mut syn::Path,
params: &Vec<(I, P)>,
) {
for segment in &mut path.segments {
let syn::PathArguments::AngleBracketed(args) = &mut segment.arguments else {
continue;
};
for arg in &mut args.args {
let syn::GenericArgument::Type(ty) = arg else {
continue;
};
let syn::Type::Path(path) = ty else {
continue;
};
if let Some(ident) = get_ident_from_type_path(path) {
if let Some((_, replacement)) = params.iter().find(|(i, _)| ident == i.borrow()) {
*ty = replacement.borrow().to_syn_type();
continue;
}
}
replace_path_params_recursively(&mut path.path, params);
}
}
}
/// Given a "to" type in a type substitution, return the TypePath inside or None if
/// it's not a valid "to" type.
fn get_valid_to_substitution_type(arg: &syn::GenericArgument) -> Option<&syn::TypePath> {
let syn::GenericArgument::Type(syn::Type::Path(type_path)) = arg else {
// We are looking for a type, not a lifetime or anything else
return None;
};
Some(type_path)
}
/// Given a "from" type in a type substitution, return the Ident inside or None if
/// it's not a valid "from" type.
fn get_valid_from_substitution_type(arg: &syn::GenericArgument) -> Option<&syn::Ident> {
let syn::GenericArgument::Type(syn::Type::Path(type_path)) = arg else {
// We are looking for a type, not a lifetime or anything else
return None;
};
get_ident_from_type_path(type_path)
}
/// Given a type path, return the single ident representing it if that's all it is.
fn get_ident_from_type_path(type_path: &syn::TypePath) -> Option<&syn::Ident> {
if type_path.qself.is_some() {
// No "<Foo as Bar>" type thing
return None;
}
if type_path.path.leading_colon.is_some() {
// No leading "::"
return None;
}
if type_path.path.segments.len() > 1 {
// The path should just be a single ident, not multiple
return None;
}
let Some(segment) = type_path.path.segments.last() else {
// Get the single ident (should be infallible)
return None;
};
if !segment.arguments.is_empty() {
// The ident shouldn't have any of it's own generic args like A<B, C>
return None;
}
Some(&segment.ident)
}
/// Whether a path is absolute - starts with `::` or `crate`.
fn is_absolute(path: &syn::Path) -> bool {
path.leading_colon.is_some()
|| path
.segments
.first()
.map_or(false, |segment| segment.ident == "crate")
}
pub struct AbsolutePath(pub syn::Path);
impl TryFrom<syn::Path> for AbsolutePath {
type Error = TypeSubstitutionError;
fn try_from(value: syn::Path) -> Result<Self, Self::Error> {
if is_absolute(&value) {
Ok(AbsolutePath(value))
} else {
Err(TypeSubstitutionError::ExpectedAbsolutePath(value.span()))
}
}
}
#[cfg(test)]
mod test {
use super::*;
macro_rules! syn_path {
($path:path) => {{
let path: syn::Path = syn::parse_quote!($path);
path
}};
}
macro_rules! type_path {
($path:path) => {{
let path: syn::Path = syn::parse_quote!($path);
TypePath::from_syn_path(path)
}};
}
fn ident(name: &'static str) -> syn::Ident {
syn::Ident::new(name, proc_macro2::Span::call_site())
}
#[test]
#[rustfmt::skip]
fn replacing_nested_type_params_works() {
// Original path, replacement ident->paths, expected output path
let paths = [
// Works ok if nothing to replace
(
syn_path!(::some::path::Foo<::other::Path<A, B>>),
vec![],
syn_path!(::some::path::Foo<::other::Path<A, B>>),
),
// Simple top level replacing
(
syn_path!(::some::path::Foo<A>),
vec![(ident("A"), type_path!(::new::Value))],
syn_path!(::some::path::Foo<::new::Value>),
),
// More deeply nested replacing works too
(
syn_path!(::some::path::Foo<::other::Path<A, B>>),
vec![(ident("A"), type_path!(::new::Value))],
syn_path!(::some::path::Foo<::other::Path<::new::Value, B>>),
),
(
syn_path!(::some::path::Foo<::other::Path<A, B>>),
vec![
(ident("A"), type_path!(::new::A)),
(ident("B"), type_path!(::new::B)),
],
syn_path!(::some::path::Foo<::other::Path<::new::A, ::new::B>>),
),
(
syn_path!(::some::path::Foo<::other::Path<A, ::more::path::to<::something::Argh<B>>>, C>),
vec![
(ident("A"), type_path!(::new::A)),
(ident("B"), type_path!(::new::B)),
],
syn_path!(::some::path::Foo<::other::Path<::new::A, ::more::path::to<::something::Argh<::new::B>>>,C>),
),
// The same ident will be replaced as many times as needed:
(
syn_path!(::some::path::Foo<::other::Path<A, ::foo::Argh<A, B>, A>>),
vec![(ident("A"), type_path!(::new::Value))],
syn_path!(::some::path::Foo<::other::Path<::new::Value, ::foo::Argh<::new::Value, B>, ::new::Value>>),
),
];
for (mut path, replacements, expected) in paths {
replace_path_params_recursively(&mut path, &replacements);
assert_eq!(path, expected);
}
}
}
codegen/src/types/tests.rs
-1323
View File
File diff suppressed because it is too large Load Diff
codegen/src/types/type_def.rs
-174
View File
@@ -1,174 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::error::CodegenError;
use super::{
CompositeDef, CompositeDefFields, Derives, TypeDefParameters, TypeGenerator, TypeParameter,
};
use proc_macro2::TokenStream;
use quote::{format_ident, quote};
use scale_info::{form::PortableForm, Type, TypeDef};
use syn::parse_quote;
/// Generates a Rust `struct` or `enum` definition based on the supplied [`scale-info::Type`].
///
/// Field type paths are resolved via the `TypeGenerator`, which contains the registry of all
/// generated types in the module.
#[derive(Debug)]
pub struct TypeDefGen {
/// The type parameters of the type to be generated
type_params: TypeDefParameters,
/// The derives with which to annotate the generated type.
derives: Derives,
/// The kind of type to be generated.
ty_kind: TypeDefGenKind,
/// Type documentation.
ty_docs: TokenStream,
}
impl TypeDefGen {
/// Construct a type definition for codegen from the given [`scale_info::Type`].
pub fn from_type(
ty: &Type<PortableForm>,
type_gen: &TypeGenerator,
) -> Result<Self, CodegenError> {
let derives = type_gen.type_derives(ty)?;
let crate_path = type_gen.crate_path();
let should_gen_docs = type_gen.should_gen_docs();
let type_params = ty
.type_params
.iter()
.enumerate()
.filter_map(|(i, tp)| match &tp.ty {
Some(ty) => {
let tp_name = format_ident!("_{}", i);
Some(TypeParameter {
concrete_type_id: ty.id,
original_name: tp.name.clone(),
name: tp_name,
})
}
None => None,
})
.collect::<Vec<_>>();
let mut type_params = TypeDefParameters::new(type_params);
let ty_kind = match &ty.type_def {
TypeDef::Composite(composite) => {
let type_name = ty.path.ident().expect("structs should have a name");
let fields = CompositeDefFields::from_scale_info_fields(
&type_name,
&composite.fields,
type_params.params(),
type_gen,
)?;
type_params.update_unused(fields.field_types());
let docs = should_gen_docs.then_some(&*ty.docs).unwrap_or_default();
let composite_def = CompositeDef::struct_def(
ty,
&type_name,
type_params.clone(),
fields,
Some(parse_quote!(pub)),
type_gen,
docs,
crate_path,
None,
)?;
TypeDefGenKind::Struct(composite_def)
}
TypeDef::Variant(variant) => {
let type_name = ty.path.ident().expect("variants should have a name");
let variants = variant
.variants
.iter()
.map(|v| {
let fields = CompositeDefFields::from_scale_info_fields(
&v.name,
&v.fields,
type_params.params(),
type_gen,
)?;
type_params.update_unused(fields.field_types());
let docs = should_gen_docs.then_some(&*v.docs).unwrap_or_default();
let variant_def = CompositeDef::enum_variant_def(&v.name, fields, docs);
Ok((v.index, variant_def))
})
.collect::<Result<Vec<_>, CodegenError>>()?;
TypeDefGenKind::Enum(type_name, variants)
}
_ => TypeDefGenKind::BuiltIn,
};
let docs = &ty.docs;
let ty_docs = should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
Ok(Self {
type_params,
derives,
ty_kind,
ty_docs,
})
}
/// are there unused type params?
pub fn has_unused_type_params(&self) -> bool {
self.type_params.has_unused_type_params()
}
}
impl quote::ToTokens for TypeDefGen {
fn to_tokens(&self, tokens: &mut TokenStream) {
match &self.ty_kind {
TypeDefGenKind::Struct(composite) => composite.to_tokens(tokens),
TypeDefGenKind::Enum(type_name, variants) => {
let mut variants = variants
.iter()
.map(|(index, def)| {
let index = proc_macro2::Literal::u8_unsuffixed(*index);
quote! {
#[codec(index = #index)]
#def
}
})
.collect::<Vec<_>>();
if let Some(phantom) = self.type_params.unused_params_phantom_data() {
variants.push(quote! {
__Ignore(#phantom)
})
}
let enum_ident = format_ident!("{}", type_name);
let type_params = &self.type_params;
let derives = &self.derives;
let docs = &self.ty_docs;
let ty_toks = quote! {
#derives
#docs
pub enum #enum_ident #type_params {
#( #variants, )*
}
};
tokens.extend(ty_toks);
}
TypeDefGenKind::BuiltIn => (), /* all built-in types should already be in scope */
}
}
}
#[allow(clippy::large_enum_variant)]
#[derive(Debug)]
pub enum TypeDefGenKind {
Struct(CompositeDef),
Enum(String, Vec<(u8, CompositeDef)>),
BuiltIn,
}
codegen/src/types/type_def_params.rs
-77
View File
@@ -1,77 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use super::TypeParameter;
use crate::types::CompositeDefFieldType;
use quote::quote;
use std::collections::BTreeSet;
/// Represents the set of generic type parameters for generating a type definition e.g. the `T` in
/// `Foo<T>`.
///
/// Additionally this allows generating a `PhantomData` type for any type params which are unused
/// in the type definition itself.
#[derive(Clone, Debug, Default)]
pub struct TypeDefParameters {
params: Vec<TypeParameter>,
unused: BTreeSet<TypeParameter>,
}
impl TypeDefParameters {
/// Create a new [`TypeDefParameters`] instance.
pub fn new(params: Vec<TypeParameter>) -> Self {
let unused = params.iter().cloned().collect();
Self { params, unused }
}
/// Update the set of unused type parameters by removing those that are used in the given
/// fields.
pub fn update_unused<'a>(&mut self, fields: impl Iterator<Item = &'a CompositeDefFieldType>) {
let mut used_type_params = BTreeSet::new();
for field in fields {
field.type_path.parent_type_params(&mut used_type_params)
}
for used_type_param in &used_type_params {
self.unused.remove(used_type_param);
}
}
/// Construct a [`core::marker::PhantomData`] for the type unused type params.
pub fn unused_params_phantom_data(&self) -> Option<syn::TypePath> {
if self.unused.is_empty() {
return None;
}
let params = if self.unused.len() == 1 {
let param = self
.unused
.iter()
.next()
.expect("Checked for exactly one unused param");
quote! { #param }
} else {
let params = self.unused.iter();
quote! { ( #( #params ), * ) }
};
Some(syn::parse_quote! { ::core::marker::PhantomData<#params> })
}
/// Returns the set of type parameters.
pub fn params(&self) -> &[TypeParameter] {
&self.params
}
/// Returns true if there are any unused type params
pub fn has_unused_type_params(&self) -> bool {
!self.unused.is_empty()
}
}
impl quote::ToTokens for TypeDefParameters {
fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
if !self.params.is_empty() {
let params = &self.params;
tokens.extend(quote! { < #( #params ),* > })
}
}
}
codegen/src/types/type_path.rs
-304
View File
@@ -1,304 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use proc_macro2::{Ident, TokenStream};
use quote::format_ident;
use scale_info::{form::PortableForm, Path, TypeDefPrimitive};
use std::collections::BTreeSet;
use syn::parse_quote;
/// An opaque struct representing a type path. The main usage of this is
/// to spit out as tokens in some `quote!{ ... }` macro; the inner structure
/// should be unimportant.
#[derive(Clone, Debug)]
pub struct TypePath(TypePathInner);
#[derive(Clone, Debug)]
pub enum TypePathInner {
Parameter(TypeParameter),
Type(TypePathType),
}
impl quote::ToTokens for TypePath {
fn to_tokens(&self, tokens: &mut TokenStream) {
let syn_type = self.to_syn_type();
syn_type.to_tokens(tokens)
}
}
impl TypePath {
/// Construct a [`TypePath`] from a [`TypeParameter`]
pub fn from_parameter(param: TypeParameter) -> TypePath {
TypePath(TypePathInner::Parameter(param))
}
/// Construct a [`TypePath`] from a [`TypeParameter`]
pub fn from_type(ty: TypePathType) -> TypePath {
TypePath(TypePathInner::Type(ty))
}
/// Construct a [`TypePath`] from a [`syn::TypePath`]
pub fn from_syn_path(path: syn::Path) -> TypePath {
// Note; this doesn't parse the parameters or anything, but since nothing external
// can inspect this structure, and the ToTokens impl works either way, it should be ok.
TypePath(TypePathInner::Type(TypePathType::Path {
path,
params: Vec::new(),
}))
}
pub(crate) fn to_syn_type(&self) -> syn::Type {
match &self.0 {
TypePathInner::Parameter(ty_param) => syn::Type::Path(parse_quote! { #ty_param }),
TypePathInner::Type(ty) => ty.to_syn_type(),
}
}
pub(crate) fn is_compact(&self) -> bool {
matches!(&self.0, TypePathInner::Type(ty) if ty.is_compact())
}
pub(crate) fn is_string(&self) -> bool {
matches!(&self.0, TypePathInner::Type(ty) if ty.is_string())
}
/// Returns the type parameters in a path which are inherited from the containing type.
///
/// # Example
///
/// ```rust
/// struct S<T> {
/// a: Vec<Option<T>>, // the parent type param here is `T`
/// }
/// ```
pub fn parent_type_params(&self, acc: &mut BTreeSet<TypeParameter>) {
match &self.0 {
TypePathInner::Parameter(type_parameter) => {
acc.insert(type_parameter.clone());
}
TypePathInner::Type(type_path) => type_path.parent_type_params(acc),
}
}
/// Gets the vector type parameter if the data is represented as `TypeDef::Sequence`.
///
/// **Note:** Utilized for transforming `std::vec::Vec<T>` into slices `&[T]` for the storage API.
pub fn vec_type_param(&self) -> Option<&TypePath> {
let ty = match &self.0 {
TypePathInner::Type(ty) => ty,
_ => return None,
};
match ty {
TypePathType::Vec { of } => Some(of),
_ => None,
}
}
}
#[derive(Clone, Debug)]
pub enum TypePathType {
Path {
path: syn::Path,
params: Vec<TypePath>,
},
Vec {
of: Box<TypePath>,
},
Array {
len: usize,
of: Box<TypePath>,
},
Tuple {
elements: Vec<TypePath>,
},
Primitive {
def: TypeDefPrimitive,
},
Compact {
inner: Box<TypePath>,
is_field: bool,
crate_path: syn::Path,
},
BitVec {
bit_order_type: Box<TypePath>,
bit_store_type: Box<TypePath>,
crate_path: syn::Path,
},
}
impl TypePathType {
pub fn from_type_def_path(
path: &Path<PortableForm>,
root_mod_ident: Ident,
params: Vec<TypePath>,
) -> Self {
let path_segments = &*path.segments;
let path: syn::Path = match path_segments {
[] => panic!("Type has no ident"),
[ident] => {
// paths to prelude types
match ident.as_str() {
"Option" => parse_quote!(::core::option::Option),
"Result" => parse_quote!(::core::result::Result),
"Cow" => parse_quote!(::std::borrow::Cow),
"BTreeMap" => parse_quote!(::std::collections::BTreeMap),
"BTreeSet" => parse_quote!(::std::collections::BTreeSet),
"Range" => parse_quote!(::core::ops::Range),
"RangeInclusive" => parse_quote!(::core::ops::RangeInclusive),
"NonZeroI8" => parse_quote!(::core::num::NonZeroI8),
"NonZeroU8" => parse_quote!(::core::num::NonZeroU8),
"NonZeroI16" => parse_quote!(::core::num::NonZeroI16),
"NonZeroU16" => parse_quote!(::core::num::NonZeroU16),
"NonZeroI32" => parse_quote!(::core::num::NonZeroI32),
"NonZeroU32" => parse_quote!(::core::num::NonZeroU32),
"NonZeroI64" => parse_quote!(::core::num::NonZeroI64),
"NonZeroU64" => parse_quote!(::core::num::NonZeroU64),
"NonZeroI128" => parse_quote!(::core::num::NonZeroI128),
"NonZeroU128" => parse_quote!(::core::num::NonZeroU128),
"NonZeroIsize" => parse_quote!(::core::num::NonZeroIsize),
"NonZeroUsize" => parse_quote!(::core::num::NonZeroUsize),
ident => panic!("Unknown prelude type '{ident}'"),
}
}
_ => {
// paths to generated types in the root types module
let mut ty_path = path_segments
.iter()
.map(|s| syn::PathSegment::from(format_ident!("{}", s)))
.collect::<syn::punctuated::Punctuated<syn::PathSegment, syn::Token![::]>>();
ty_path.insert(0, syn::PathSegment::from(root_mod_ident));
parse_quote!( #ty_path )
}
};
Self::Path { path, params }
}
/// Visits a type path, collecting all the generic type parameters from the containing type.
///
/// # Example
///
/// ```rust
/// struct S<T> {
/// a: Vec<Option<T>>, // the parent type param here is `T`
/// }
/// ```
fn parent_type_params(&self, acc: &mut BTreeSet<TypeParameter>) {
match self {
TypePathType::Path { params, .. } => {
for p in params {
p.parent_type_params(acc)
}
}
TypePathType::Vec { of } => of.parent_type_params(acc),
TypePathType::Array { of, .. } => of.parent_type_params(acc),
TypePathType::Tuple { elements } => {
for e in elements {
e.parent_type_params(acc)
}
}
TypePathType::Primitive { .. } => (),
TypePathType::Compact { inner, .. } => inner.parent_type_params(acc),
TypePathType::BitVec {
bit_order_type,
bit_store_type,
crate_path: _,
} => {
bit_order_type.parent_type_params(acc);
bit_store_type.parent_type_params(acc);
}
}
}
pub(crate) fn is_compact(&self) -> bool {
matches!(self, TypePathType::Compact { .. })
}
pub(crate) fn is_string(&self) -> bool {
matches!(
self,
TypePathType::Primitive {
def: TypeDefPrimitive::Str
}
)
}
fn to_syn_type(&self) -> syn::Type {
match &self {
TypePathType::Path { path, params } => {
let path = if params.is_empty() {
parse_quote! { #path }
} else {
parse_quote! { #path< #( #params ),* > }
};
syn::Type::Path(path)
}
TypePathType::Vec { of } => {
let type_path = parse_quote! { ::std::vec::Vec<#of> };
syn::Type::Path(type_path)
}
TypePathType::Array { len, of } => {
let array = parse_quote! { [#of; #len] };
syn::Type::Array(array)
}
TypePathType::Tuple { elements } => {
let tuple = parse_quote! { (#( # elements, )* ) };
syn::Type::Tuple(tuple)
}
TypePathType::Primitive { def } => syn::Type::Path(match def {
TypeDefPrimitive::Bool => parse_quote!(::core::primitive::bool),
TypeDefPrimitive::Char => parse_quote!(::core::primitive::char),
TypeDefPrimitive::Str => parse_quote!(::std::string::String),
TypeDefPrimitive::U8 => parse_quote!(::core::primitive::u8),
TypeDefPrimitive::U16 => parse_quote!(::core::primitive::u16),
TypeDefPrimitive::U32 => parse_quote!(::core::primitive::u32),
TypeDefPrimitive::U64 => parse_quote!(::core::primitive::u64),
TypeDefPrimitive::U128 => parse_quote!(::core::primitive::u128),
TypeDefPrimitive::U256 => unimplemented!("not a rust primitive"),
TypeDefPrimitive::I8 => parse_quote!(::core::primitive::i8),
TypeDefPrimitive::I16 => parse_quote!(::core::primitive::i16),
TypeDefPrimitive::I32 => parse_quote!(::core::primitive::i32),
TypeDefPrimitive::I64 => parse_quote!(::core::primitive::i64),
TypeDefPrimitive::I128 => parse_quote!(::core::primitive::i128),
TypeDefPrimitive::I256 => unimplemented!("not a rust primitive"),
}),
TypePathType::Compact {
inner,
is_field,
crate_path,
} => {
let path = if *is_field {
// compact fields can use the inner compact type directly and be annotated with
// the `compact` attribute e.g. `#[codec(compact)] my_compact_field: u128`
parse_quote! ( #inner )
} else {
parse_quote! ( #crate_path::ext::codec::Compact<#inner> )
};
syn::Type::Path(path)
}
TypePathType::BitVec {
bit_order_type,
bit_store_type,
crate_path,
} => {
let type_path = parse_quote! { #crate_path::utils::bits::DecodedBits<#bit_store_type, #bit_order_type> };
syn::Type::Path(type_path)
}
}
}
}
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
pub struct TypeParameter {
pub(super) concrete_type_id: u32,
pub(super) original_name: String,
pub(super) name: Ident,
}
impl quote::ToTokens for TypeParameter {
fn to_tokens(&self, tokens: &mut TokenStream) {
self.name.to_tokens(tokens)
}
}