Files
pezkuwi-subxt/codegen/src/api/storage.rs
T
Tadeo Hepperle 43809ee592 Storage: Support iterating over NMaps with partial keys (#1079)
* implement partial key iters

* format

* make tests compile

* fix docs and try example

* codegen: Fetch and decode metadata version then fallback (#1092)

* codegen: Fetch and decode metadata version then fallback

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add `unstable-metadata` attribute to the subxt macro

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* docs: Add missing comma

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* macro: Add `GenerateRuntimeApi`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update subxt/src/lib.rs

Co-authored-by: James Wilson <james@jsdw.me>

* Update macro/src/lib.rs

Co-authored-by: James Wilson <james@jsdw.me>

* subxt: Adjust docs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Import `GenerateRuntimeApi`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>

* Bump darling from 0.20.1 to 0.20.3 (#1085)

Bumps [darling](https://github.com/TedDriggs/darling) from 0.20.1 to 0.20.3.
- [Release notes](https://github.com/TedDriggs/darling/releases)
- [Changelog](https://github.com/TedDriggs/darling/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TedDriggs/darling/compare/v0.20.1...v0.20.3)

---
updated-dependencies:
- dependency-name: darling
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>

* Bump either from 1.8.1 to 1.9.0 (#1084)

Bumps [either](https://github.com/bluss/either) from 1.8.1 to 1.9.0.
- [Commits](https://github.com/bluss/either/compare/1.8.1...1.9.0)

---
updated-dependencies:
- dependency-name: either
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>

* Bump clap from 4.3.11 to 4.3.19 (#1083)

Bumps [clap](https://github.com/clap-rs/clap) from 4.3.11 to 4.3.19.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.11...v4.3.19)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Tadeo Hepperle <62739623+tadeohepperle@users.noreply.github.com>

* Bump trybuild from 1.0.81 to 1.0.82 (#1082)

Bumps [trybuild](https://github.com/dtolnay/trybuild) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/trybuild/releases)
- [Commits](https://github.com/dtolnay/trybuild/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: trybuild
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>

* Prep for 0.30.1 release (#1094)

* Set minimum supported `rust-version` to `1.70` (#1097)

* Bump serde_json from 1.0.103 to 1.0.104 (#1100)

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.103...v1.0.104)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump serde from 1.0.175 to 1.0.179 (#1101)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.175 to 1.0.179.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.175...v1.0.179)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Tests: support 'substrate-node' too and allow multiple binary paths (#1102)

* Support 'substrate-node' too and allow multiple binary paths

* fmt

* clippy

* fix path

* adjust book

* remove the partial iteration example. there was nothing good to show

* revert spaces in changelog

* Support more types in Storage entry constructors  (#1105)

* implement test for type alias being used

* Bump serde from 1.0.179 to 1.0.183 (#1112)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.179 to 1.0.183.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.179...v1.0.183)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump regex from 1.9.1 to 1.9.3 (#1110)

Bumps [regex](https://github.com/rust-lang/regex) from 1.9.1 to 1.9.3.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.9.1...1.9.3)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* revert yaml changes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update subxt/src/book/usage/storage.rs

Co-authored-by: James Wilson <james@jsdw.me>

* remove dynamic_iter

* fix example

* format

* add example, adjust book

* Update subxt/src/book/usage/storage.rs

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
2023-08-11 16:06:41 +02:00

293 lines
9.8 KiB
Rust

// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::types::TypePath;
use crate::{types::TypeGenerator, CratePath};
use heck::ToSnakeCase as _;
use proc_macro2::{Ident, TokenStream as TokenStream2, TokenStream};
use quote::{format_ident, quote};
use scale_info::TypeDef;
use subxt_metadata::{
PalletMetadata, StorageEntryMetadata, StorageEntryModifier, StorageEntryType,
};
use super::CodegenError;
/// Generate functions which create storage addresses from the provided pallet's metadata.
/// These addresses can be used to access and iterate over storage values.
///
/// # Arguments
///
/// - `metadata` - Runtime metadata from which the storages are generated.
/// - `type_gen` - The type generator containing all types defined by metadata.
/// - `pallet` - Pallet metadata from which the storages are generated.
/// - `types_mod_ident` - The ident of the base module that we can use to access the generated types from.
pub fn generate_storage(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
types_mod_ident: &syn::Ident,
crate_path: &CratePath,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let Some(storage) = pallet.storage() else {
return Ok(quote!());
};
let storage_fns = storage
.entries()
.iter()
.map(|entry| {
generate_storage_entry_fns(type_gen, pallet, entry, crate_path, should_gen_docs)
})
.collect::<Result<Vec<_>, CodegenError>>()?;
Ok(quote! {
pub mod storage {
use super::#types_mod_ident;
pub struct StorageApi;
impl StorageApi {
#( #storage_fns )*
}
}
})
}
fn generate_storage_entry_fns(
type_gen: &TypeGenerator,
pallet: &PalletMetadata,
storage_entry: &StorageEntryMetadata,
crate_path: &CratePath,
should_gen_docs: bool,
) -> Result<TokenStream2, CodegenError> {
let keys: Vec<(Ident, TypePath)> = match storage_entry.entry_type() {
StorageEntryType::Plain(_) => vec![],
StorageEntryType::Map { key_ty, .. } => {
match &type_gen.resolve_type(*key_ty).type_def {
// An N-map; return each of the keys separately.
TypeDef::Tuple(tuple) => tuple
.fields
.iter()
.enumerate()
.map(|(i, f)| {
let ident: Ident = format_ident!("_{}", syn::Index::from(i));
let ty_path = type_gen.resolve_type_path(f.id);
(ident, ty_path)
})
.collect::<Vec<_>>(),
// A map with a single key; return the single key.
_ => {
let ident = format_ident!("_0");
let ty_path = type_gen.resolve_type_path(*key_ty);
vec![(ident, ty_path)]
}
}
}
};
let pallet_name = pallet.name();
let storage_name = storage_entry.name();
let Some(storage_hash) = pallet.storage_hash(storage_name) else {
return Err(CodegenError::MissingStorageMetadata(pallet_name.into(), storage_name.into()));
};
let snake_case_name = storage_entry.name().to_snake_case();
let storage_entry_ty = match storage_entry.entry_type() {
StorageEntryType::Plain(ty) => *ty,
StorageEntryType::Map { value_ty, .. } => *value_ty,
};
let storage_entry_value_ty = type_gen.resolve_type_path(storage_entry_ty);
let docs = storage_entry.docs();
let docs = should_gen_docs
.then_some(quote! { #( #[doc = #docs ] )* })
.unwrap_or_default();
let is_defaultable_type = match storage_entry.modifier() {
StorageEntryModifier::Default => quote!(#crate_path::storage::address::Yes),
StorageEntryModifier::Optional => quote!(()),
};
let all_fns = (0..=keys.len()).map(|n_keys| {
let keys_slice = &keys[..n_keys];
let (fn_name, is_fetchable, is_iterable) = if n_keys == keys.len() {
let fn_name = format_ident!("{snake_case_name}");
(fn_name, true, false)
} else {
let fn_name = if n_keys == 0 {
format_ident!("{snake_case_name}_iter")
} else {
format_ident!("{snake_case_name}_iter{}", n_keys)
};
(fn_name, false, true)
};
let is_fetchable_type = is_fetchable.then_some(quote!(#crate_path::storage::address::Yes)).unwrap_or(quote!(()));
let is_iterable_type = is_iterable.then_some(quote!(#crate_path::storage::address::Yes)).unwrap_or(quote!(()));
let key_impls = keys_slice.iter().map(|(field_name, _)| quote!( #crate_path::storage::address::make_static_storage_map_key(#field_name.borrow()) ));
let key_args = keys_slice.iter().map(|(field_name, field_type)| {
let field_ty = primitive_type_alias(field_type);
quote!( #field_name: impl ::std::borrow::Borrow<#field_ty> )
});
quote!(
#docs
pub fn #fn_name(
&self,
#(#key_args,)*
) -> #crate_path::storage::address::Address::<
#crate_path::storage::address::StaticStorageMapKey,
#storage_entry_value_ty,
#is_fetchable_type,
#is_defaultable_type,
#is_iterable_type
> {
#crate_path::storage::address::Address::new_static(
#pallet_name,
#storage_name,
vec![#(#key_impls,)*],
[#(#storage_hash,)*]
)
}
)
});
Ok(quote! {
#( #all_fns
)*
})
}
fn primitive_type_alias(type_path: &TypePath) -> TokenStream {
// Vec<T> is cast to [T]
if let Some(ty) = type_path.vec_type_param() {
return quote!([#ty]);
}
// String is cast to str
if type_path.is_string() {
return quote!(::core::primitive::str);
}
quote!(#type_path)
}
#[cfg(test)]
mod tests {
use crate::RuntimeGenerator;
use frame_metadata::v15;
use quote::{format_ident, quote};
use scale_info::{meta_type, MetaType};
use std::borrow::Cow;
use subxt_metadata::Metadata;
fn metadata_with_storage_entries(
storage_entries: impl IntoIterator<Item = (&'static str, MetaType)>,
) -> Metadata {
let storage_entries: Vec<v15::StorageEntryMetadata> = storage_entries
.into_iter()
.map(|(name, key)| v15::StorageEntryMetadata {
name,
modifier: v15::StorageEntryModifier::Optional,
ty: v15::StorageEntryType::Map {
hashers: vec![],
key,
value: meta_type::<bool>(),
},
default: vec![],
docs: vec![],
})
.collect();
let pallet_1 = v15::PalletMetadata {
name: "Pallet1",
storage: Some(v15::PalletStorageMetadata {
prefix: Default::default(),
entries: storage_entries,
}),
calls: None,
event: None,
constants: vec![],
error: None,
index: 0,
docs: vec![],
};
let extrinsic_metadata = v15::ExtrinsicMetadata {
version: 0,
signed_extensions: vec![],
address_ty: meta_type::<()>(),
call_ty: meta_type::<()>(),
signature_ty: meta_type::<()>(),
extra_ty: meta_type::<()>(),
};
let metadata: Metadata = v15::RuntimeMetadataV15::new(
vec![pallet_1],
extrinsic_metadata,
meta_type::<()>(),
vec![],
v15::OuterEnums {
call_enum_ty: meta_type::<()>(),
event_enum_ty: meta_type::<()>(),
error_enum_ty: meta_type::<()>(),
},
v15::CustomMetadata {
map: Default::default(),
},
)
.try_into()
.expect("can build valid metadata");
metadata
}
#[test]
fn borrow_type_replacements() {
let storage_entries = [
("vector", meta_type::<Vec<u8>>()),
("boxed", meta_type::<Box<u16>>()),
("string", meta_type::<String>()),
("static_string", meta_type::<&'static str>()),
("cow_string", meta_type::<Cow<'_, str>>()),
];
let expected_borrowed_types = [
quote!([::core::primitive::u8]),
quote!(::core::primitive::u16),
quote!(::core::primitive::str),
quote!(::core::primitive::str),
quote!(::core::primitive::str),
];
let metadata = metadata_with_storage_entries(storage_entries);
let item_mod = syn::parse_quote!(
pub mod api {}
);
let generator = RuntimeGenerator::new(metadata);
let generated = generator
.generate_runtime(
item_mod,
Default::default(),
Default::default(),
"::subxt_path".into(),
false,
)
.expect("should be able to generate runtime");
let generated_str = generated.to_string();
for ((name, _), expected_type) in storage_entries
.into_iter()
.zip(expected_borrowed_types.into_iter())
{
let name_ident = format_ident!("{}", name);
let expected_storage_constructor = quote!(
fn #name_ident(
&self,
_0: impl ::std::borrow::Borrow<#expected_type>,
)
);
assert!(generated_str.contains(&expected_storage_constructor.to_string()));
}
}
}