Storage: Support iterating over NMaps with partial keys (#1079)

* implement partial key iters

* format

* make tests compile

* fix docs and try example

* codegen: Fetch and decode metadata version then fallback (#1092)

* codegen: Fetch and decode metadata version then fallback

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add `unstable-metadata` attribute to the subxt macro

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* docs: Add missing comma

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* macro: Add `GenerateRuntimeApi`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update subxt/src/lib.rs

Co-authored-by: James Wilson <james@jsdw.me>

* Update macro/src/lib.rs

Co-authored-by: James Wilson <james@jsdw.me>

* subxt: Adjust docs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Import `GenerateRuntimeApi`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>

* Bump darling from 0.20.1 to 0.20.3 (#1085)

Bumps [darling](https://github.com/TedDriggs/darling) from 0.20.1 to 0.20.3.
- [Release notes](https://github.com/TedDriggs/darling/releases)
- [Changelog](https://github.com/TedDriggs/darling/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TedDriggs/darling/compare/v0.20.1...v0.20.3)

---
updated-dependencies:
- dependency-name: darling
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>

* Bump either from 1.8.1 to 1.9.0 (#1084)

Bumps [either](https://github.com/bluss/either) from 1.8.1 to 1.9.0.
- [Commits](https://github.com/bluss/either/compare/1.8.1...1.9.0)

---
updated-dependencies:
- dependency-name: either
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>

* Bump clap from 4.3.11 to 4.3.19 (#1083)

Bumps [clap](https://github.com/clap-rs/clap) from 4.3.11 to 4.3.19.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.11...v4.3.19)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Tadeo Hepperle <62739623+tadeohepperle@users.noreply.github.com>

* Bump trybuild from 1.0.81 to 1.0.82 (#1082)

Bumps [trybuild](https://github.com/dtolnay/trybuild) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/trybuild/releases)
- [Commits](https://github.com/dtolnay/trybuild/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: trybuild
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>

* Prep for 0.30.1 release (#1094)

* Set minimum supported `rust-version` to `1.70` (#1097)

* Bump serde_json from 1.0.103 to 1.0.104 (#1100)

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.103...v1.0.104)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump serde from 1.0.175 to 1.0.179 (#1101)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.175 to 1.0.179.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.175...v1.0.179)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Tests: support 'substrate-node' too and allow multiple binary paths (#1102)

* Support 'substrate-node' too and allow multiple binary paths

* fmt

* clippy

* fix path

* adjust book

* remove the partial iteration example. there was nothing good to show

* revert spaces in changelog

* Support more types in Storage entry constructors  (#1105)

* implement test for type alias being used

* Bump serde from 1.0.179 to 1.0.183 (#1112)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.179 to 1.0.183.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.179...v1.0.183)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump regex from 1.9.1 to 1.9.3 (#1110)

Bumps [regex](https://github.com/rust-lang/regex) from 1.9.1 to 1.9.3.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.9.1...1.9.3)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* revert yaml changes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update subxt/src/book/usage/storage.rs

Co-authored-by: James Wilson <james@jsdw.me>

* remove dynamic_iter

* fix example

* format

* add example, adjust book

* Update subxt/src/book/usage/storage.rs

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
This commit is contained in:
Tadeo Hepperle
2023-08-11 16:06:41 +02:00
committed by GitHub
parent 8ba113f368
commit 43809ee592
13 changed files with 4873 additions and 3978 deletions
+24 -3
View File
@@ -39,7 +39,7 @@
//! ```
//!
//! As well as accessing specific entries, some storage locations can also be iterated over (such as
//! the map of account information). To do this, suffix `_root` onto the query constructor (this
//! the map of account information). To do this, suffix `_iter` onto the query constructor (this
//! will only be available on static constructors when iteration is actually possible):
//!
//! ```rust,no_run
@@ -47,11 +47,25 @@
//! pub mod polkadot {}
//!
//! // A static query capable of iterating over accounts:
//! let storage_query = polkadot::storage().system().account_root();
//! let storage_query = polkadot::storage().system().account_iter();
//! // A dynamic query to do the same:
//! let storage_query = subxt::dynamic::storage_root("System", "Account");
//! let storage_query = subxt::dynamic::storage("System", "Account", Vec::<u8>::new());
//! ```
//!
//! Some storage entries are maps with multiple keys. As an example, we might end up with
//! an API like `runtime::storage().foo().bar(u8, bool, u16, String)` to fetch some entry "bar".
//! When this is the case, the codegen will generate multiple iterator query functions alongside
//! the function to fetch an individual value:
//!
//! - `runtime::storage().foo().bar(u8, bool, u16, String)`: fetch a single entry from the "bar" map.
//! - `runtime::storage().foo().bar_iter()`: iterate over all of the entries in the "bar" map.
//! - `runtime::storage().foo().bar_iter1(u8)`: iterate over all of the entries in the "bar" map under
//! a given `u8`.
//! - `runtime::storage().foo().bar_iter2(u8, bool)`: iterate over all of the entries in the "bar" map under
//! a given `u8` and `bool` value.
//! - `runtime::storage().foo().bar_iter3(u8, bool, u16)`: iterate over all of the entries in the "bar" map under
//! a given `u8`, `bool` and `u16` value.
//!
//! All valid storage queries implement [`crate::storage::StorageAddress`]. As well as describing
//! how to build a valid storage query, this trait also has some associated types that determine the
//! shape of the result you'll get back, and determine what you can do with it (ie, can you iterate
@@ -99,6 +113,13 @@
#![doc = include_str!("../../../examples/storage_iterating_dynamic.rs")]
//! ```
//!
//! Here is an example of iterating over partial keys. In this example some multi-signature operations
//! are sent to the node. We can iterate over the pending multisig operations of a single multisig account:
//!
//! ```rust,ignore
#![doc = include_str!("../../../examples/storage_iterating_partial.rs")]
//! ```
//!
//! ### Advanced
//!
//! For more advanced use cases, have a look at [`crate::storage::Storage::fetch_raw`] and
+1 -1
View File
@@ -26,7 +26,7 @@ pub use crate::tx::dynamic as tx;
pub use crate::constants::dynamic as constant;
// Lookup storage values dynamically.
pub use crate::storage::{dynamic as storage, dynamic_root as storage_root};
pub use crate::storage::dynamic as storage;
// Execute runtime API function call dynamically.
pub use crate::runtime_api::dynamic as runtime_api_call;
+3 -3
View File
@@ -21,11 +21,11 @@ pub use crate::rpc::types::StorageKey;
/// entry lives and how to properly decode it.
pub mod address {
pub use super::storage_address::{
dynamic, dynamic_root, make_static_storage_map_key, Address, DynamicAddress,
StaticStorageMapKey, StorageAddress, Yes,
dynamic, make_static_storage_map_key, Address, DynamicAddress, StaticStorageMapKey,
StorageAddress, Yes,
};
}
// For consistency with other modules, also expose
// the basic address stuff at the root of the module.
pub use storage_address::{dynamic, dynamic_root, Address, DynamicAddress, StorageAddress};
pub use storage_address::{dynamic, Address, DynamicAddress, StorageAddress};
+2 -10
View File
@@ -3,7 +3,7 @@
// see LICENSE for license details.
use crate::{
dynamic::{DecodedValueThunk, Value},
dynamic::DecodedValueThunk,
error::{Error, MetadataError, StorageAddressError},
metadata::{DecodeWithMetadata, EncodeWithMetadata, Metadata},
utils::{Encoded, Static},
@@ -50,7 +50,7 @@ pub trait StorageAddress {
pub struct Yes;
/// A concrete storage address. This can be created from static values (ie those generated
/// via the `subxt` macro) or dynamic values via [`dynamic`] and [`dynamic_root`].
/// via the `subxt` macro) or dynamic values via [`dynamic`].
pub struct Address<StorageKey, ReturnTy, Fetchable, Defaultable, Iterable> {
pallet_name: Cow<'static, str>,
entry_name: Cow<'static, str>,
@@ -228,14 +228,6 @@ pub fn make_static_storage_map_key<T: codec::Encode>(t: T) -> StaticStorageMapKe
Static(Encoded(t.encode()))
}
/// Construct a new dynamic storage lookup to the root of some entry.
pub fn dynamic_root(
pallet_name: impl Into<String>,
entry_name: impl Into<String>,
) -> DynamicAddress<Value> {
DynamicAddress::new(pallet_name, entry_name, vec![])
}
/// Construct a new dynamic storage lookup.
pub fn dynamic<StorageKey: EncodeWithMetadata>(
pallet_name: impl Into<String>,
+1 -1
View File
@@ -179,7 +179,7 @@ where
/// let api = OnlineClient::<PolkadotConfig>::new().await.unwrap();
///
/// // Address to the root of a storage entry that we'd like to iterate over.
/// let address = polkadot::storage().xcm_pallet().version_notifiers_root();
/// let address = polkadot::storage().xcm_pallet().version_notifiers_iter();
///
/// // Iterate over keys and values at that address.
/// let mut iter = api