pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-11 23:31:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into lenxv/light-client-testing

Browse Source 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
This commit is contained in:
Alexandru Vasile
2024-03-26 18:05:45 +02:00
parent 5039f1cccb 92c1ba7f66
commit 410aaed1d0
109 changed files with 12570 additions and 5033 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/dependabot.yml
+2 -2
View File
@@ -5,12 +5,12 @@ updates:
schedule:
interval: weekly
ignore:
# these need to be updated together, so dependabot PRs
# these need to be updated together, so dependabot PRs
# are just noise. So, ignore them:
- dependency-name: sp-core
- dependency-name: sp-keyring
- dependency-name: sp-runtime
- dependency-name: sp-core-hashing
- dependency-name: sp-crypto-hashing
- dependency-name: sp-version
- package-ecosystem: github-actions
directory: '/'
CHANGELOG.md
+111 -4
View File
@@ -4,8 +4,115 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.35.0] - 2024-03-21
This release contains several fixes, adds `no_std` support to a couple of crates (`subxt-signer` and `subxt-metadata`) and introduces a few quality of life improvements, which I'll quickly cover:
### Reworked light client ([#1475](https://github.com/paritytech/subxt/pull/1475))
This PR reworks the light client interface. The "basic" usage of connecting to a parachain now looks like this:
```rust
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}
use subxt::lightclient::LightClient;
// Instantiate a light client with the Polkadot relay chain given its chain spec.
let (lightclient, polkadot_rpc) = LightClient::relay_chain(POLKADOT_SPEC)?;
// Connect the light client to some parachain by giving a chain spec for it.
let asset_hub_rpc = lightclient.parachain(ASSET_HUB_SPEC)?;
// Now, we can create Subxt clients from these Smoldot backed RPC clients:
let polkadot_api = OnlineClient::<PolkadotConfig>::from_rpc_client(polkadot_rpc).await?;
let asset_hub_api = OnlineClient::<PolkadotConfig>::from_rpc_client(asset_hub_rpc).await?;
```
This interface mirrors the requirement that we must connect to a relay chain before we can connect to a parachain. It also moves the light client specific logic into an `RpcClientT` implementation, rather than exposing it as a `subxt::client::LightClient`.
### Typed Storage Keys ([#1419](https://github.com/paritytech/subxt/pull/1419))
This PR changes the storage interface so that, where possible, we now also decode the storage keys as well as the values when iterating over storage entries:
```rust
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}
// Create a new API client, configured to talk to Polkadot nodes.
let api = OnlineClient::<PolkadotConfig>::new().await?;
// Build a storage query to iterate over account information.
let storage_query = polkadot::storage().system().account_iter();
// Get back an iterator of results (here, we are fetching 10 items at
// a time from the node, but we always iterate over one at a time).
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;
while let Some(Ok(kv)) = results.next().await {
// We used to get a tuple of key bytes + value. Now we get back a
// `kv` struct containing the bytes and value as well as the actual
// decoded keys:
println!("Decoded key(s): {:?}", kv.keys);
println!("Key bytes: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value);
}
```
When using the static interface, keys come back as a tuple of values corresponding to the different hashers used in constructing the key. When using a dynamic interface, keys will be encoded/decoded from the type given so long as it implements `subxt::storage::StorageKey`, eg `Vec<scale_value::Value>`.
### Extrinsic Params Refinement ([#1439](https://github.com/paritytech/subxt/pull/1439))
Prior to this PR, one could configure extrinsic signed extensions by providing some params like so:
```rust
// Configure the transaction parameters; we give a small tip and set the
// transaction to live for 32 blocks from the `latest_block` above:
let tx_params = Params::new()
.tip(1_000)
.mortal(latest_block.header(), 32)
.build();
let hash = api.tx().sign_and_submit(&tx, &from, tx_params).await?;
```
If you want to customize the account nonce, you'd use a different call like `create_signed_with_nonce` instead.
One of the downsides of the above approach is that, if you don't provide any explicit params, transactions will be immortal by default (because the signed extensions didn't have the information to do any better).
Now, with the help of a `RefineParams` trait, transactions will default to being mortal and living for 32 blocks unless an explicit mortality is provided as above.
One notable change is that the offline-only `create_signed_with_nonce` and `create_partial_signed_with_nonce` functions have lost the `_with_nonce` suffix. Since we can't discover nonce/mortality settings offline, you should now provide `Params` and set an explicit nonce (and mortality, if you like) when using these calls, otherwise the nonce will be set to 0 and the mortality to `Immortal`.
For a full list of changes, please see the following:
### Added
- Reworked light client ([#1475](https://github.com/paritytech/subxt/pull/1475))
- `no_std` compatibility for `subxt-signer` ([#1477](https://github.com/paritytech/subxt/pull/1477))
- Typed Storage Keys ([#1419](https://github.com/paritytech/subxt/pull/1419))
- Extrinsic Params Refinement ([#1439](https://github.com/paritytech/subxt/pull/1439))
- Make storage_page_size for the LegacyBackend configurable ([#1458](https://github.com/paritytech/subxt/pull/1458))
- `no_std` compatibility for `subxt-metadata` ([#1401](https://github.com/paritytech/subxt/pull/1401))
- Experimental `reconnecting-rpc-client` ([#1396](https://github.com/paritytech/subxt/pull/1396))
### Changed
- `scale-type-resolver` integration ([#1460](https://github.com/paritytech/subxt/pull/1460))
- subxt: Derive `std::cmp` traits for subxt payloads and addresses ([#1429](https://github.com/paritytech/subxt/pull/1429))
- CLI: Return error on wrongly specified type paths ([#1397](https://github.com/paritytech/subxt/pull/1397))
- rpc v2: chainhead support multiple finalized block hashes in `FollowEvent::Initialized` ([#1476](https://github.com/paritytech/subxt/pull/1476))
- rpc v2: rename transaction to transactionWatch ([#1399](https://github.com/paritytech/subxt/pull/1399))
### Fixed
- Avoid a panic in case we try decoding naff bytes ([#1444](https://github.com/paritytech/subxt/pull/1444))
- Fix error mapping to wrong transaction status ([#1445](https://github.com/paritytech/subxt/pull/1445))
- Update DispatchError to match latest in polkadot-sdk ([#1442](https://github.com/paritytech/subxt/pull/1442))
- Handle errors when fetching storage keys from Unstablebackend ([#1440](https://github.com/paritytech/subxt/pull/1440))
- Swap type aliases around to be semantically correct ([#1441](https://github.com/paritytech/subxt/pull/1441))
## [0.34.0] - 2024-01-23
This release introduces a bunch of features that make subxt easier to use. Let's look at a few of them.
### Codegen - Integrating [`scale-typegen`](https://github.com/paritytech/scale-typegen) and adding type aliases ([#1249](https://github.com/paritytech/subxt/pull/1249))
@@ -23,7 +130,7 @@ If you provide an invalid type path, the macro will tell you so. It also suggest
```rust
#[subxt::subxt(
runtime_metadata_path = "metadata.scale",
runtime_metadata_path = "metadata.scale",
derive_for_type(path = "Junctions", derive = "Clone")
)]
pub mod polkadot {}
@@ -34,7 +141,7 @@ This gives you a compile-time error like this:
```md
Type `Junctions` does not exist at path `Junctions`
A type with the same name is present at:
A type with the same name is present at:
xcm::v3::junctions::Junctions
xcm::v2::multilocation::Junctions
```
@@ -78,7 +185,7 @@ Our CLI tool now allows you to explore runtime APIs and events ([#1290](https://
# Show details about a runtime API call:
subxt explore --url wss://westend-rpc.polkadot.io api StakingAPI nominations_quota
# Execute a runtime API call from the CLI:
subxt explore --url wss://westend-rpc.polkadot.io api core version -e
subxt explore --url wss://westend-rpc.polkadot.io api core version -e
# Discover what events a pallet can emit:
subxt explore --url wss://westend-rpc.polkadot.io pallet Balances events
```
Cargo.lock
Generated
+454 -362
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+51 -43
View File
@@ -19,13 +19,20 @@ members = [
# We exclude any crates that would depend on non mutually
# exclusive feature flags and thus can't compile with the
# workspace:
exclude = ["testing/no-std-tests", "testing/wasm-rpc-tests", "testing/wasm-lightclient-tests", "signer/wasm-tests", "examples/wasm-example", "examples/parachain-example"]
exclude = [
"testing/no-std-tests",
"testing/wasm-rpc-tests",
"testing/wasm-lightclient-tests",
"signer/wasm-tests",
"examples/wasm-example",
"examples/parachain-example"
]
resolver = "2"
[workspace.package]
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
version = "0.34.0"
version = "0.35.0"
rust-version = "1.74.0"
license = "Apache-2.0 OR GPL-3.0"
repository = "https://github.com/paritytech/subxt"
@@ -52,55 +59,56 @@ type_complexity = "allow"
all = "deny"
[workspace.dependencies]
async-trait = "0.1.74"
async-trait = "0.1.79"
assert_matches = "1.5.0"
base58 = { version = "0.2.0" }
bitvec = { version = "1", default-features = false }
blake2 = { version = "0.10.6", default-features = false }
clap = { version = "4.5.1", features = ["derive", "cargo"] }
clap = { version = "4.5.3", features = ["derive", "cargo"] }
cfg-if = "1.0.0"
criterion = "0.4"
codec = { package = "parity-scale-codec", version = "3.6.9", default-features = false }
color-eyre = "0.6.1"
color-eyre = "0.6.3"
console_error_panic_hook = "0.1.7"
darling = "0.20.8"
derivative = "2.2.0"
derive_more = "0.99.17"
either = "1.10.0"
either = { version = "1.10.0", default-features = false }
frame-metadata = { version = "16.0.0", default-features = false }
futures = { version = "0.3.30", default-features = false, features = ["std"] }
getrandom = { version = "0.2", default-features = false }
hashbrown = "0.14.3"
hex = "0.4.3"
hex = { version = "0.4.3", default-features = false }
heck = "0.4.1"
impl-serde = { version = "0.4.0" }
impl-serde = { version = "0.4.0", default-features = false }
indoc = "2"
jsonrpsee = { version = "0.22" }
pretty_assertions = "1.4.0"
primitive-types = { version = "0.12.2", default-features = false, features = ["codec", "scale-info", "serde"] }
primitive-types = { version = "0.12.2", default-features = false }
proc-macro-error = "1.0.4"
proc-macro2 = "1.0.78"
proc-macro2 = "1.0.79"
quote = "1.0.35"
regex = "1.10.3"
scale-info = { version = "2.10.0", default-features = false }
scale-value = "0.13.0"
scale-bits = "0.4.0"
scale-decode = "0.10.0"
scale-encode = "0.5.0"
serde = { version = "1.0.197" }
serde_json = { version = "1.0.114" }
regex = { version = "1.10.3", default-features = false }
scale-info = { version = "2.11.0", default-features = false }
scale-value = { version = "0.14.1", default-features = false }
scale-bits = { version = "0.5.0", default-features = false }
scale-decode = { version = "0.11.1", default-features = false }
scale-encode = { version = "0.6.0", default-features = false }
serde = { version = "1.0.197", default-features = false, features = ["derive"] }
serde_json = { version = "1.0.114", default-features = false }
syn = { version = "2.0.15", features = ["full", "extra-traits"] }
thiserror = "1.0.57"
thiserror = "1.0.58"
tokio = { version = "1.36", default-features = false }
tracing = "0.1.40"
tracing = { version = "0.1.40", default-features = false }
tracing-wasm = "0.2.1"
tracing-subscriber = "0.3.18"
trybuild = "1.0.89"
trybuild = "1.0.90"
url = "2.5.0"
wabt = "0.10.0"
wasm-bindgen-test = "0.3.24"
which = "5.0.0"
scale-typegen-description = "0.1.0"
scale-typegen = "0.1.1"
scale-typegen-description = "0.2.0"
scale-typegen = "0.2.0"
strip-ansi-escapes = "0.2.0"
# Light client support:
@@ -109,42 +117,42 @@ smoldot-light = { version = "0.14.0", default-features = false }
tokio-stream = "0.1.14"
futures-util = "0.3.30"
rand = "0.8.5"
pin-project = "1.1.4"
pin-project = "1.1.5"
# Light client wasm:
web-sys = { version = "0.3.67", features = ["BinaryType", "CloseEvent", "MessageEvent", "WebSocket"] }
wasm-bindgen = "0.2.90"
web-sys = { version = "0.3.69", features = ["BinaryType", "CloseEvent", "MessageEvent", "WebSocket"] }
wasm-bindgen = "0.2.92"
send_wrapper = "0.6.0"
js-sys = "0.3.68"
wasm-bindgen-futures = "0.4.38"
js-sys = "0.3.69"
wasm-bindgen-futures = "0.4.42"
futures-timer = "3"
instant = { version = "0.1.12", default-features = false }
tokio-util = "0.7.10"
# Substrate crates:
sp-core = { version = "28.0.0", default-features = false }
sp-core-hashing = { version = "15.0.0", default-features = false }
sp-runtime = "31.0.0"
sp-keyring = "31.0.0"
sp-core = { version = "31.0.0", default-features = false }
sp-crypto-hashing = { version = "0.1.0", default-features = false }
sp-runtime = "34.0.0"
sp-keyring = "34.0.0"
# Subxt workspace crates:
subxt = { version = "0.34.0", path = "subxt", default-features = false }
subxt-macro = { version = "0.34.0", path = "macro" }
subxt-metadata = { version = "0.34.0", path = "metadata" }
subxt-codegen = { version = "0.34.0", path = "codegen" }
subxt-signer = { version = "0.34.0", path = "signer" }
subxt-lightclient = { version = "0.34.0", path = "lightclient", default-features = false }
subxt = { version = "0.35.0", path = "subxt", default-features = false }
subxt-macro = { version = "0.35.0", path = "macro" }
subxt-metadata = { version = "0.35.0", path = "metadata", default-features = false }
subxt-codegen = { version = "0.35.0", path = "codegen" }
subxt-signer = { version = "0.35.0", path = "signer", default-features = false }
subxt-lightclient = { version = "0.35.0", path = "lightclient", default-features = false }
test-runtime = { path = "testing/test-runtime" }
substrate-runner = { path = "testing/substrate-runner" }
# subxt-signer deps that I expect aren't useful anywhere else:
bip39 = "2.0.0"
hmac = "0.12.1"
bip39 = { version = "2.0.0", default-features = false }
hmac = { version = "0.12.1", default-features = false }
pbkdf2 = { version = "0.12.2", default-features = false }
schnorrkel = "0.11.4"
secp256k1 = "0.28.2"
schnorrkel = { version = "0.11.4", default-features = false }
secp256k1 = { version = "0.28.2", default-features = false }
secrecy = "0.8.0"
sha2 = "0.10.8"
sha2 = { version = "0.10.8", default-features = false }
zeroize = { version = "1", default-features = false }
[profile.dev.package.smoldot-light]
artifacts/demo_chain_specs/polkadot.json
+4 -4
View File
File diff suppressed because one or more lines are too long
artifacts/polkadot_metadata_full.scale
BIN
View File
Binary file not shown.
artifacts/polkadot_metadata_small.scale
BIN
View File
Binary file not shown.
artifacts/polkadot_metadata_tiny.scale
BIN
View File
Binary file not shown.
cli/Cargo.toml
+1
View File
@@ -15,6 +15,7 @@ description = "Command line utilities for working with subxt codegen"
[[bin]]
name = "subxt"
path = "src/main.rs"
doc = false
[lints]
workspace = true
cli/src/commands/explore/pallets/constants.rs
+5 -2
View File
@@ -64,8 +64,11 @@ pub fn explore_constants(
.highlight();
// value
let value =
scale_value::scale::decode_as_type(&mut constant.value(), constant.ty(), metadata.types())?;
let value = scale_value::scale::decode_as_type(
&mut constant.value(),
&constant.ty(),
metadata.types(),
)?;
let value = format_scale_value(&value).indent(4);
writedoc!(
cli/src/commands/explore/pallets/storage.rs
+1 -1
View File
@@ -169,7 +169,7 @@ pub async fn explore_storage(
{value_str}
"}?;
let key_bytes = value.encode_as_type(type_id, metadata.types())?;
let key_bytes = value.encode_as_type(&type_id, metadata.types())?;
let bytes_composite = Value::from_bytes(key_bytes);
vec![bytes_composite]
}
cli/src/commands/explore/runtime_apis/mod.rs
+3 -4
View File
@@ -19,7 +19,7 @@ use subxt_metadata::RuntimeApiMetadata;
/// Runs for a specified runtime API trait.
/// Cases to consider:
/// ```norun
/// ```txt
/// method is:
/// None => Show pallet docs + available methods
/// Some (invalid) => Show Error + available methods
@@ -161,9 +161,8 @@ pub async fn run<'a>(
{value_str}
"}?;
// encode, then decode. This ensures that the scale value is of the correct shape for the param:
let bytes = value.encode_as_type(ty.ty, metadata.types())?;
let value = Value::decode_as_type(&mut &bytes[..], ty.ty, metadata.types())?
.map_context(|_| ());
let bytes = value.encode_as_type(&ty.ty, metadata.types())?;
let value = Value::decode_as_type(&mut &bytes[..], &ty.ty, metadata.types())?;
Ok(value)
})
.collect::<color_eyre::Result<Vec<Value>>>()?;
codegen/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ syn = { workspace = true }
scale-info = { workspace = true }
subxt-metadata = { workspace = true }
jsonrpsee = { workspace = true, features = ["async-client", "client-ws-transport-native-tls", "http-client"], optional = true }
hex = { workspace = true }
hex = { workspace = true, features = ["std"] }
tokio = { workspace = true, features = ["rt-multi-thread"], optional = true }
thiserror = { workspace = true }
scale-typegen = { workspace = true }
codegen/src/api/storage.rs
+124 -25
View File
@@ -8,7 +8,7 @@ use quote::{format_ident, quote};
use scale_info::TypeDef;
use scale_typegen::{typegen::type_path::TypePath, TypeGenerator};
use subxt_metadata::{
PalletMetadata, StorageEntryMetadata, StorageEntryModifier, StorageEntryType,
PalletMetadata, StorageEntryMetadata, StorageEntryModifier, StorageEntryType, StorageHasher,
};
use super::CodegenError;
@@ -75,8 +75,15 @@ fn generate_storage_entry_fns(
let alias_module_name = format_ident!("{snake_case_name}");
let alias_storage_path = quote!( types::#alias_module_name::#alias_name );
let storage_entry_map = |idx, id| {
let ident: Ident = format_ident!("_{}", idx);
struct MapEntryKey {
arg_name: Ident,
alias_type_def: TokenStream,
alias_type_path: TokenStream,
hasher: StorageHasher,
}
let map_entry_key = |idx, id, hasher| -> MapEntryKey {
let arg_name: Ident = format_ident!("_{}", idx);
let ty_path = type_gen
.resolve_type_path(id)
.expect("type is in metadata; qed");
@@ -84,34 +91,67 @@ fn generate_storage_entry_fns(
let alias_name = format_ident!("Param{}", idx);
let alias_type = primitive_type_alias(&ty_path);
let alias_type = quote!( pub type #alias_name = #alias_type; );
let path_to_alias = quote!( types::#alias_module_name::#alias_name );
let alias_type_def = quote!( pub type #alias_name = #alias_type; );
let alias_type_path = quote!( types::#alias_module_name::#alias_name );
(ident, alias_type, path_to_alias)
MapEntryKey {
arg_name,
alias_type_def,
alias_type_path,
hasher,
}
};
let keys: Vec<(Ident, TokenStream, TokenStream)> = match storage_entry.entry_type() {
let keys: Vec<MapEntryKey> = match storage_entry.entry_type() {
StorageEntryType::Plain(_) => vec![],
StorageEntryType::Map { key_ty, .. } => {
StorageEntryType::Map {
key_ty, hashers, ..
} => {
match &type_gen
.resolve_type(*key_ty)
.expect("key type should be present")
.type_def
{
// An N-map; return each of the keys separately.
TypeDef::Tuple(tuple) => tuple
.fields
.iter()
.enumerate()
.map(|(idx, f)| storage_entry_map(idx, f.id))
.collect::<Vec<_>>(),
TypeDef::Tuple(tuple) => {
let key_count = tuple.fields.len();
let hasher_count = hashers.len();
if hasher_count != 1 && hasher_count != key_count {
return Err(CodegenError::InvalidStorageHasherCount {
storage_entry_name: storage_entry.name().to_owned(),
key_count,
hasher_count,
});
}
let mut map_entry_keys: Vec<MapEntryKey> = vec![];
for (idx, field) in tuple.fields.iter().enumerate() {
// Note: these are in bounds because of the checks above, qed;
let hasher = if idx >= hasher_count {
hashers[0]
} else {
hashers[idx]
};
map_entry_keys.push(map_entry_key(idx, field.id, hasher));
}
map_entry_keys
}
// A map with a single key; return the single key.
_ => {
vec![storage_entry_map(0, *key_ty)]
let Some(hasher) = hashers.first() else {
return Err(CodegenError::InvalidStorageHasherCount {
storage_entry_name: storage_entry.name().to_owned(),
key_count: 1,
hasher_count: 0,
});
};
vec![map_entry_key(0, *key_ty, *hasher)]
}
}
}
};
let pallet_name = pallet.name();
let storage_name = storage_entry.name();
let Some(storage_hash) = pallet.storage_hash(storage_name) else {
@@ -133,6 +173,10 @@ fn generate_storage_entry_fns(
StorageEntryModifier::Optional => quote!(()),
};
// Note: putting `#crate_path::storage::address::StaticStorageKey` into this variable is necessary
// to get the line width below a certain limit. If not done, rustfmt will refuse to format the following big expression.
// for more information see [this post](https://users.rust-lang.org/t/rustfmt-silently-fails-to-work/75485/4).
let static_storage_key: TokenStream = quote!(#crate_path::storage::address::StaticStorageKey);
let all_fns = (0..=keys.len()).map(|n_keys| {
let keys_slice = &keys[..n_keys];
let (fn_name, is_fetchable, is_iterable) = if n_keys == keys.len() {
@@ -146,12 +190,65 @@ fn generate_storage_entry_fns(
};
(fn_name, false, true)
};
let is_fetchable_type = is_fetchable.then_some(quote!(#crate_path::storage::address::Yes)).unwrap_or(quote!(()));
let is_iterable_type = is_iterable.then_some(quote!(#crate_path::storage::address::Yes)).unwrap_or(quote!(()));
let key_impls = keys_slice.iter().map(|(field_name, _, _)| quote!( #crate_path::storage::address::make_static_storage_map_key(#field_name.borrow()) ));
let key_args = keys_slice.iter().map(|(field_name, _, path_to_alias )| {
quote!( #field_name: impl ::std::borrow::Borrow<#path_to_alias> )
});
let is_fetchable_type = is_fetchable
.then_some(quote!(#crate_path::storage::address::Yes))
.unwrap_or(quote!(()));
let is_iterable_type = is_iterable
.then_some(quote!(#crate_path::storage::address::Yes))
.unwrap_or(quote!(()));
let (keys, keys_type) = match keys_slice.len() {
0 => (quote!(()), quote!(())),
1 => {
let key = &keys_slice[0];
if key.hasher.ends_with_key() {
let arg = &key.arg_name;
let keys = quote!(#static_storage_key::new(#arg.borrow()));
let path = &key.alias_type_path;
let path = quote!(#static_storage_key<#path>);
(keys, path)
} else {
(quote!(()), quote!(()))
}
}
_ => {
let keys_iter = keys_slice.iter().map(
|MapEntryKey {
arg_name, hasher, ..
}| {
if hasher.ends_with_key() {
quote!( #static_storage_key::new(#arg_name.borrow()) )
} else {
quote!(())
}
},
);
let keys = quote!( (#(#keys_iter,)*) );
let paths_iter = keys_slice.iter().map(
|MapEntryKey {
alias_type_path,
hasher,
..
}| {
if hasher.ends_with_key() {
quote!( #static_storage_key<#alias_type_path> )
} else {
quote!(())
}
},
);
let paths = quote!( (#(#paths_iter,)*) );
(keys, paths)
}
};
let key_args = keys_slice.iter().map(
|MapEntryKey {
arg_name,
alias_type_path,
..
}| quote!( #arg_name: impl ::std::borrow::Borrow<#alias_type_path> ),
);
quote!(
#docs
@@ -159,7 +256,7 @@ fn generate_storage_entry_fns(
&self,
#(#key_args,)*
) -> #crate_path::storage::address::Address::<
#crate_path::storage::address::StaticStorageMapKey,
#keys_type,
#alias_storage_path,
#is_fetchable_type,
#is_defaultable_type,
@@ -168,14 +265,16 @@ fn generate_storage_entry_fns(
#crate_path::storage::address::Address::new_static(
#pallet_name,
#storage_name,
vec![#(#key_impls,)*],
#keys,
[#(#storage_hash,)*]
)
}
)
});
let alias_types = keys.iter().map(|(_, alias_type, _)| alias_type);
let alias_types = keys
.iter()
.map(|MapEntryKey { alias_type_def, .. }| alias_type_def);
let types_mod_ident = type_gen.types_mod_ident();
// Generate type alias for the return type only, since
@@ -231,7 +330,7 @@ mod tests {
name,
modifier: v15::StorageEntryModifier::Optional,
ty: v15::StorageEntryType::Map {
hashers: vec![],
hashers: vec![v15::StorageHasher::Blake2_128Concat],
key,
value: meta_type::<bool>(),
},
codegen/src/error.rs
+12 -6
View File
@@ -39,15 +39,21 @@ pub enum CodegenError {
#[error("Call variant for type {0} must have all named fields. Make sure you are providing a valid substrate-based metadata")]
InvalidCallVariant(u32),
/// Type should be an variant/enum.
#[error(
"{0} type should be an variant/enum type. Make sure you are providing a valid substrate-based metadata"
)]
#[error("{0} type should be an variant/enum type. Make sure you are providing a valid substrate-based metadata")]
InvalidType(String),
/// Extrinsic call type could not be found.
#[error(
"Extrinsic call type could not be found. Make sure you are providing a valid substrate-based metadata"
)]
#[error("Extrinsic call type could not be found. Make sure you are providing a valid substrate-based metadata")]
MissingCallType,
/// There are too many or too few hashers.
#[error("Could not generate functions for storage entry {storage_entry_name}. There are {key_count} keys, but only {hasher_count} hashers. The number of hashers must equal the number of keys or be exactly 1.")]
InvalidStorageHasherCount {
/// The name of the storage entry
storage_entry_name: String,
/// Number of keys
key_count: usize,
/// Number of hashers
hasher_count: usize,
},
/// Cannot generate types.
#[error("Type Generation failed: {0}")]
TypeGeneration(#[from] TypegenError),
examples/parachain-example/Cargo.lock
Generated
+10 -10
View File
@@ -2512,7 +2512,7 @@ dependencies = [
]
[[package]]
name = "sp-core-hashing"
name = "sp-crypto-hashing"
version = "13.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb8524f01591ee58b46cd83c9dbc0fcffd2fd730dabec4f59326cd58a00f17e2"
@@ -2551,7 +2551,7 @@ checksum = "81cdd64d312baedb58e21336b31bc043b77e01cc99033ce76ef539f78e965ebc"
[[package]]
name = "subxt"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"async-trait",
"base58",
@@ -2573,7 +2573,7 @@ dependencies = [
"scale-value",
"serde",
"serde_json",
"sp-core-hashing",
"sp-crypto-hashing",
"subxt-lightclient",
"subxt-macro",
"subxt-metadata",
@@ -2585,7 +2585,7 @@ dependencies = [
[[package]]
name = "subxt-codegen"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"frame-metadata 16.0.0",
"heck",
@@ -2604,7 +2604,7 @@ dependencies = [
[[package]]
name = "subxt-lightclient"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"futures",
"futures-util",
@@ -2619,7 +2619,7 @@ dependencies = [
[[package]]
name = "subxt-macro"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"darling 0.20.3",
"parity-scale-codec",
@@ -2632,18 +2632,18 @@ dependencies = [
[[package]]
name = "subxt-metadata"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"frame-metadata 16.0.0",
"parity-scale-codec",
"scale-info",
"sp-core-hashing",
"sp-crypto-hashing",
"thiserror",
]
[[package]]
name = "subxt-signer"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bip39",
"hex",
@@ -2655,7 +2655,7 @@ dependencies = [
"secp256k1",
"secrecy",
"sha2 0.10.8",
"sp-core-hashing",
"sp-crypto-hashing",
"subxt",
"thiserror",
"zeroize",
examples/wasm-example/Cargo.lock
Generated
+88 -172
View File
@@ -466,43 +466,19 @@ dependencies = [
[[package]]
name = "darling"
version = "0.14.4"
version = "0.20.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b750cb3417fd1b327431a470f388520309479ab0bf5e323505daf0290cd3850"
checksum = "54e36fcd13ed84ffdfda6f5be89b31287cbb80c439841fe69e04841435464391"
dependencies = [
"darling_core 0.14.4",
"darling_macro 0.14.4",
]
[[package]]
name = "darling"
version = "0.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0209d94da627ab5605dcccf08bb18afa5009cfbef48d8a8b7d7bdbc79be25c5e"
dependencies = [
"darling_core 0.20.3",
"darling_macro 0.20.3",
"darling_core",
"darling_macro",
]
[[package]]
name = "darling_core"
version = "0.14.4"
version = "0.20.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "109c1ca6e6b7f82cc233a97004ea8ed7ca123a9af07a8230878fcfda9b158bf0"
dependencies = [
"fnv",
"ident_case",
"proc-macro2",
"quote",
"strsim",
"syn 1.0.109",
]
[[package]]
name = "darling_core"
version = "0.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "177e3443818124b357d8e76f53be906d60937f0d3a90773a664fa63fa253e621"
checksum = "9c2cf1c23a687a1feeb728783b993c4e1ad83d99f351801977dd809b48d0a70f"
dependencies = [
"fnv",
"ident_case",
@@ -514,22 +490,11 @@ dependencies = [
[[package]]
name = "darling_macro"
version = "0.14.4"
version = "0.20.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4aab4dbc9f7611d8b55048a3a16d2d010c2c8334e46304b40ac1cc14bf3b48e"
checksum = "a668eda54683121533a393014d8692171709ff57a7d61f187b6e782719f8933f"
dependencies = [
"darling_core 0.14.4",
"quote",
"syn 1.0.109",
]
[[package]]
name = "darling_macro"
version = "0.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "836a9bbc7ad63342d6d6e7b815ccab164bc77a2d95d84bc3117a8c0d5c98e2d5"
dependencies = [
"darling_core 0.20.3",
"darling_core",
"quote",
"syn 2.0.48",
]
@@ -610,9 +575,9 @@ dependencies = [
[[package]]
name = "either"
version = "1.9.0"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a26ae43d7bcc3b814de94796a5e736d4029efb0ee900c12e2d54c993ad1a1e07"
checksum = "11157ac094ffbdde99aa67b23417ebdd801842852b500e395a45a9c0aac03e4a"
[[package]]
name = "equivalent"
@@ -653,9 +618,6 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "835c052cb0c08c1acf6ffd71c022172e18723949c8282f2b9f27efbc51e64534"
dependencies = [
"byteorder",
"rand",
"rustc-hex",
"static_assertions",
]
@@ -694,7 +656,6 @@ dependencies = [
"cfg-if",
"parity-scale-codec",
"scale-info",
"serde",
]
[[package]]
@@ -1227,15 +1188,6 @@ dependencies = [
"unicode-normalization",
]
[[package]]
name = "impl-codec"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba6a270039626615617f3f36d15fc827041df3b78c439da2cadfa47455a77f2f"
dependencies = [
"parity-scale-codec",
]
[[package]]
name = "impl-serde"
version = "0.4.0"
@@ -1329,18 +1281,18 @@ checksum = "b1a46d1a171d865aa5f83f92695765caa047a9b4cbae2cbf37dbd613a793fd4c"
[[package]]
name = "js-sys"
version = "0.3.67"
version = "0.3.69"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a1d36f1235bc969acba30b7f5990b864423a6068a10f7c90ae8f0112e3a59d1"
checksum = "29c15563dc2726973df627357ce0c9ddddbea194836909d655df6a75d2cf296d"
dependencies = [
"wasm-bindgen",
]
[[package]]
name = "jsonrpsee"
version = "0.21.0"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9579d0ca9fb30da026bac2f0f7d9576ec93489aeb7cd4971dd5b4617d82c79b2"
checksum = "16fcc9dd231e72d22993f1643d5f7f0db785737dbe3c3d7ca222916ab4280795"
dependencies = [
"jsonrpsee-client-transport",
"jsonrpsee-core",
@@ -1350,9 +1302,9 @@ dependencies = [
[[package]]
name = "jsonrpsee-client-transport"
version = "0.21.0"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9f9ed46590a8d5681975f126e22531698211b926129a40a2db47cbca429220"
checksum = "0476c96eb741b40d39dcb39d0124e3b9be9840ec77653c42a0996563ae2a53f7"
dependencies = [
"futures-channel",
"futures-util",
@@ -1373,9 +1325,9 @@ dependencies = [
[[package]]
name = "jsonrpsee-core"
version = "0.21.0"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "776d009e2f591b78c038e0d053a796f94575d66ca4e77dd84bfc5e81419e436c"
checksum = "b974d8f6139efbe8425f32cb33302aba6d5e049556b5bfc067874e7a0da54a2e"
dependencies = [
"anyhow",
"async-lock",
@@ -1398,9 +1350,9 @@ dependencies = [
[[package]]
name = "jsonrpsee-http-client"
version = "0.21.0"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78b7de9f3219d95985eb77fd03194d7c1b56c19bce1abfcc9d07462574b15572"
checksum = "19dc795a277cff37f27173b3ca790d042afcc0372c34a7ca068d2e76de2cb6d1"
dependencies = [
"async-trait",
"hyper",
@@ -1418,9 +1370,9 @@ dependencies = [
[[package]]
name = "jsonrpsee-types"
version = "0.21.0"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3266dfb045c9174b24c77c2dfe0084914bb23a6b2597d70c9dc6018392e1cd1b"
checksum = "b13dac43c1a9fc2648b37f306b0a5b0e29b2a6e1c36a33b95c1948da2494e9c5"
dependencies = [
"anyhow",
"beef",
@@ -1713,18 +1665,18 @@ checksum = "e3148f5046208a5d56bcfc03053e3ca6334e51da8dfb19b6cdc8b306fae3283e"
[[package]]
name = "pin-project"
version = "1.1.3"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fda4ed1c6c173e3fc7a83629421152e01d7b1f9b7f65fb301e490e8cfc656422"
checksum = "b6bf43b791c5b9e34c3d182969b4abb522f9343702850a2e57f460d00d09b4b3"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.3"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4359fd9c9171ec6e8c62926d6faaf553a8dc3f64e1507e76da7911b4f6a04405"
checksum = "2f38a4412a78282e09a2cf38d195ea5420d15ba0602cb375210efbc877243965"
dependencies = [
"proc-macro2",
"quote",
@@ -1794,9 +1746,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b34d9fd68ae0b74a41b21c03c2f62847aa0ffea044eee893b4c140b37e244e2"
dependencies = [
"fixed-hash",
"impl-codec",
"impl-serde",
"scale-info",
"uint",
]
@@ -1947,12 +1896,6 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hex"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e75f6a532d0fd9f7f13144f392b6ad56a32696bfcd9c78f797f16bbb6f072d6"
[[package]]
name = "rustc_version"
version = "0.4.0"
@@ -2084,71 +2027,42 @@ checksum = "f98d2aa92eebf49b69786be48e4477826b256916e84a57ff2a4f21923b48eb4c"
[[package]]
name = "scale-bits"
version = "0.4.0"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "036575c29af9b6e4866ffb7fa055dbf623fe7a9cc159b33786de6013a6969d89"
checksum = "662d10dcd57b1c2a3c41c9cf68f71fb09747ada1ea932ad961aca7e2ca28315f"
dependencies = [
"parity-scale-codec",
"scale-info",
"scale-type-resolver",
"serde",
]
[[package]]
name = "scale-decode"
version = "0.10.0"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7caaf753f8ed1ab4752c6afb20174f03598c664724e0e32628e161c21000ff76"
checksum = "afc79ba56a1c742f5aeeed1f1801f3edf51f7e818f0a54582cac6f131364ea7b"
dependencies = [
"derive_more",
"parity-scale-codec",
"primitive-types",
"scale-bits",
"scale-decode-derive",
"scale-info",
"scale-type-resolver",
"smallvec",
]
[[package]]
name = "scale-decode-derive"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3475108a1b62c7efd1b5c65974f30109a598b2f45f23c9ae030acb9686966db"
dependencies = [
"darling 0.14.4",
"proc-macro-crate 1.3.1",
"proc-macro2",
"quote",
"syn 1.0.109",
]
[[package]]
name = "scale-encode"
version = "0.5.0"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d70cb4b29360105483fac1ed567ff95d65224a14dd275b6303ed0a654c78de5"
checksum = "628800925a33794fb5387781b883b5e14d130fece9af5a63613867b8de07c5c7"
dependencies = [
"derive_more",
"parity-scale-codec",
"primitive-types",
"scale-bits",
"scale-encode-derive",
"scale-info",
"scale-type-resolver",
"smallvec",
]
[[package]]
name = "scale-encode-derive"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "995491f110efdc6bea96d6a746140e32bfceb4ea47510750a5467295a4707a25"
dependencies = [
"darling 0.14.4",
"proc-macro-crate 1.3.1",
"proc-macro2",
"quote",
"syn 1.0.109",
]
[[package]]
name = "scale-info"
version = "2.10.0"
@@ -2176,10 +2090,20 @@ dependencies = [
]
[[package]]
name = "scale-typegen"
name = "scale-type-resolver"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00860983481ac590ac87972062909bef0d6a658013b592ccc0f2feb272feab11"
checksum = "10b800069bfd43374e0f96f653e0d46882a2cb16d6d961ac43bea80f26c76843"
dependencies = [
"scale-info",
"smallvec",
]
[[package]]
name = "scale-typegen"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d6108609f017741c78d35967c7afe4aeaa3999b848282581041428e10d23b63"
dependencies = [
"proc-macro2",
"quote",
@@ -2190,12 +2114,10 @@ dependencies = [
[[package]]
name = "scale-value"
version = "0.13.0"
version = "0.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "58223c7691bf0bd46b43c9aea6f0472d1067f378d574180232358d7c6e0a8089"
checksum = "c07ccfee963104335c971aaf8b7b0e749be8569116322df23f1f75c4ca9e4a28"
dependencies = [
"base58",
"blake2",
"derive_more",
"either",
"frame-metadata 15.1.0",
@@ -2204,8 +2126,7 @@ dependencies = [
"scale-decode",
"scale-encode",
"scale-info",
"serde",
"yap",
"scale-type-resolver",
]
[[package]]
@@ -2289,9 +2210,9 @@ checksum = "cd0b0ec5f1c1ca621c432a25813d8d60c88abe6d3e08a3eb9cf37d97a0fe3d73"
[[package]]
name = "serde"
version = "1.0.195"
version = "1.0.197"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63261df402c67811e9ac6def069e4786148c4563f4b50fd4bf30aa370d626b02"
checksum = "3fb1c873e1b9b056a4dc4c0c198b24c3ffa059243875552b2bd0933b1aee4ce2"
dependencies = [
"serde_derive",
]
@@ -2318,9 +2239,9 @@ dependencies = [
[[package]]
name = "serde_derive"
version = "1.0.195"
version = "1.0.197"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46fe8f8603d81ba86327b23a2e9cdf49e1255fb94a4c5f297f6ee0547178ea2c"
checksum = "7eb0b34b42edc17f6b7cac84a52a1c5f0e1bb2227e997ca9011ea3dd34e8610b"
dependencies = [
"proc-macro2",
"quote",
@@ -2329,9 +2250,9 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.111"
version = "1.0.114"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "176e46fa42316f18edd598015a5166857fc835ec732f5215eac6b7bdbf0a84f4"
checksum = "c5f09b1bd632ef549eaa9f60a1f8de742bdbc698e6cee2095fc84dde5f549ae0"
dependencies = [
"itoa",
"ryu",
@@ -2539,10 +2460,10 @@ dependencies = [
]
[[package]]
name = "sp-core-hashing"
version = "13.0.0"
name = "sp-crypto-hashing"
version = "15.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb8524f01591ee58b46cd83c9dbc0fcffd2fd730dabec4f59326cd58a00f17e2"
checksum = "1e0f4990add7b2cefdeca883c0efa99bb4d912cb2196120e1500c0cc099553b0"
dependencies = [
"blake2b_simd",
"byteorder",
@@ -2578,7 +2499,7 @@ checksum = "81cdd64d312baedb58e21336b31bc043b77e01cc99033ce76ef539f78e965ebc"
[[package]]
name = "subxt"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"async-trait",
"base58",
@@ -2601,7 +2522,7 @@ dependencies = [
"scale-value",
"serde",
"serde_json",
"sp-core-hashing",
"sp-crypto-hashing",
"subxt-lightclient",
"subxt-macro",
"subxt-metadata",
@@ -2612,7 +2533,7 @@ dependencies = [
[[package]]
name = "subxt-codegen"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"frame-metadata 16.0.0",
"getrandom",
@@ -2632,7 +2553,7 @@ dependencies = [
[[package]]
name = "subxt-lightclient"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"futures",
"futures-timer",
@@ -2657,9 +2578,9 @@ dependencies = [
[[package]]
name = "subxt-macro"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"darling 0.20.3",
"darling",
"parity-scale-codec",
"proc-macro-error",
"quote",
@@ -2670,13 +2591,14 @@ dependencies = [
[[package]]
name = "subxt-metadata"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"derive_more",
"frame-metadata 16.0.0",
"hashbrown 0.14.3",
"parity-scale-codec",
"scale-info",
"sp-core-hashing",
"thiserror",
"sp-crypto-hashing",
]
[[package]]
@@ -2709,18 +2631,18 @@ checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "thiserror"
version = "1.0.56"
version = "1.0.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d54378c645627613241d077a3a79db965db602882668f9136ac42af9ecb730ad"
checksum = "1e45bcbe8ed29775f228095caf2cd67af7a4ccf756ebff23a306bf3e8b47b24b"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "1.0.56"
version = "1.0.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa0faa943b50f3db30a20aa7e265dbc66076993efed8463e8de414e5d06d3471"
checksum = "a953cb265bef375dae3de6663da4d3804eee9682ea80d8e2542529b73c531c81"
dependencies = [
"proc-macro2",
"quote",
@@ -2744,9 +2666,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.35.1"
version = "1.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c89b4efa943be685f629b149f53829423f8f5531ea21249408e8e2f8671ec104"
checksum = "61285f6515fa018fb2d1e46eb21223fff441ee8db5d0f1435e8ab4f5cdb80931"
dependencies = [
"backtrace",
"bytes",
@@ -3010,9 +2932,9 @@ checksum = "9c8d87e72b64a3b4db28d11ce29237c246188f4f51057d65a7eab63b7987e423"
[[package]]
name = "wasm-bindgen"
version = "0.2.90"
version = "0.2.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1223296a201415c7fad14792dbefaace9bd52b62d33453ade1c5b5f07555406"
checksum = "4be2531df63900aeb2bca0daaaddec08491ee64ceecbee5076636a3b026795a8"
dependencies = [
"cfg-if",
"wasm-bindgen-macro",
@@ -3020,9 +2942,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-backend"
version = "0.2.90"
version = "0.2.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fcdc935b63408d58a32f8cc9738a0bffd8f05cc7c002086c6ef20b7312ad9dcd"
checksum = "614d787b966d3989fa7bb98a654e369c762374fd3213d212cfc0251257e747da"
dependencies = [
"bumpalo",
"log",
@@ -3047,9 +2969,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.90"
version = "0.2.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e4c238561b2d428924c49815533a8b9121c664599558a5d9ec51f8a1740a999"
checksum = "a1f8823de937b71b9460c0c34e25f3da88250760bec0ebac694b49997550d726"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@@ -3057,9 +2979,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.90"
version = "0.2.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bae1abb6806dc1ad9e560ed242107c0f6c84335f1749dd4e8ddb012ebd5e25a7"
checksum = "e94f17b526d0a461a191c78ea52bbce64071ed5c04c9ffe424dcb38f74171bb7"
dependencies = [
"proc-macro2",
"quote",
@@ -3070,9 +2992,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.90"
version = "0.2.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4d91413b1c31d7539ba5ef2451af3f0b833a005eb27a631cec32bc0635a8602b"
checksum = "af190c94f2773fdb3729c55b007a722abb5384da03bc0986df4c289bf5567e96"
[[package]]
name = "wasm-example"
@@ -3134,9 +3056,9 @@ dependencies = [
[[package]]
name = "web-sys"
version = "0.3.67"
version = "0.3.69"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "58cd2333b6e0be7a39605f0e255892fd7418a682d8da8fe042fe25128794d2ed"
checksum = "77afa9a11836342370f4817622a2f0f418b134426d91a82dfb48f532d2ec13ef"
dependencies = [
"js-sys",
"wasm-bindgen",
@@ -3304,12 +3226,6 @@ dependencies = [
"zeroize",
]
[[package]]
name = "yap"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff4524214bc4629eba08d78ceb1d6507070cc0bcbbed23af74e19e6e924a24cf"
[[package]]
name = "yew"
version = "0.20.0"
examples/wasm-example/Cargo.toml
+2 -2
View File
@@ -9,10 +9,10 @@ edition = "2021"
futures = "0.3.28"
subxt = { path = "../../subxt", default-features = false, features = ["jsonrpsee", "web"], target_arch = "wasm32" }
yew = { version = "0.20.0", features = ["csr"] }
web-sys = "0.3.63"
web-sys = "0.3.69"
hex = "0.4.3"
yew-router = "0.17.0"
js-sys = "0.3.63"
js-sys = "0.3.69"
wasm-bindgen = "0.2.86"
wasm-bindgen-futures = "0.4.36"
anyhow = "1.0.71"
examples/wasm-example/src/routes/signing.rs
+3 -1
View File
@@ -7,6 +7,7 @@ use subxt::ext::codec::{Decode, Encode};
use subxt::tx::SubmittableExtrinsic;
use subxt::tx::TxPayload;
use subxt::utils::{AccountId32, MultiSignature};
use subxt::config::DefaultExtrinsicParamsBuilder;
use crate::services::{extension_signature_for_extrinsic, get_accounts, polkadot, Account};
use web_sys::HtmlInputElement;
@@ -155,7 +156,8 @@ impl Component for SigningExamplesComponent {
return Message::Error(anyhow!("MultiSignature Decoding"));
};
let Ok(partial_signed) = api.tx().create_partial_signed_with_nonce(&remark_call, account_nonce, Default::default()) else {
let params = DefaultExtrinsicParamsBuilder::new().nonce(account_nonce).build();
let Ok(partial_signed) = api.tx().create_partial_signed_offline(&remark_call, params) else {
return Message::Error(anyhow!("PartialExtrinsic creation failed"));
};
lightclient/Cargo.toml
+9 -21
View File
@@ -24,23 +24,14 @@ default = ["native"]
# Exactly 1 of "web" and "native" is expected.
native = [
"smoldot-light/std",
"tokio-stream",
"tokio/sync",
"tokio/rt",
"futures-util",
]
# Enable this for web/wasm builds.
# Exactly 1 of "web" and "native" is expected.
web = [
"getrandom/js",
"smoldot",
"smoldot/std",
"smoldot-light",
"tokio-stream",
"tokio/sync",
"futures-util",
# For the light-client platform.
"wasm-bindgen-futures",
@@ -56,29 +47,26 @@ web = [
]
[dependencies]
futures = { workspace = true }
futures = { workspace = true, features = ["async-await"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true, features = ["raw_value"] }
serde_json = { workspace = true, features = ["default", "raw_value"] }
thiserror = { workspace = true }
tracing = { workspace = true }
smoldot-light = { workspace = true }
tokio-stream = { workspace = true }
tokio = { workspace = true, features = ["sync"] }
futures-util = { workspace = true }
# Light client support:
smoldot = { workspace = true, optional = true }
smoldot-light = { workspace = true, optional = true }
either = { workspace = true, optional = true }
tokio = { workspace = true, optional = true }
tokio-stream = { workspace = true, optional = true }
futures-util = { workspace = true, optional = true }
# Only needed for web
js-sys = { workspace = true, optional = true }
send_wrapper = { workspace = true, optional = true }
web-sys = { workspace = true, optional = true }
wasm-bindgen = { workspace = true, optional = true }
wasm-bindgen-futures = { workspace = true, optional = true }
smoldot = { workspace = true, optional = true }
pin-project = { workspace = true, optional = true }
futures-timer = { workspace = true, optional = true }
instant = { workspace = true, optional = true }
pin-project = { workspace = true, optional = true }
# Included if "web" feature is enabled, to enable its js feature.
getrandom = { workspace = true, optional = true }
[package.metadata.docs.rs]
lightclient/src/background.rs
+330 -363
View File
@@ -1,43 +1,47 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// Copyright 2019-2024 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use futures::stream::StreamExt;
use futures_util::future::{self, Either};
use serde::Deserialize;
use crate::rpc::RpcResponse;
use crate::shared_client::SharedClient;
use crate::{JsonRpcError, LightClientRpcError};
use futures::{stream::StreamExt, FutureExt};
use serde_json::value::RawValue;
use smoldot_light::platform::PlatformRef;
use std::{collections::HashMap, str::FromStr};
use tokio::sync::{mpsc, oneshot};
use tokio_stream::wrappers::UnboundedReceiverStream;
use crate::client::AddedChain;
const LOG_TARGET: &str = "subxt-light-client-background-task";
use super::LightClientRpcError;
use smoldot_light::ChainId;
const LOG_TARGET: &str = "subxt-light-client-background";
/// The response of an RPC method.
/// Response from [`BackgroundTaskHandle::request()`].
pub type MethodResponse = Result<Box<RawValue>, LightClientRpcError>;
/// Response from [`BackgroundTaskHandle::subscribe()`].
pub type SubscriptionResponse = Result<
(
SubscriptionId,
mpsc::UnboundedReceiver<Result<Box<RawValue>, JsonRpcError>>,
),
LightClientRpcError,
>;
/// Type of subscription IDs we can get back.
pub type SubscriptionId = String;
/// Message protocol between the front-end client that submits the RPC requests
/// and the backend handler that produces responses from the chain.
///
/// The light client uses a single object [`smoldot_light::JsonRpcResponses`] to
/// handle all requests and subscriptions from a chain. A background task is spawned
/// to multiplex the rpc responses and to provide them back to their rightful submitters.
/// and the background task which fetches responses from Smoldot. Hidden behind
/// the [`BackgroundTaskHandle`].
#[derive(Debug)]
pub enum FromSubxt {
enum Message {
/// The RPC method request.
Request {
/// The method of the request.
method: String,
/// The parameters of the request.
params: String,
/// Channel used to send back the result.
params: Option<Box<RawValue>>,
/// Channel used to send back the method response.
sender: oneshot::Sender<MethodResponse>,
/// The ID of the chain used to identify the chain.
chain_id: ChainId,
},
/// The RPC subscription (pub/sub) request.
Subscription {
@@ -46,29 +50,160 @@ pub enum FromSubxt {
/// The method to unsubscribe.
unsubscribe_method: String,
/// The parameters of the request.
params: String,
/// Channel used to send back the subscription ID if successful.
sub_id: oneshot::Sender<MethodResponse>,
/// Channel used to send back the notifications.
sender: mpsc::UnboundedSender<Box<RawValue>>,
/// The ID of the chain used to identify the chain.
chain_id: ChainId,
params: Option<Box<RawValue>>,
/// Channel used to send back the subscription response.
sender: oneshot::Sender<SubscriptionResponse>,
},
}
/// Background task data.
#[allow(clippy::type_complexity)]
pub struct BackgroundTask<TPlatform: PlatformRef, TChain> {
/// Smoldot light client implementation that leverages the exposed platform.
client: smoldot_light::Client<TPlatform, TChain>,
/// Per-chain data.
chain_data: HashMap<smoldot_light::ChainId, ChainData>,
/// A handle to communicate with the background task.
#[derive(Clone, Debug)]
pub struct BackgroundTaskHandle {
to_backend: mpsc::UnboundedSender<Message>,
}
/// The data that we store for each chain.
#[derive(Default)]
struct ChainData {
/// Generates an unique monotonically increasing ID for each chain.
impl BackgroundTaskHandle {
/// Make an RPC request via the background task.
pub async fn request(&self, method: String, params: Option<Box<RawValue>>) -> MethodResponse {
let (tx, rx) = oneshot::channel();
self.to_backend
.send(Message::Request {
method,
params,
sender: tx,
})
.map_err(|_e| LightClientRpcError::BackgroundTaskDropped)?;
match rx.await {
Err(_e) => Err(LightClientRpcError::BackgroundTaskDropped),
Ok(response) => response,
}
}
/// Subscribe to some RPC method via the background task.
pub async fn subscribe(
&self,
method: String,
params: Option<Box<RawValue>>,
unsubscribe_method: String,
) -> SubscriptionResponse {
let (tx, rx) = oneshot::channel();
self.to_backend
.send(Message::Subscription {
method,
params,
unsubscribe_method,
sender: tx,
})
.map_err(|_e| LightClientRpcError::BackgroundTaskDropped)?;
match rx.await {
Err(_e) => Err(LightClientRpcError::BackgroundTaskDropped),
Ok(response) => response,
}
}
}
/// A background task which runs with [`BackgroundTask::run()`] and manages messages
/// coming to/from Smoldot.
#[allow(clippy::type_complexity)]
pub struct BackgroundTask<TPlatform: PlatformRef, TChain> {
channels: BackgroundTaskChannels,
data: BackgroundTaskData<TPlatform, TChain>,
}
impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
/// Constructs a new [`BackgroundTask`].
pub(crate) fn new(
client: SharedClient<TPlatform, TChain>,
chain_id: smoldot_light::ChainId,
from_back: smoldot_light::JsonRpcResponses,
) -> (BackgroundTask<TPlatform, TChain>, BackgroundTaskHandle) {
let (tx, rx) = mpsc::unbounded_channel();
let bg_task = BackgroundTask {
channels: BackgroundTaskChannels {
from_front: UnboundedReceiverStream::new(rx),
from_back,
},
data: BackgroundTaskData {
client,
chain_id,
last_request_id: 0,
pending_subscriptions: HashMap::new(),
requests: HashMap::new(),
subscriptions: HashMap::new(),
},
};
let bg_handle = BackgroundTaskHandle { to_backend: tx };
(bg_task, bg_handle)
}
/// Run the background task, which:
/// - Forwards messages/subscription requests to Smoldot from the front end.
/// - Forwards responses back from Smoldot to the front end.
pub async fn run(self) {
let chain_id = self.data.chain_id;
let mut channels = self.channels;
let mut data = self.data;
loop {
tokio::pin! {
let from_front_fut = channels.from_front.next().fuse();
let from_back_fut = channels.from_back.next().fuse();
}
futures::select! {
// Message coming from the front end/client.
front_message = from_front_fut => {
let Some(message) = front_message else {
tracing::trace!(target: LOG_TARGET, "Subxt channel closed");
break;
};
tracing::trace!(
target: LOG_TARGET,
"Received register message {:?}",
message
);
data.handle_requests(message).await;
},
// Message coming from Smoldot.
back_message = from_back_fut => {
let Some(back_message) = back_message else {
tracing::trace!(target: LOG_TARGET, "Smoldot RPC responses channel closed");
break;
};
tracing::trace!(
target: LOG_TARGET,
"Received smoldot RPC chain {:?} result {:?}",
chain_id, back_message
);
data.handle_rpc_response(back_message);
}
}
}
tracing::trace!(target: LOG_TARGET, "Task closed");
}
}
struct BackgroundTaskChannels {
/// Messages sent into this background task from the front end.
from_front: UnboundedReceiverStream<Message>,
/// Messages sent into the background task from Smoldot.
from_back: smoldot_light::JsonRpcResponses,
}
struct BackgroundTaskData<TPlatform: PlatformRef, TChain> {
/// A smoldot light client that can be shared.
client: SharedClient<TPlatform, TChain>,
/// Knowing the chain ID helps with debugging, but isn't overwise necessary.
chain_id: smoldot_light::ChainId,
/// Know which Id to use next for new requests/subscriptions.
last_request_id: usize,
/// Map the request ID of a RPC method to the frontend `Sender`.
requests: HashMap<usize, oneshot::Sender<MethodResponse>>,
@@ -78,20 +213,12 @@ struct ChainData {
/// The RPC method request is made in the background and the response should
/// not be sent back to the user.
/// Map the request ID of a RPC method to the frontend `Sender`.
id_to_subscription: HashMap<usize, PendingSubscription>,
pending_subscriptions: HashMap<usize, PendingSubscription>,
/// Map the subscription ID to the frontend `Sender`.
///
/// The subscription ID is entirely generated by the node (smoldot). Therefore, it is
/// possible for two distinct subscriptions of different chains to have the same subscription ID.
subscriptions: HashMap<usize, ActiveSubscription>,
}
impl ChainData {
/// Fetch and increment the request ID.
fn next_id(&mut self) -> usize {
self.last_request_id = self.last_request_id.wrapping_add(1);
self.last_request_id
}
subscriptions: HashMap<String, ActiveSubscription>,
}
/// The state needed to resolve the subscription ID and send
@@ -100,72 +227,52 @@ struct PendingSubscription {
/// Send the method response ID back to the user.
///
/// It contains the subscription ID if successful, or an JSON RPC error object.
sub_id_sender: oneshot::Sender<MethodResponse>,
/// The subscription state that is added to the `subscriptions` map only
/// if the subscription ID is successfully sent back to the user.
subscription_state: ActiveSubscription,
}
impl PendingSubscription {
/// Transforms the pending subscription into an active subscription.
fn into_parts(self) -> (oneshot::Sender<MethodResponse>, ActiveSubscription) {
(self.sub_id_sender, self.subscription_state)
}
}
/// The state of the subscription.
struct ActiveSubscription {
/// Channel to send the subscription notifications back to frontend.
sender: mpsc::UnboundedSender<Box<RawValue>>,
response_sender: oneshot::Sender<SubscriptionResponse>,
/// The unsubscribe method to call when the user drops the receiver
/// part of the channel.
unsubscribe_method: String,
}
impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
/// Constructs a new [`BackgroundTask`].
pub fn new(
client: smoldot_light::Client<TPlatform, TChain>,
) -> BackgroundTask<TPlatform, TChain> {
BackgroundTask {
client,
chain_data: Default::default(),
}
}
/// The state of the subscription.
struct ActiveSubscription {
/// Channel to send the subscription notifications back to frontend.
notification_sender: mpsc::UnboundedSender<Result<Box<RawValue>, JsonRpcError>>,
/// The unsubscribe method to call when the user drops the receiver
/// part of the channel.
unsubscribe_method: String,
}
fn for_chain_id(
&mut self,
chain_id: smoldot_light::ChainId,
) -> (
&mut ChainData,
&mut smoldot_light::Client<TPlatform, TChain>,
) {
let chain_data = self.chain_data.entry(chain_id).or_default();
let client = &mut self.client;
(chain_data, client)
impl<TPlatform: PlatformRef, TChain> BackgroundTaskData<TPlatform, TChain> {
/// Fetch and increment the request ID.
fn next_id(&mut self) -> usize {
self.last_request_id = self.last_request_id.wrapping_add(1);
self.last_request_id
}
/// Handle the registration messages received from the user.
async fn handle_requests(&mut self, message: FromSubxt) {
async fn handle_requests(&mut self, message: Message) {
match message {
FromSubxt::Request {
Message::Request {
method,
params,
sender,
chain_id,
} => {
let (chain_data, client) = self.for_chain_id(chain_id);
let id = chain_data.next_id();
let id = self.next_id();
let chain_id = self.chain_id;
let params = match &params {
Some(params) => params.get(),
None => "null",
};
let request = format!(
r#"{{"jsonrpc":"2.0","id":"{}", "method":"{}","params":{}}}"#,
id, method, params
);
chain_data.requests.insert(id, sender);
self.requests.insert(id, sender);
tracing::trace!(target: LOG_TARGET, "Tracking request id={id} chain={chain_id:?}");
let result = client.json_rpc_request(request, chain_id);
let result = self.client.json_rpc_request(request, chain_id);
if let Err(err) = result {
tracing::warn!(
target: LOG_TARGET,
@@ -173,14 +280,14 @@ impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
err.to_string()
);
let sender = chain_data
let sender = self
.requests
.remove(&id)
.expect("Channel is inserted above; qed");
// Send the error back to frontend.
if sender
.send(Err(LightClientRpcError::Request(err.to_string())))
.send(Err(LightClientRpcError::SmoldotError(err.to_string())))
.is_err()
{
tracing::warn!(
@@ -192,52 +299,49 @@ impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
tracing::trace!(target: LOG_TARGET, "Submitted to smoldot request with id={id}");
}
}
FromSubxt::Subscription {
Message::Subscription {
method,
unsubscribe_method,
params,
sub_id,
sender,
chain_id,
} => {
let (chain_data, client) = self.for_chain_id(chain_id);
let id = chain_data.next_id();
let id = self.next_id();
let chain_id = self.chain_id;
// For subscriptions we need to make a plain RPC request to the subscription method.
// The server will return as a result the subscription ID.
let params = match &params {
Some(params) => params.get(),
None => "null",
};
let request = format!(
r#"{{"jsonrpc":"2.0","id":"{}", "method":"{}","params":{}}}"#,
id, method, params
);
tracing::trace!(target: LOG_TARGET, "Tracking subscription request id={id} chain={chain_id:?}");
let subscription_id_state = PendingSubscription {
sub_id_sender: sub_id,
subscription_state: ActiveSubscription {
sender,
unsubscribe_method,
},
let pending_subscription = PendingSubscription {
response_sender: sender,
unsubscribe_method,
};
chain_data
.id_to_subscription
.insert(id, subscription_id_state);
self.pending_subscriptions.insert(id, pending_subscription);
let result = client.json_rpc_request(request, chain_id);
let result = self.client.json_rpc_request(request, chain_id);
if let Err(err) = result {
tracing::warn!(
target: LOG_TARGET,
"Cannot send RPC request to lightclient {:?}",
err.to_string()
);
let subscription_id_state = chain_data
.id_to_subscription
let subscription_id_state = self
.pending_subscriptions
.remove(&id)
.expect("Channels are inserted above; qed");
// Send the error back to frontend.
if subscription_id_state
.sub_id_sender
.send(Err(LightClientRpcError::Request(err.to_string())))
.response_sender
.send(Err(LightClientRpcError::SmoldotError(err.to_string())))
.is_err()
{
tracing::warn!(
@@ -253,20 +357,75 @@ impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
}
/// Parse the response received from the light client and sent it to the appropriate user.
fn handle_rpc_response(&mut self, chain_id: smoldot_light::ChainId, response: String) {
tracing::trace!(target: LOG_TARGET, "Received from smoldot response={response} chain={chain_id:?}");
let (chain_data, _client) = self.for_chain_id(chain_id);
fn handle_rpc_response(&mut self, response: String) {
let chain_id = self.chain_id;
tracing::trace!(target: LOG_TARGET, "Received from smoldot response='{response}' chain={chain_id:?}");
match RpcResponse::from_str(&response) {
Ok(RpcResponse::Error { id, error }) => {
Ok(RpcResponse::Method { id, result }) => {
let Ok(id) = id.parse::<usize>() else {
tracing::warn!(target: LOG_TARGET, "Cannot send response. Id={id} chain={chain_id:?} is not a valid number");
return;
};
// Send the response back.
if let Some(sender) = self.requests.remove(&id) {
if sender.send(Ok(result)).is_err() {
tracing::warn!(
target: LOG_TARGET,
"Cannot send method response to id={id} chain={chain_id:?}",
);
}
} else if let Some(pending_subscription) = self.pending_subscriptions.remove(&id) {
let Ok(sub_id) = serde_json::from_str::<SubscriptionId>(result.get()) else {
tracing::warn!(
target: LOG_TARGET,
"Subscription id='{result}' chain={chain_id:?} is not a valid string",
);
return;
};
tracing::trace!(target: LOG_TARGET, "Received subscription id={sub_id} chain={chain_id:?}");
let (sub_tx, sub_rx) = mpsc::unbounded_channel();
// Send the method response and a channel to receive notifications back.
if pending_subscription
.response_sender
.send(Ok((sub_id.clone(), sub_rx)))
.is_err()
{
tracing::warn!(
target: LOG_TARGET,
"Cannot send subscription ID response to id={id} chain={chain_id:?}",
);
return;
}
// Store the other end of the notif channel to send future subscription notifications to.
self.subscriptions.insert(
sub_id,
ActiveSubscription {
notification_sender: sub_tx,
unsubscribe_method: pending_subscription.unsubscribe_method,
},
);
} else {
tracing::warn!(
target: LOG_TARGET,
"Response id={id} chain={chain_id:?} is not tracked",
);
}
}
Ok(RpcResponse::MethodError { id, error }) => {
let Ok(id) = id.parse::<usize>() else {
tracing::warn!(target: LOG_TARGET, "Cannot send error. Id={id} chain={chain_id:?} is not a valid number");
return;
};
if let Some(sender) = chain_data.requests.remove(&id) {
if let Some(sender) = self.requests.remove(&id) {
if sender
.send(Err(LightClientRpcError::Request(error.to_string())))
.send(Err(LightClientRpcError::JsonRpcError(JsonRpcError(error))))
.is_err()
{
tracing::warn!(
@@ -274,12 +433,10 @@ impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
"Cannot send method response to id={id} chain={chain_id:?}",
);
}
} else if let Some(subscription_id_state) =
chain_data.id_to_subscription.remove(&id)
{
} else if let Some(subscription_id_state) = self.pending_subscriptions.remove(&id) {
if subscription_id_state
.sub_id_sender
.send(Err(LightClientRpcError::Request(error.to_string())))
.response_sender
.send(Err(LightClientRpcError::JsonRpcError(JsonRpcError(error))))
.is_err()
{
tracing::warn!(
@@ -289,93 +446,44 @@ impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
}
}
}
Ok(RpcResponse::Method { id, result }) => {
let Ok(id) = id.parse::<usize>() else {
tracing::warn!(target: LOG_TARGET, "Cannot send response. Id={id} chain={chain_id:?} is not a valid number");
return;
};
// Send the response back.
if let Some(sender) = chain_data.requests.remove(&id) {
if sender.send(Ok(result)).is_err() {
tracing::warn!(
target: LOG_TARGET,
"Cannot send method response to id={id} chain={chain_id:?}",
);
}
} else if let Some(pending_subscription) = chain_data.id_to_subscription.remove(&id)
{
let Ok(sub_id) = result
.get()
.trim_start_matches('"')
.trim_end_matches('"')
.parse::<usize>()
else {
tracing::warn!(
target: LOG_TARGET,
"Subscription id={result} chain={chain_id:?} is not a valid number",
);
return;
};
tracing::trace!(target: LOG_TARGET, "Received subscription id={sub_id} chain={chain_id:?}");
let (sub_id_sender, active_subscription) = pending_subscription.into_parts();
if sub_id_sender.send(Ok(result)).is_err() {
tracing::warn!(
target: LOG_TARGET,
"Cannot send method response to id={id} chain={chain_id:?}",
);
return;
}
// Track this subscription ID if send is successful.
chain_data.subscriptions.insert(sub_id, active_subscription);
} else {
Ok(RpcResponse::Notification {
method,
subscription_id,
result,
}) => {
let Some(active_subscription) = self.subscriptions.get_mut(&subscription_id) else {
tracing::warn!(
target: LOG_TARGET,
"Response id={id} chain={chain_id:?} is not tracked",
"Subscription response id={subscription_id} chain={chain_id:?} method={method} is not tracked",
);
return;
};
if active_subscription
.notification_sender
.send(Ok(result))
.is_err()
{
self.unsubscribe(&subscription_id, chain_id);
}
}
Ok(RpcResponse::Subscription { method, id, result }) => {
let Ok(id) = id.parse::<usize>() else {
tracing::warn!(target: LOG_TARGET, "Cannot send subscription. Id={id} chain={chain_id:?} is not a valid number");
return;
};
let Some(subscription_state) = chain_data.subscriptions.get_mut(&id) else {
Ok(RpcResponse::NotificationError {
method,
subscription_id,
error,
}) => {
let Some(active_subscription) = self.subscriptions.get_mut(&subscription_id) else {
tracing::warn!(
target: LOG_TARGET,
"Subscription response id={id} chain={chain_id:?} method={method} is not tracked",
"Subscription error id={subscription_id} chain={chain_id:?} method={method} is not tracked",
);
return;
};
if subscription_state.sender.send(result).is_ok() {
// Nothing else to do, user is informed about the notification.
return;
}
// User dropped the receiver, unsubscribe from the method and remove internal tracking.
let Some(subscription_state) = chain_data.subscriptions.remove(&id) else {
// State is checked to be some above, so this should never happen.
return;
};
// Make a call to unsubscribe from this method.
let unsub_id = chain_data.next_id();
let request = format!(
r#"{{"jsonrpc":"2.0","id":"{}", "method":"{}","params":["{}"]}}"#,
unsub_id, subscription_state.unsubscribe_method, id
);
if let Err(err) = self.client.json_rpc_request(request, chain_id) {
tracing::warn!(
target: LOG_TARGET,
"Failed to unsubscribe id={id:?} chain={chain_id:?} method={:?} err={err:?}", subscription_state.unsubscribe_method
);
} else {
tracing::debug!(target: LOG_TARGET,"Unsubscribe id={id:?} chain={chain_id:?} method={:?}", subscription_state.unsubscribe_method);
if active_subscription
.notification_sender
.send(Err(JsonRpcError(error)))
.is_err()
{
self.unsubscribe(&subscription_id, chain_id);
}
}
Err(err) => {
@@ -384,169 +492,28 @@ impl<TPlatform: PlatformRef, TChain> BackgroundTask<TPlatform, TChain> {
}
}
/// Perform the main background task:
/// - receiving requests from subxt RPC method / subscriptions
/// - provides the results from the light client back to users.
pub async fn start_task(
&mut self,
from_subxt: mpsc::UnboundedReceiver<FromSubxt>,
from_node: Vec<AddedChain>,
) {
let from_subxt_event = tokio_stream::wrappers::UnboundedReceiverStream::new(from_subxt);
// Unsubscribe from a subscription.
fn unsubscribe(&mut self, subscription_id: &str, chain_id: smoldot_light::ChainId) {
let Some(active_subscription) = self.subscriptions.remove(subscription_id) else {
// Subscription doesn't exist so nothing more to do.
return;
};
let from_node = from_node.into_iter().map(|rpc| {
Box::pin(futures::stream::unfold(rpc, |mut rpc| async move {
let response = rpc.rpc_responses.next().await;
Some(((response, rpc.chain_id), rpc))
}))
});
let stream_combinator = futures::stream::select_all(from_node);
// Build a call to unsubscribe from this method.
let unsub_id = self.next_id();
let request = format!(
r#"{{"jsonrpc":"2.0","id":"{}", "method":"{}","params":["{}"]}}"#,
unsub_id, active_subscription.unsubscribe_method, subscription_id
);
tokio::pin!(from_subxt_event, stream_combinator);
let mut from_subxt_event_fut = from_subxt_event.next();
let mut from_node_event_fut = stream_combinator.next();
loop {
match future::select(from_subxt_event_fut, from_node_event_fut).await {
// Message received from subxt.
Either::Left((subxt_message, previous_fut)) => {
let Some(message) = subxt_message else {
tracing::trace!(target: LOG_TARGET, "Subxt channel closed");
break;
};
tracing::trace!(
target: LOG_TARGET,
"Received register message {:?}",
message
);
self.handle_requests(message).await;
from_subxt_event_fut = from_subxt_event.next();
from_node_event_fut = previous_fut;
}
// Message received from rpc handler: lightclient response.
Either::Right((node_message, previous_fut)) => {
let Some((node_message, chain)) = node_message else {
tracing::trace!(target: LOG_TARGET, "Smoldot closed all RPC channels");
break;
};
// Smoldot returns `None` if the chain has been removed (which subxt does not remove).
let Some(response) = node_message else {
tracing::trace!(target: LOG_TARGET, "Smoldot RPC responses channel closed");
break;
};
tracing::trace!(
target: LOG_TARGET,
"Received smoldot RPC chain {:?} result {:?}",
chain, response
);
self.handle_rpc_response(chain, response);
// Advance backend, save frontend.
from_subxt_event_fut = previous_fut;
from_node_event_fut = stream_combinator.next();
}
}
// Submit it.
if let Err(err) = self.client.json_rpc_request(request, chain_id) {
tracing::warn!(
target: LOG_TARGET,
"Failed to unsubscribe id={subscription_id} chain={chain_id:?} method={:?} err={err:?}", active_subscription.unsubscribe_method
);
} else {
tracing::debug!(target: LOG_TARGET,"Unsubscribe id={subscription_id} chain={chain_id:?} method={:?}", active_subscription.unsubscribe_method);
}
tracing::trace!(target: LOG_TARGET, "Task closed");
}
}
/// The RPC response from the light-client.
/// This can either be a response of a method, or a notification from a subscription.
#[derive(Debug, Clone)]
enum RpcResponse {
Method {
/// Response ID.
id: String,
/// The result of the method call.
result: Box<RawValue>,
},
Subscription {
/// RPC method that generated the notification.
method: String,
/// Subscription ID.
id: String,
/// Result.
result: Box<RawValue>,
},
Error {
/// Response ID.
id: String,
/// Error.
error: Box<RawValue>,
},
}
impl std::str::FromStr for RpcResponse {
type Err = serde_json::Error;
fn from_str(response: &str) -> Result<Self, Self::Err> {
// Helper structures to deserialize from raw RPC strings.
#[derive(Deserialize, Debug)]
struct Response {
/// JSON-RPC version.
#[allow(unused)]
jsonrpc: String,
/// Result.
result: Box<RawValue>,
/// Request ID
id: String,
}
#[derive(Deserialize)]
struct NotificationParams {
/// The ID of the subscription.
subscription: String,
/// Result.
result: Box<RawValue>,
}
#[derive(Deserialize)]
struct ResponseNotification {
/// JSON-RPC version.
#[allow(unused)]
jsonrpc: String,
/// RPC method that generated the notification.
method: String,
/// Result.
params: NotificationParams,
}
#[derive(Deserialize)]
struct ErrorResponse {
/// JSON-RPC version.
#[allow(unused)]
jsonrpc: String,
/// Request ID.
id: String,
/// Error.
error: Box<RawValue>,
}
// Check if the response can be mapped as an RPC method response.
let result: Result<Response, _> = serde_json::from_str(response);
if let Ok(response) = result {
return Ok(RpcResponse::Method {
id: response.id,
result: response.result,
});
}
let result: Result<ResponseNotification, _> = serde_json::from_str(response);
if let Ok(notification) = result {
return Ok(RpcResponse::Subscription {
id: notification.params.subscription,
method: notification.method,
result: notification.params.result,
});
}
let error: ErrorResponse = serde_json::from_str(response)?;
Ok(RpcResponse::Error {
id: error.id,
error: error.error,
})
}
}
lightclient/src/chain_config.rs
+71
View File
@@ -0,0 +1,71 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use serde_json::Value;
use std::borrow::Cow;
/// Something went wrong building chain config.
#[non_exhaustive]
#[derive(thiserror::Error, Debug)]
pub enum ChainConfigError {
/// The provided chain spec is the wrong shape.
#[error("Invalid chain spec format")]
InvalidSpecFormat,
}
/// Configuration to connect to a chain.
pub struct ChainConfig<'a> {
// The chain spec to use.
chain_spec: Cow<'a, str>,
}
impl<'a> From<&'a str> for ChainConfig<'a> {
fn from(chain_spec: &'a str) -> Self {
ChainConfig::chain_spec(chain_spec)
}
}
impl<'a> From<String> for ChainConfig<'a> {
fn from(chain_spec: String) -> Self {
ChainConfig::chain_spec(chain_spec)
}
}
impl<'a> ChainConfig<'a> {
/// Construct a chain config from a chain spec.
pub fn chain_spec(chain_spec: impl Into<Cow<'a, str>>) -> Self {
ChainConfig {
chain_spec: chain_spec.into(),
}
}
/// Set the bootnodes to the given ones.
pub fn set_bootnodes<S: AsRef<str>>(
self,
bootnodes: impl IntoIterator<Item = S>,
) -> Result<Self, ChainConfigError> {
let mut chain_spec_json: Value = serde_json::from_str(&self.chain_spec)
.map_err(|_e| ChainConfigError::InvalidSpecFormat)?;
if let Value::Object(map) = &mut chain_spec_json {
let bootnodes = bootnodes
.into_iter()
.map(|s| Value::String(s.as_ref().to_owned()))
.collect();
map.insert("bootNodes".to_string(), Value::Array(bootnodes));
} else {
return Err(ChainConfigError::InvalidSpecFormat);
}
Ok(ChainConfig {
chain_spec: Cow::Owned(chain_spec_json.to_string()),
})
}
// Used internally to fetch the chain spec back out.
pub(crate) fn as_chain_spec(&self) -> &str {
&self.chain_spec
}
}
lightclient/src/client.rs
-212
View File
@@ -1,212 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use std::iter;
use super::{
background::{BackgroundTask, FromSubxt, MethodResponse},
LightClientRpcError,
};
use serde_json::value::RawValue;
use tokio::sync::{mpsc, mpsc::error::SendError, oneshot};
use super::platform::build_platform;
pub const LOG_TARGET: &str = "subxt-light-client";
/// A raw light-client RPC implementation that can connect to multiple chains.
#[derive(Clone)]
pub struct RawLightClientRpc {
/// Communicate with the backend task that multiplexes the responses
/// back to the frontend.
to_backend: mpsc::UnboundedSender<FromSubxt>,
}
impl RawLightClientRpc {
/// Construct a [`LightClientRpc`] that can communicated with the provided chain.
///
/// # Note
///
/// This uses the same underlying instance created by [`LightClientRpc::new_from_client`].
pub fn for_chain(&self, chain_id: smoldot_light::ChainId) -> LightClientRpc {
LightClientRpc {
to_backend: self.to_backend.clone(),
chain_id,
}
}
}
/// The light-client RPC implementation that is used to connect with the chain.
#[derive(Clone)]
pub struct LightClientRpc {
/// Communicate with the backend task that multiplexes the responses
/// back to the frontend.
to_backend: mpsc::UnboundedSender<FromSubxt>,
/// The chain ID to target for requests.
chain_id: smoldot_light::ChainId,
}
impl LightClientRpc {
/// Constructs a new [`LightClientRpc`], providing the chain specification.
///
/// The chain specification can be downloaded from a trusted network via
/// the `sync_state_genSyncSpec` RPC method. This parameter expects the
/// chain spec in text format (ie not in hex-encoded scale-encoded as RPC methods
/// will provide).
///
/// ## Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ### Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ### Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
pub fn new(
config: smoldot_light::AddChainConfig<
'_,
(),
impl IntoIterator<Item = smoldot_light::ChainId>,
>,
) -> Result<LightClientRpc, LightClientRpcError> {
tracing::trace!(target: LOG_TARGET, "Create light client");
let mut client = smoldot_light::Client::new(build_platform());
let config = smoldot_light::AddChainConfig {
specification: config.specification,
json_rpc: config.json_rpc,
database_content: config.database_content,
potential_relay_chains: config.potential_relay_chains.into_iter(),
user_data: config.user_data,
};
let smoldot_light::AddChainSuccess {
chain_id,
json_rpc_responses,
} = client
.add_chain(config)
.map_err(|err| LightClientRpcError::AddChainError(err.to_string()))?;
let rpc_responses = json_rpc_responses.expect("Light client RPC configured; qed");
let raw_client = Self::new_from_client(
client,
iter::once(AddedChain {
chain_id,
rpc_responses,
}),
);
Ok(raw_client.for_chain(chain_id))
}
/// Constructs a new [`RawLightClientRpc`] from the raw smoldot client.
///
/// Receives a list of RPC objects as a result of calling `smoldot_light::Client::add_chain`.
/// This [`RawLightClientRpc`] can target different chains using [`RawLightClientRpc::for_chain`] method.
///
/// ## Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ### Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ### Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
pub fn new_from_client<TPlat>(
client: smoldot_light::Client<TPlat>,
chains: impl IntoIterator<Item = AddedChain>,
) -> RawLightClientRpc
where
TPlat: smoldot_light::platform::PlatformRef + Clone,
{
let (to_backend, backend) = mpsc::unbounded_channel();
let chains = chains.into_iter().collect();
let future = async move {
let mut task = BackgroundTask::new(client);
task.start_task(backend, chains).await;
};
#[cfg(feature = "native")]
tokio::spawn(future);
#[cfg(feature = "web")]
wasm_bindgen_futures::spawn_local(future);
RawLightClientRpc { to_backend }
}
/// Returns the chain ID of the current light-client.
pub fn chain_id(&self) -> smoldot_light::ChainId {
self.chain_id
}
/// Submits an RPC method request to the light-client.
///
/// This method sends a request to the light-client to execute an RPC method with the provided parameters.
/// The parameters are parsed into a valid JSON object in the background.
pub fn method_request(
&self,
method: String,
params: String,
) -> Result<oneshot::Receiver<MethodResponse>, SendError<FromSubxt>> {
let (sender, receiver) = oneshot::channel();
self.to_backend.send(FromSubxt::Request {
method,
params,
sender,
chain_id: self.chain_id,
})?;
Ok(receiver)
}
/// Makes an RPC subscription call to the light-client.
///
/// This method sends a request to the light-client to establish an RPC subscription with the provided parameters.
/// The parameters are parsed into a valid JSON object in the background.
#[allow(clippy::type_complexity)]
pub fn subscription_request(
&self,
method: String,
params: String,
unsubscribe_method: String,
) -> Result<
(
oneshot::Receiver<MethodResponse>,
mpsc::UnboundedReceiver<Box<RawValue>>,
),
SendError<FromSubxt>,
> {
let (sub_id, sub_id_rx) = oneshot::channel();
let (sender, receiver) = mpsc::unbounded_channel();
self.to_backend.send(FromSubxt::Subscription {
method,
unsubscribe_method,
params,
sub_id,
sender,
chain_id: self.chain_id,
})?;
Ok((sub_id_rx, receiver))
}
}
/// The added chain of the light-client.
pub struct AddedChain {
/// The id of the chain.
pub chain_id: smoldot_light::ChainId,
/// Producer of RPC responses for the chain.
pub rpc_responses: smoldot_light::JsonRpcResponses,
}
lightclient/src/lib.rs
+239 -32
View File
@@ -2,52 +2,259 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
//! Low level light client implementation for RPC method and
//! subscriptions requests.
//!
//! The client implementation supports both native and wasm
//! environments.
//!
//! This leverages the smoldot crate to connect to the chain.
//! A wrapper around [`smoldot_light`] which provides an light client capable of connecting
//! to Substrate based chains.
#![deny(missing_docs)]
#![cfg_attr(docsrs, feature(doc_cfg))]
#[cfg(any(
all(feature = "web", feature = "native"),
not(any(feature = "web", feature = "native"))
))]
compile_error!("subxt: exactly one of the 'web' and 'native' features should be used.");
compile_error!("subxt-lightclient: exactly one of the 'web' and 'native' features should be used.");
mod background;
mod client;
mod platform;
mod shared_client;
// mod receiver;
mod background;
mod chain_config;
mod rpc;
// Used to enable the js feature for wasm.
#[cfg(feature = "web")]
#[allow(unused_imports)]
pub use getrandom as _;
use background::{BackgroundTask, BackgroundTaskHandle};
use futures::Stream;
use platform::DefaultPlatform;
use serde_json::value::RawValue;
use shared_client::SharedClient;
use std::future::Future;
use tokio::sync::mpsc;
pub use client::{AddedChain, LightClientRpc, RawLightClientRpc};
pub use chain_config::{ChainConfig, ChainConfigError};
/// Re-exports of the smoldot related objects.
pub mod smoldot {
pub use smoldot_light::{
platform::PlatformRef, AddChainConfig, AddChainConfigJsonRpc, ChainId, Client,
JsonRpcResponses,
};
#[cfg(feature = "native")]
#[cfg_attr(docsrs, doc(cfg(feature = "native")))]
pub use smoldot_light::platform::default::DefaultPlatform;
}
/// Light client error.
/// Things that can go wrong when constructing the [`LightClient`].
#[derive(Debug, thiserror::Error)]
pub enum LightClientRpcError {
pub enum LightClientError {
/// Error encountered while adding the chain to the light-client.
#[error("Failed to add the chain to the light client: {0}.")]
AddChainError(String),
/// Error originated while trying to submit a RPC request.
#[error("RPC request cannot be sent: {0}.")]
Request(String),
}
/// Things that can go wrong calling methods of [`LightClientRpc`].
#[derive(Debug, thiserror::Error)]
pub enum LightClientRpcError {
/// Error response from the JSON-RPC server.
#[error("{0}")]
JsonRpcError(JsonRpcError),
/// Smoldot could not handle the RPC call.
#[error("Smoldot could not handle the RPC call: {0}.")]
SmoldotError(String),
/// Background task dropped.
#[error("The background task was dropped.")]
BackgroundTaskDropped,
}
/// An error response from the JSON-RPC server (ie smoldot) in response to
/// a method call or as a subscription notification.
#[derive(Debug, thiserror::Error)]
#[error("RPC Error: {0}.")]
pub struct JsonRpcError(Box<RawValue>);
/// This represents a single light client connection to the network. Instantiate
/// it with [`LightClient::relay_chain()`] to communicate with a relay chain, and
/// then call [`LightClient::parachain()`] to establish connections to parachains.
#[derive(Clone)]
pub struct LightClient {
client: SharedClient<DefaultPlatform>,
relay_chain_id: smoldot_light::ChainId,
}
impl LightClient {
/// Given a chain spec, establish a connection to a relay chain. Any subsequent calls to
/// [`LightClient::parachain()`] will set this as the relay chain.
///
/// # Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ## Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ## Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
pub fn relay_chain<'a>(
chain_config: impl Into<ChainConfig<'a>>,
) -> Result<(Self, LightClientRpc), LightClientError> {
let mut client = smoldot_light::Client::new(platform::build_platform());
let chain_config = chain_config.into();
let chain_spec = chain_config.as_chain_spec();
let config = smoldot_light::AddChainConfig {
specification: chain_spec,
json_rpc: smoldot_light::AddChainConfigJsonRpc::Enabled {
max_pending_requests: u32::MAX.try_into().unwrap(),
max_subscriptions: u32::MAX,
},
database_content: "",
potential_relay_chains: std::iter::empty(),
user_data: (),
};
let added_chain = client
.add_chain(config)
.map_err(|err| LightClientError::AddChainError(err.to_string()))?;
let relay_chain_id = added_chain.chain_id;
let rpc_responses = added_chain
.json_rpc_responses
.expect("Light client RPC configured; qed");
let shared_client: SharedClient<_> = client.into();
let light_client_rpc =
LightClientRpc::new_raw(shared_client.clone(), relay_chain_id, rpc_responses);
let light_client = Self {
client: shared_client,
relay_chain_id,
};
Ok((light_client, light_client_rpc))
}
/// Given a chain spec, establish a connection to a parachain.
///
/// # Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ## Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ## Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
pub fn parachain<'a>(
&self,
chain_config: impl Into<ChainConfig<'a>>,
) -> Result<LightClientRpc, LightClientError> {
let chain_config = chain_config.into();
let chain_spec = chain_config.as_chain_spec();
let config = smoldot_light::AddChainConfig {
specification: chain_spec,
json_rpc: smoldot_light::AddChainConfigJsonRpc::Enabled {
max_pending_requests: u32::MAX.try_into().unwrap(),
max_subscriptions: u32::MAX,
},
database_content: "",
potential_relay_chains: std::iter::once(self.relay_chain_id),
user_data: (),
};
let added_chain = self
.client
.add_chain(config)
.map_err(|err| LightClientError::AddChainError(err.to_string()))?;
let chain_id = added_chain.chain_id;
let rpc_responses = added_chain
.json_rpc_responses
.expect("Light client RPC configured; qed");
Ok(LightClientRpc::new_raw(
self.client.clone(),
chain_id,
rpc_responses,
))
}
}
/// This represents a single RPC connection to a specific chain, and is constructed by calling
/// one of the methods on [`LightClient`]. Using this, you can make RPC requests to the chain.
#[derive(Clone, Debug)]
pub struct LightClientRpc {
handle: BackgroundTaskHandle,
}
impl LightClientRpc {
// Dev note: this would provide a "low leveL" interface if one is needed.
// Do we actually need to provide this, or can we entirely hide Smoldot?
pub(crate) fn new_raw<TPlat, TChain>(
client: impl Into<SharedClient<TPlat, TChain>>,
chain_id: smoldot_light::ChainId,
rpc_responses: smoldot_light::JsonRpcResponses,
) -> Self
where
TPlat: smoldot_light::platform::PlatformRef + Send + 'static,
TChain: Send + 'static,
{
let (background_task, background_handle) =
BackgroundTask::new(client.into(), chain_id, rpc_responses);
// For now we spawn the background task internally, but later we can expose
// methods to give this back to the user so that they can exert backpressure.
spawn(async move { background_task.run().await });
LightClientRpc {
handle: background_handle,
}
}
/// Make an RPC request to a chain, getting back a result.
pub async fn request(
&self,
method: String,
params: Option<Box<RawValue>>,
) -> Result<Box<RawValue>, LightClientRpcError> {
self.handle.request(method, params).await
}
/// Subscribe to some RPC method, getting back a stream of notifications.
pub async fn subscribe(
&self,
method: String,
params: Option<Box<RawValue>>,
unsub: String,
) -> Result<LightClientRpcSubscription, LightClientRpcError> {
let (id, notifications) = self.handle.subscribe(method, params, unsub).await?;
Ok(LightClientRpcSubscription { id, notifications })
}
}
/// A stream of notifications handed back when [`LightClientRpc::subscribe`] is called.
pub struct LightClientRpcSubscription {
notifications: mpsc::UnboundedReceiver<Result<Box<RawValue>, JsonRpcError>>,
id: String,
}
impl LightClientRpcSubscription {
/// Return the subscription ID
pub fn id(&self) -> &str {
&self.id
}
}
impl Stream for LightClientRpcSubscription {
type Item = Result<Box<RawValue>, JsonRpcError>;
fn poll_next(
mut self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<Self::Item>> {
self.notifications.poll_recv(cx)
}
}
/// A quick helper to spawn a task that works for WASM.
fn spawn<F: Future + Send + 'static>(future: F) {
#[cfg(feature = "native")]
tokio::spawn(async move {
future.await;
});
#[cfg(feature = "web")]
wasm_bindgen_futures::spawn_local(async move {
future.await;
});
}
lightclient/src/platform/mod.rs
+5 -5
View File
@@ -11,16 +11,16 @@ mod wasm_platform;
#[cfg(feature = "web")]
mod wasm_socket;
pub use helpers::build_platform;
pub use helpers::{build_platform, DefaultPlatform};
#[cfg(feature = "native")]
mod helpers {
use smoldot_light::platform::default::DefaultPlatform as Platform;
use std::sync::Arc;
pub type PlatformType = Arc<Platform>;
pub type DefaultPlatform = Arc<Platform>;
pub fn build_platform() -> PlatformType {
pub fn build_platform() -> DefaultPlatform {
Platform::new(
"subxt-light-client".into(),
env!("CARGO_PKG_VERSION").into(),
@@ -32,9 +32,9 @@ mod helpers {
mod helpers {
use super::wasm_platform::SubxtPlatform as Platform;
pub type PlatformType = Platform;
pub type DefaultPlatform = Platform;
pub fn build_platform() -> PlatformType {
pub fn build_platform() -> DefaultPlatform {
Platform::new()
}
}
lightclient/src/rpc.rs
+132
View File
@@ -0,0 +1,132 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use serde::Deserialize;
use serde_json::value::RawValue;
/// The RPC response from the light-client.
/// This can either be a response of a method, or a notification from a subscription.
#[derive(Debug, Clone)]
pub enum RpcResponse {
Method {
/// Response ID.
id: String,
/// The result of the method call.
result: Box<RawValue>,
},
MethodError {
/// Response ID.
id: String,
/// Error.
error: Box<RawValue>,
},
Notification {
/// RPC method that generated the notification.
method: String,
/// Subscription ID.
subscription_id: String,
/// Result.
result: Box<RawValue>,
},
NotificationError {
/// RPC method that generated the notification.
method: String,
/// Subscription ID.
subscription_id: String,
/// Result.
error: Box<RawValue>,
},
}
impl std::str::FromStr for RpcResponse {
type Err = ();
fn from_str(response: &str) -> Result<Self, Self::Err> {
// Valid response
#[derive(Deserialize, Debug)]
struct Response {
#[allow(unused)]
jsonrpc: String,
id: String,
result: Box<RawValue>,
}
// Error response
#[derive(Deserialize)]
struct ResponseError {
#[allow(unused)]
jsonrpc: String,
id: String,
error: Box<RawValue>,
}
// Valid notification (subscription) response
#[derive(Deserialize)]
struct Notification {
#[allow(unused)]
jsonrpc: String,
method: String,
params: NotificationResultParams,
}
#[derive(Deserialize)]
struct NotificationResultParams {
subscription: String,
result: Box<RawValue>,
}
// Error notification (subscription) response
#[derive(Deserialize)]
struct NotificationError {
#[allow(unused)]
jsonrpc: String,
method: String,
params: NotificationErrorParams,
}
#[derive(Deserialize)]
struct NotificationErrorParams {
/// The ID of the subscription.
subscription: String,
error: Box<RawValue>,
}
// Try deserializing the response payload to one of the above. We can
// do this more efficiently eg how jsonrpsee_types does.
let result: Result<Response, _> = serde_json::from_str(response);
if let Ok(response) = result {
return Ok(RpcResponse::Method {
id: response.id,
result: response.result,
});
}
let result: Result<Notification, _> = serde_json::from_str(response);
if let Ok(response) = result {
return Ok(RpcResponse::Notification {
subscription_id: response.params.subscription,
method: response.method,
result: response.params.result,
});
}
let result: Result<ResponseError, _> = serde_json::from_str(response);
if let Ok(response) = result {
return Ok(RpcResponse::MethodError {
id: response.id,
error: response.error,
});
}
let result: Result<NotificationError, _> = serde_json::from_str(response);
if let Ok(response) = result {
return Ok(RpcResponse::NotificationError {
method: response.method,
subscription_id: response.params.subscription,
error: response.params.error,
});
}
// We couldn't decode into any of the above. We could pick one of the above`
// errors to return, but there's no real point since the string is obviously
// different from any of them.
Err(())
}
}
lightclient/src/shared_client.rs
+47
View File
@@ -0,0 +1,47 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use smoldot_light as sl;
use std::sync::{Arc, Mutex};
/// This wraps [`smoldot_light::Client`] so that it can be cloned and shared.
#[derive(Clone)]
pub struct SharedClient<TPlat: sl::platform::PlatformRef, TChain = ()> {
client: Arc<Mutex<sl::Client<TPlat, TChain>>>,
}
impl<TPlat: sl::platform::PlatformRef, TChain> From<sl::Client<TPlat, TChain>>
for SharedClient<TPlat, TChain>
{
fn from(client: sl::Client<TPlat, TChain>) -> Self {
SharedClient {
client: Arc::new(Mutex::new(client)),
}
}
}
impl<TPlat: sl::platform::PlatformRef, TChain> SharedClient<TPlat, TChain> {
/// Delegates to [`smoldot_light::Client::json_rpc_request()`].
pub(crate) fn json_rpc_request(
&self,
json_rpc_request: impl Into<String>,
chain_id: sl::ChainId,
) -> Result<(), sl::HandleRpcError> {
self.client
.lock()
.expect("mutex should not be poisoned")
.json_rpc_request(json_rpc_request, chain_id)
}
/// Delegates to [`smoldot_light::Client::add_chain()`].
pub(crate) fn add_chain(
&self,
config: sl::AddChainConfig<'_, TChain, impl Iterator<Item = sl::ChainId>>,
) -> Result<sl::AddChainSuccess, sl::AddChainError> {
self.client
.lock()
.expect("mutex should not be poisoned")
.add_chain(config)
}
}
metadata/Cargo.toml
+1 -1
View File
@@ -21,7 +21,7 @@ std = ["scale-info/std", "frame-metadata/std"]
scale-info = { workspace = true, default-features = false }
frame-metadata = { workspace = true, default-features = false, features = ["current", "decode"] }
codec = { package = "parity-scale-codec", workspace = true, default-features = false, features = ["derive"] }
sp-core-hashing = { workspace = true }
sp-crypto-hashing = { workspace = true }
hashbrown = { workspace = true }
derive_more = { workspace = true }
metadata/src/from_into/v14.rs
+2 -6
View File
@@ -316,9 +316,7 @@ fn generate_outer_enums(
) -> Result<v15::OuterEnums<scale_info::form::PortableForm>, TryFromError> {
let find_type = |name: &str| {
metadata.types.types.iter().find_map(|ty| {
let Some(ident) = ty.ty.path.ident() else {
return None;
};
let ident = ty.ty.path.ident()?;
if ident != name {
return None;
@@ -368,9 +366,7 @@ fn generate_outer_error_enum_type(
.pallets
.iter()
.filter_map(|pallet| {
let Some(error) = &pallet.error else {
return None;
};
let error = pallet.error.as_ref()?;
// Note: using the `alloc::format!` macro like in `let path = format!("{}Error", pallet.name);`
// leads to linker errors about extern function `_Unwind_Resume` not being defined.
metadata/src/lib.rs
+29
View File
@@ -475,6 +475,35 @@ pub enum StorageHasher {
Identity,
}
impl StorageHasher {
/// The hash produced by a [`StorageHasher`] can have these two components, in order:
///
/// 1. A fixed size hash. (not present for [`StorageHasher::Identity`]).
/// 2. The SCALE encoded key that was used as an input to the hasher (only present for
/// [`StorageHasher::Twox64Concat`], [`StorageHasher::Blake2_128Concat`] or [`StorageHasher::Identity`]).
///
/// This function returns the number of bytes used to represent the first of these.
pub fn len_excluding_key(&self) -> usize {
match self {
StorageHasher::Blake2_128Concat => 16,
StorageHasher::Twox64Concat => 8,
StorageHasher::Blake2_128 => 16,
StorageHasher::Blake2_256 => 32,
StorageHasher::Twox128 => 16,
StorageHasher::Twox256 => 32,
StorageHasher::Identity => 0,
}
}
/// Returns true if the key used to produce the hash is appended to the hash itself.
pub fn ends_with_key(&self) -> bool {
matches!(
self,
StorageHasher::Blake2_128Concat | StorageHasher::Twox64Concat | StorageHasher::Identity
)
}
}
/// Is the storage entry optional, or does it have a default value.
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
pub enum StorageEntryModifier {
metadata/src/utils/validation.rs
+1 -1
View File
@@ -35,7 +35,7 @@ enum TypeBeingHashed {
/// Hashing function utilized internally.
fn hash(data: &[u8]) -> Hash {
sp_core_hashing::twox_256(data)
sp_crypto_hashing::twox_256(data)
}
/// XOR two hashes together. Only use this when you don't care about the order
signer/Cargo.toml
+10 -7
View File
@@ -15,7 +15,8 @@ description = "Sign extrinsics to be submitted by Subxt"
keywords = ["parity", "subxt", "extrinsic", "signer"]
[features]
default = ["sr25519", "ecdsa", "subxt", "native"]
default = ["sr25519", "ecdsa", "subxt", "std", "native"]
std = ["regex/std", "sp-crypto-hashing/std", "pbkdf2/std", "sha2/std", "hmac/std", "bip39/std", "schnorrkel/std", "secp256k1/std", "sp-core/std"]
# Pick the signer implementation(s) you need by enabling the
# corresponding features. Note: I had more difficulties getting
@@ -34,26 +35,28 @@ web = ["getrandom/js", "subxt?/web"]
native = ["subxt?/native"]
[dependencies]
subxt = { workspace = true, optional = true, default-features = false }
regex = { workspace = true }
subxt = { workspace = true, optional = true }
regex = { workspace = true, features = ["unicode"] }
hex = { workspace = true }
cfg-if = { workspace = true }
codec = { package = "parity-scale-codec", workspace = true, features = ["derive"] }
sp-core-hashing = { workspace = true }
thiserror = { workspace = true }
sp-crypto-hashing = { workspace = true }
derive_more = { workspace = true }
pbkdf2 = { workspace = true }
sha2 = { workspace = true }
hmac = { workspace = true }
zeroize = { workspace = true }
bip39 = { workspace = true }
schnorrkel = { workspace = true, optional = true }
secp256k1 = { workspace = true, features = ["recovery", "global-context"], optional = true }
secp256k1 = { workspace = true, optional = true, features = ["alloc", "recovery"] }
secrecy = { workspace = true }
# We only pull this in to enable the JS flag for schnorrkel to use.
getrandom = { workspace = true, optional = true }
[dev-dependencies]
sp-core = { workspace = true, features = ["std"] }
sp-core = { workspace = true }
sp-keyring = { workspace = true }
[package.metadata.cargo-machete]
signer/src/crypto/derive_junction.rs
+1 -1
View File
@@ -40,7 +40,7 @@ impl DeriveJunction {
let mut cc: [u8; JUNCTION_ID_LEN] = Default::default();
index.using_encoded(|data| {
if data.len() > JUNCTION_ID_LEN {
cc.copy_from_slice(&sp_core_hashing::blake2_256(data));
cc.copy_from_slice(&sp_crypto_hashing::blake2_256(data));
} else {
cc[0..data.len()].copy_from_slice(data);
}
signer/src/crypto/mod.rs
+5
View File
@@ -6,8 +6,13 @@
mod derive_junction;
mod secret_uri;
// No need for the cfg other than to avoid an unused_imports lint warning.
#[cfg(any(feature = "sr25519", feature = "ecdsa"))]
mod seed_from_entropy;
pub use derive_junction::DeriveJunction;
pub use secret_uri::{SecretUri, SecretUriError, DEV_PHRASE};
#[cfg(any(feature = "sr25519", feature = "ecdsa"))]
pub use seed_from_entropy::seed_from_entropy;
signer/src/crypto/secret_uri.rs
+9 -4
View File
@@ -3,6 +3,8 @@
// see LICENSE for license details.
use super::DeriveJunction;
use alloc::vec::Vec;
use derive_more::Display;
use regex::Regex;
use secrecy::SecretString;
@@ -88,7 +90,7 @@ pub struct SecretUri {
pub junctions: Vec<DeriveJunction>,
}
impl std::str::FromStr for SecretUri {
impl core::str::FromStr for SecretUri {
type Err = SecretUriError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
@@ -115,14 +117,17 @@ impl std::str::FromStr for SecretUri {
}
/// This is returned if `FromStr` cannot parse a string into a `SecretUri`.
#[derive(Debug, Copy, Clone, PartialEq, thiserror::Error)]
#[derive(Debug, Copy, Clone, PartialEq, Display)]
pub enum SecretUriError {
/// Parsing the secret URI from a string failed; wrong format.
#[error("Invalid secret phrase format")]
#[display(fmt = "Invalid secret phrase format")]
InvalidFormat,
}
once_static! {
#[cfg(feature = "std")]
impl std::error::Error for SecretUriError {}
once_static_cloned! {
/// Interpret a phrase like:
///
/// ```text
signer/src/crypto/seed_from_entropy.rs
+1
View File
@@ -2,6 +2,7 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use alloc::string::String;
use hmac::Hmac;
use pbkdf2::pbkdf2;
use sha2::Sha512;
signer/src/ecdsa.rs
+32 -20
View File
@@ -6,8 +6,10 @@
use codec::Encode;
use crate::crypto::{seed_from_entropy, DeriveJunction, SecretUri};
use core::str::FromStr;
use derive_more::{Display, From};
use hex::FromHex;
use secp256k1::{ecdsa::RecoverableSignature, Message, SecretKey, SECP256K1};
use secp256k1::{ecdsa::RecoverableSignature, Message, Secp256k1, SecretKey};
use secrecy::ExposeSecret;
const SEED_LENGTH: usize = 32;
@@ -68,7 +70,7 @@ impl Keypair {
let seed = Seed::from_hex(hex_str)?;
Self::from_seed(seed)?
} else {
let phrase = bip39::Mnemonic::parse(phrase.expose_secret().as_str())?;
let phrase = bip39::Mnemonic::from_str(phrase.expose_secret().as_str())?;
let pass_str = password.as_ref().map(|p| p.expose_secret().as_str());
Self::from_phrase(&phrase, pass_str)?
};
@@ -91,8 +93,9 @@ impl Keypair {
/// keypair.sign(b"Hello world!");
/// ```
pub fn from_phrase(mnemonic: &bip39::Mnemonic, password: Option<&str>) -> Result<Self, Error> {
let big_seed = seed_from_entropy(&mnemonic.to_entropy(), password.unwrap_or(""))
.ok_or(Error::InvalidSeed)?;
let (arr, len) = mnemonic.to_entropy_array();
let big_seed =
seed_from_entropy(&arr[0..len], password.unwrap_or("")).ok_or(Error::InvalidSeed)?;
let seed: Seed = big_seed[..SEED_LENGTH]
.try_into()
@@ -109,7 +112,8 @@ impl Keypair {
pub fn from_seed(seed: Seed) -> Result<Self, Error> {
let secret = SecretKey::from_slice(&seed).map_err(|_| Error::InvalidSeed)?;
Ok(Self(secp256k1::Keypair::from_secret_key(
SECP256K1, &secret,
&Secp256k1::signing_only(),
&secret,
)))
}
@@ -140,7 +144,7 @@ impl Keypair {
DeriveJunction::Soft(_) => return Err(Error::SoftJunction),
DeriveJunction::Hard(junction_bytes) => {
acc = ("Secp256k1HDKD", acc, junction_bytes)
.using_encoded(sp_core_hashing::blake2_256)
.using_encoded(sp_crypto_hashing::blake2_256)
}
}
}
@@ -157,13 +161,13 @@ impl Keypair {
/// Sign some message. These bytes can be used directly in a Substrate `MultiSignature::Ecdsa(..)`.
pub fn sign(&self, message: &[u8]) -> Signature {
// From sp_core::ecdsa::sign:
let message_hash = sp_core_hashing::blake2_256(message);
let message_hash = sp_crypto_hashing::blake2_256(message);
// From sp_core::ecdsa::sign_prehashed:
let wrapped = Message::from_digest_slice(&message_hash).expect("Message is 32 bytes; qed");
let recsig: RecoverableSignature =
SECP256K1.sign_ecdsa_recoverable(&wrapped, &self.0.secret_key());
Secp256k1::signing_only().sign_ecdsa_recoverable(&wrapped, &self.0.secret_key());
// From sp_core::ecdsa's `impl From<RecoverableSignature> for Signature`:
let (recid, sig) = recsig.serialize_compact();
let (recid, sig): (_, [u8; 64]) = recsig.serialize_compact();
let mut signature_bytes: [u8; 65] = [0; 65];
signature_bytes[..64].copy_from_slice(&sig);
signature_bytes[64] = (recid.to_i32() & 0xFF) as u8;
@@ -190,33 +194,41 @@ pub fn verify<M: AsRef<[u8]>>(sig: &Signature, message: M, pubkey: &PublicKey) -
let Ok(public) = secp256k1::PublicKey::from_slice(&pubkey.0) else {
return false;
};
let message_hash = sp_core_hashing::blake2_256(message.as_ref());
let message_hash = sp_crypto_hashing::blake2_256(message.as_ref());
let wrapped = Message::from_digest_slice(&message_hash).expect("Message is 32 bytes; qed");
signature.verify(&wrapped, &public).is_ok()
Secp256k1::verification_only()
.verify_ecdsa(&wrapped, &signature, &public)
.is_ok()
}
/// An error handed back if creating a keypair fails.
#[derive(Debug, PartialEq, thiserror::Error)]
#[derive(Debug, PartialEq, Display, From)]
pub enum Error {
/// Invalid seed.
#[error("Invalid seed (was it the wrong length?)")]
#[display(fmt = "Invalid seed (was it the wrong length?)")]
#[from(ignore)]
InvalidSeed,
/// Invalid seed.
#[error("Invalid seed for ECDSA, contained soft junction")]
#[display(fmt = "Invalid seed for ECDSA, contained soft junction")]
#[from(ignore)]
SoftJunction,
/// Invalid phrase.
#[error("Cannot parse phrase: {0}")]
Phrase(#[from] bip39::Error),
#[display(fmt = "Cannot parse phrase: {_0}")]
Phrase(bip39::Error),
/// Invalid hex.
#[error("Cannot parse hex string: {0}")]
Hex(#[from] hex::FromHexError),
#[display(fmt = "Cannot parse hex string: {_0}")]
Hex(hex::FromHexError),
}
#[cfg(feature = "std")]
impl std::error::Error for Error {}
/// Dev accounts, helpful for testing but not to be used in production,
/// since the secret keys are known.
pub mod dev {
use super::*;
use std::str::FromStr;
use core::str::FromStr;
once_static_cloned! {
/// Equivalent to `{DEV_PHRASE}//Alice`.
@@ -287,7 +299,7 @@ mod subxt_compat {
/// We often want this type, and using this method avoids any
/// ambiguous type resolution issues.
pub fn to_account_id(self) -> AccountId32 {
AccountId32(sp_core_hashing::blake2_256(&self.0))
AccountId32(sp_crypto_hashing::blake2_256(&self.0))
}
/// A shortcut to obtain a [`MultiAddress`] from a [`PublicKey`].
/// We often want this type, and using this method avoids any
signer/src/lib.rs
+3 -6
View File
@@ -14,6 +14,9 @@
//! subxt transactions for chains supporting sr25519 signatures.
#![cfg_attr(docsrs, feature(doc_cfg))]
#![cfg_attr(not(feature = "std"), no_std)]
extern crate alloc;
#[macro_use]
mod utils;
@@ -40,9 +43,3 @@ pub use secrecy::{ExposeSecret, SecretString};
// SecretUri's can be parsed from strings and used to generate key pairs.
// DeriveJunctions are the "path" part of these SecretUris.
pub use crypto::{DeriveJunction, SecretUri, SecretUriError, DEV_PHRASE};
#[cfg(any(
all(feature = "web", feature = "native"),
not(any(feature = "web", feature = "native"))
))]
compile_error!("subxt-signer: exactly one of the 'web' and 'native' features should be used.");
signer/src/sr25519.rs
+19 -10
View File
@@ -4,7 +4,11 @@
//! An sr25519 keypair implementation.
use core::str::FromStr;
use crate::crypto::{seed_from_entropy, DeriveJunction, SecretUri};
use derive_more::{Display, From};
use hex::FromHex;
use schnorrkel::{
derive::{ChainCode, Derivation},
@@ -72,7 +76,7 @@ impl Keypair {
let seed = Seed::from_hex(hex_str)?;
Self::from_seed(seed)?
} else {
let phrase = bip39::Mnemonic::parse(phrase.expose_secret().as_str())?;
let phrase = bip39::Mnemonic::from_str(phrase.expose_secret().as_str())?;
let pass_str = password.as_ref().map(|p| p.expose_secret().as_str());
Self::from_phrase(&phrase, pass_str)?
};
@@ -95,8 +99,9 @@ impl Keypair {
/// keypair.sign(b"Hello world!");
/// ```
pub fn from_phrase(mnemonic: &bip39::Mnemonic, password: Option<&str>) -> Result<Self, Error> {
let big_seed = seed_from_entropy(&mnemonic.to_entropy(), password.unwrap_or(""))
.ok_or(Error::InvalidSeed)?;
let (arr, len) = mnemonic.to_entropy_array();
let big_seed =
seed_from_entropy(&arr[0..len], password.unwrap_or("")).ok_or(Error::InvalidSeed)?;
let seed: Seed = big_seed[..SEED_LENGTH]
.try_into()
@@ -187,24 +192,28 @@ pub fn verify<M: AsRef<[u8]>>(sig: &Signature, message: M, pubkey: &PublicKey) -
}
/// An error handed back if creating a keypair fails.
#[derive(Debug, thiserror::Error)]
#[derive(Debug, Display, From)]
pub enum Error {
/// Invalid seed.
#[error("Invalid seed (was it the wrong length?)")]
#[display(fmt = "Invalid seed (was it the wrong length?)")]
#[from(ignore)]
InvalidSeed,
/// Invalid phrase.
#[error("Cannot parse phrase: {0}")]
Phrase(#[from] bip39::Error),
#[display(fmt = "Cannot parse phrase: {_0}")]
Phrase(bip39::Error),
/// Invalid hex.
#[error("Cannot parse hex string: {0}")]
Hex(#[from] hex::FromHexError),
#[display(fmt = "Cannot parse hex string: {_0}")]
Hex(hex::FromHexError),
}
#[cfg(feature = "std")]
impl std::error::Error for Error {}
/// Dev accounts, helpful for testing but not to be used in production,
/// since the secret keys are known.
pub mod dev {
use super::*;
use std::str::FromStr;
use core::str::FromStr;
once_static_cloned! {
/// Equivalent to `{DEV_PHRASE}//Alice`.
signer/src/utils.rs
+11 -16
View File
@@ -7,34 +7,29 @@
/// Use like:
///
/// ```rust,ignore
/// once_static!{
/// once_static_cloned!{
/// /// Some documentation.
/// fn foo() -> Vec<u8> {
/// vec![1,2,3,4]
/// }
/// }
/// ```
macro_rules! once_static {
($($(#[$attr:meta])* $vis:vis fn $name:ident() -> $ty:ty { $expr:expr } )+) => {
$(
$(#[$attr])*
$vis fn $name() -> &'static $ty {
static VAR: std::sync::OnceLock<$ty> = std::sync::OnceLock::new();
VAR.get_or_init(|| { $expr })
}
)+
};
}
/// Like `once_static!` but clones the item out of static storage. Useful if it
///
/// Clones the item out of static storage. Useful if it
/// takes a while to create the item but cloning it is fairly cheap.
macro_rules! once_static_cloned {
($($(#[$attr:meta])* $vis:vis fn $name:ident() -> $ty:ty { $expr:expr } )+) => {
$(
$(#[$attr])*
$vis fn $name() -> $ty {
static VAR: std::sync::OnceLock<$ty> = std::sync::OnceLock::new();
VAR.get_or_init(|| { $expr }).clone()
cfg_if::cfg_if! {
if #[cfg(feature = "std")] {
static VAR: std::sync::OnceLock<$ty> = std::sync::OnceLock::new();
VAR.get_or_init(|| { $expr }).clone()
} else {
{ $expr }
}
}
}
)+
};
signer/wasm-tests/Cargo.toml
+2 -3
View File
@@ -8,14 +8,13 @@ publish = false
wasm-bindgen-test = "0.3.24"
tracing-wasm = "0.2.1"
console_error_panic_hook = "0.1.7"
serde_json = "1"
# This crate is not a part of the workspace, because we want to
# enable the "web" feature here but don't want it enabled as part
# of workspace builds. Also disable the "subxt" feature here because
# we want to ensure it works in isolation of that.
subxt-signer = { path = "..", default-features = false, features = ["web", "sr25519", "ecdsa"] }
subxt-signer = { path = "..", default-features = false, features = ["web", "sr25519", "ecdsa", "std"] }
# this shouldn't be needed, it's in workspace.exclude, but still
# I get the complaint unless I add it...
[workspace]
[workspace]
subxt/Cargo.toml
+26 -27
View File
@@ -25,20 +25,20 @@ default = ["jsonrpsee", "native"]
# Enable this for native (ie non web/wasm builds).
# Exactly 1 of "web" and "native" is expected.
native = [
"jsonrpsee?/async-client",
"jsonrpsee?/client-ws-transport-native-tls",
"subxt-lightclient?/native",
"jsonrpsee?/async-client",
"jsonrpsee?/client-ws-transport-native-tls",
"subxt-lightclient?/native",
"tokio-util"
]
# Enable this for web/wasm builds.
# Exactly 1 of "web" and "native" is expected.
web = [
"jsonrpsee?/async-wasm-client",
"jsonrpsee?/client-web-transport",
"getrandom/js",
"subxt-lightclient?/web",
"subxt-macro/web",
"jsonrpsee?/async-wasm-client",
"jsonrpsee?/client-web-transport",
"getrandom/js",
"subxt-lightclient?/web",
"subxt-macro/web",
"instant/wasm-bindgen"
]
@@ -46,7 +46,9 @@ web = [
unstable-reconnecting-rpc-client = ["dep:reconnecting-jsonrpsee-ws-client"]
# Enable this to use jsonrpsee (allowing for example `OnlineClient::from_url`).
jsonrpsee = ["dep:jsonrpsee"]
jsonrpsee = [
"dep:jsonrpsee",
]
# Enable this to pull in extra Substrate dependencies which make it possible to
# use the `sp_core::crypto::Pair` Signer implementation, as well as adding some
@@ -61,20 +63,20 @@ unstable-metadata = []
# Activate this to expose the Light Client functionality.
# Note that this feature is experimental and things may break or not work as expected.
unstable-light-client = ["subxt-lightclient", "tokio-stream"]
unstable-light-client = ["subxt-lightclient"]
[dependencies]
async-trait = { workspace = true }
codec = { package = "parity-scale-codec", workspace = true, features = ["derive"] }
scale-info = { workspace = true }
scale-value = { workspace = true }
scale-bits = { workspace = true }
scale-decode = { workspace = true }
scale-encode = { workspace = true }
scale-info = { workspace = true, features = ["default"] }
scale-value = { workspace = true, features = ["default"] }
scale-bits = { workspace = true, features = ["default"] }
scale-decode = { workspace = true, features = ["default"] }
scale-encode = { workspace = true, features = ["default"] }
futures = { workspace = true }
hex = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true, features = ["raw_value"] }
serde_json = { workspace = true, features = ["default", "raw_value"] }
thiserror = { workspace = true }
tracing = { workspace = true }
frame-metadata = { workspace = true }
@@ -84,8 +86,8 @@ instant = { workspace = true }
# Provides some deserialization, types like U256/H256 and hashing impls like twox/blake256:
impl-serde = { workspace = true }
primitive-types = { workspace = true }
sp-core-hashing = { workspace = true }
primitive-types = { workspace = true, features = ["codec", "scale-info", "serde"] }
sp-crypto-hashing = { workspace = true }
# For ss58 encoding AccountId32 to serialize them properly:
base58 = { workspace = true }
@@ -100,12 +102,9 @@ sp-runtime = { workspace = true, optional = true }
# Other subxt crates we depend on.
subxt-macro = { workspace = true }
subxt-metadata = { workspace = true }
subxt-metadata = { workspace = true, features = ["std"] }
subxt-lightclient = { workspace = true, optional = true, default-features = false }
# Light client support:
tokio-stream = { workspace = true, optional = true }
# Reconnecting jsonrpc ws client
reconnecting-jsonrpsee-ws-client = { version = "0.3", optional = true }
@@ -136,13 +135,13 @@ wasm-bindgen-futures = { workspace = true }
tracing-subscriber = { workspace = true }
[[example]]
name = "light_client_tx_basic"
path = "examples/light_client_tx_basic.rs"
name = "light_client_basic"
path = "examples/light_client_basic.rs"
required-features = ["unstable-light-client", "jsonrpsee"]
[[example]]
name = "light_client_parachains"
path = "examples/light_client_parachains.rs"
name = "light_client_local_node"
path = "examples/light_client_local_node.rs"
required-features = ["unstable-light-client", "jsonrpsee", "native"]
[[example]]
@@ -155,4 +154,4 @@ features = ["default", "substrate-compat", "unstable-light-client"]
rustdoc-args = ["--cfg", "docsrs"]
[package.metadata.playground]
features = ["default", "substrate-compat", "unstable-light-client"]
features = ["default", "substrate-compat", "unstable-light-client"]
subxt/examples/light_client_basic.rs
+47
View File
@@ -0,0 +1,47 @@
#![allow(missing_docs)]
use futures::StreamExt;
use subxt::{client::OnlineClient, lightclient::LightClient, PolkadotConfig};
// Generate an interface that we can use from the node's metadata.
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}
const POLKADOT_SPEC: &str = include_str!("../../artifacts/demo_chain_specs/polkadot.json");
const ASSET_HUB_SPEC: &str =
include_str!("../../artifacts/demo_chain_specs/polkadot_asset_hub.json");
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// The lightclient logs are informative:
tracing_subscriber::fmt::init();
// Instantiate a light client with the Polkadot relay chain,
// and connect it to Asset Hub, too.
let (lightclient, polkadot_rpc) = LightClient::relay_chain(POLKADOT_SPEC)?;
let asset_hub_rpc = lightclient.parachain(ASSET_HUB_SPEC)?;
// Create Subxt clients from these Smoldot backed RPC clients.
let polkadot_api = OnlineClient::<PolkadotConfig>::from_rpc_client(polkadot_rpc).await?;
let asset_hub_api = OnlineClient::<PolkadotConfig>::from_rpc_client(asset_hub_rpc).await?;
// Use them!
let polkadot_sub = polkadot_api
.blocks()
.subscribe_finalized()
.await?
.map(|block| ("Polkadot", block));
let parachain_sub = asset_hub_api
.blocks()
.subscribe_finalized()
.await?
.map(|block| ("AssetHub", block));
let mut stream_combinator = futures::stream::select(polkadot_sub, parachain_sub);
while let Some((chain, block)) = stream_combinator.next().await {
let block = block?;
println!(" Chain {:?} hash={:?}", chain, block.hash());
}
Ok(())
}
subxt/examples/light_client_tx_basic.rs → subxt/examples/light_client_local_node.rs
+19 -10
View File
@@ -1,5 +1,10 @@
#![allow(missing_docs)]
use subxt::{client::LightClient, PolkadotConfig};
use subxt::utils::fetch_chainspec_from_rpc_node;
use subxt::{
client::OnlineClient,
lightclient::{ChainConfig, LightClient},
PolkadotConfig,
};
use subxt_signer::sr25519::dev;
// Generate an interface that we can use from the node's metadata.
@@ -11,19 +16,23 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
// The smoldot logs are informative:
tracing_subscriber::fmt::init();
// Create a light client by fetching the chain spec of a local running node.
// In this case, because we start one single node, the bootnodes must be overwritten
// for the light client to connect to the local node.
// Use a utility function to obtain a chain spec from a locally running node:
let chain_spec = fetch_chainspec_from_rpc_node("ws://127.0.0.1:9944").await?;
// Configure the bootnodes of this chain spec. In this case, because we start one
// single node, the bootnodes must be overwritten for the light client to connect
// to the local node.
//
// The `12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp` is the P2P address
// from a local polkadot node starting with
// `--node-key 0000000000000000000000000000000000000000000000000000000000000001`
let api = LightClient::<PolkadotConfig>::builder()
.bootnodes([
"/ip4/127.0.0.1/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp",
])
.build_from_url("ws://127.0.0.1:9944")
.await?;
let chain_config = ChainConfig::chain_spec(chain_spec.get()).set_bootnodes([
"/ip4/127.0.0.1/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp",
])?;
// Start the light client up, establishing a connection to the local node.
let (_light_client, chain_rpc) = LightClient::relay_chain(chain_config)?;
let api = OnlineClient::<PolkadotConfig>::from_rpc_client(chain_rpc).await?;
// Build a balance transfer extrinsic.
let dest = dev::bob().public_key().into();
subxt/examples/light_client_parachains.rs
-102
View File
@@ -1,102 +0,0 @@
#![allow(missing_docs)]
use futures::StreamExt;
use std::{iter, num::NonZeroU32};
use subxt::{
client::{LightClient, RawLightClient},
PolkadotConfig,
};
// Generate an interface that we can use from the node's metadata.
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}
const POLKADOT_SPEC: &str = include_str!("../../artifacts/demo_chain_specs/polkadot.json");
const ASSET_HUB_SPEC: &str =
include_str!("../../artifacts/demo_chain_specs/polkadot_asset_hub.json");
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// The smoldot logs are informative:
tracing_subscriber::fmt::init();
// Connecting to a parachain is a multi step process.
// Step 1. Construct a new smoldot client.
let mut client =
subxt_lightclient::smoldot::Client::new(subxt_lightclient::smoldot::DefaultPlatform::new(
"subxt-example-light-client".into(),
"version-0".into(),
));
// Step 2. Connect to the relay chain of the parachain. For this example, the Polkadot relay chain.
let polkadot_connection = client
.add_chain(subxt_lightclient::smoldot::AddChainConfig {
specification: POLKADOT_SPEC,
json_rpc: subxt_lightclient::smoldot::AddChainConfigJsonRpc::Enabled {
max_pending_requests: NonZeroU32::new(128).unwrap(),
max_subscriptions: 1024,
},
potential_relay_chains: iter::empty(),
database_content: "",
user_data: (),
})
.expect("Light client chain added with valid spec; qed");
let polkadot_json_rpc_responses = polkadot_connection
.json_rpc_responses
.expect("Light client configured with json rpc enabled; qed");
let polkadot_chain_id = polkadot_connection.chain_id;
// Step 3. Connect to the parachain. For this example, the Asset hub parachain.
let assethub_connection = client
.add_chain(subxt_lightclient::smoldot::AddChainConfig {
specification: ASSET_HUB_SPEC,
json_rpc: subxt_lightclient::smoldot::AddChainConfigJsonRpc::Enabled {
max_pending_requests: NonZeroU32::new(128).unwrap(),
max_subscriptions: 1024,
},
// The chain specification of the asset hub parachain mentions that the identifier
// of its relay chain is `polkadot`.
potential_relay_chains: [polkadot_chain_id].into_iter(),
database_content: "",
user_data: (),
})
.expect("Light client chain added with valid spec; qed");
let parachain_json_rpc_responses = assethub_connection
.json_rpc_responses
.expect("Light client configured with json rpc enabled; qed");
let parachain_chain_id = assethub_connection.chain_id;
// Step 4. Turn the smoldot client into a raw client.
let raw_light_client = RawLightClient::builder()
.add_chain(polkadot_chain_id, polkadot_json_rpc_responses)
.add_chain(parachain_chain_id, parachain_json_rpc_responses)
.build(client)
.await?;
// Step 5. Obtain a client to target the relay chain and the parachain.
let polkadot_api: LightClient<PolkadotConfig> =
raw_light_client.for_chain(polkadot_chain_id).await?;
let parachain_api: LightClient<PolkadotConfig> =
raw_light_client.for_chain(parachain_chain_id).await?;
// Step 6. Subscribe to the finalized blocks of the chains.
let polkadot_sub = polkadot_api
.blocks()
.subscribe_finalized()
.await?
.map(|block| ("Polkadot", block));
let parachain_sub = parachain_api
.blocks()
.subscribe_finalized()
.await?
.map(|block| ("AssetHub", block));
let mut stream_combinator = futures::stream::select(polkadot_sub, parachain_sub);
while let Some((chain, block)) = stream_combinator.next().await {
let block = block?;
println!(" Chain {:?} hash={:?}", chain, block.hash());
}
Ok(())
}
subxt/examples/rpc_legacy.rs
+5 -2
View File
@@ -40,7 +40,10 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
.await?;
let current_header = rpc.chain_get_header(None).await?.unwrap();
let ext_params = Params::new().mortal(&current_header, 8).build();
let ext_params = Params::new()
.mortal(&current_header, 8)
.nonce(current_nonce)
.build();
let balance_transfer = polkadot::tx()
.balances()
@@ -48,7 +51,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
let ext_hash = api
.tx()
.create_signed_with_nonce(&balance_transfer, &alice, current_nonce, ext_params)?
.create_signed_offline(&balance_transfer, &alice, ext_params)?
.submit()
.await?;
subxt/examples/setup_config_custom.rs
+12 -7
View File
@@ -1,7 +1,9 @@
#![allow(missing_docs)]
use codec::Encode;
use subxt::client::OfflineClientT;
use subxt::config::{Config, ExtrinsicParams, ExtrinsicParamsEncoder, ExtrinsicParamsError};
use subxt::config::{
Config, ExtrinsicParams, ExtrinsicParamsEncoder, ExtrinsicParamsError, RefineParams,
};
use subxt_signer::sr25519::dev;
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")]
@@ -51,24 +53,27 @@ impl CustomExtrinsicParamsBuilder {
}
}
impl<T: Config> RefineParams<T> for CustomExtrinsicParamsBuilder {}
// Describe how to fetch and then encode the params:
impl<T: Config> ExtrinsicParams<T> for CustomExtrinsicParams<T> {
type OtherParams = CustomExtrinsicParamsBuilder;
type Params = CustomExtrinsicParamsBuilder;
// Gather together all of the params we will need to encode:
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
client: Client,
other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(Self {
genesis_hash: client.genesis_hash(),
tip: other_params.tip,
foo: other_params.foo,
tip: params.tip,
foo: params.foo,
})
}
}
impl<T: Config> RefineParams<T> for CustomExtrinsicParams<T> {}
// Encode the relevant params when asked:
impl<T: Config> ExtrinsicParamsEncoder for CustomExtrinsicParams<T> {
fn encode_extra_to(&self, v: &mut Vec<u8>) {
@@ -86,7 +91,7 @@ async fn main() {
let tx_payload = runtime::tx().system().remark(b"Hello".to_vec());
// Build your custom "OtherParams":
// Build your custom "Params":
let tx_config = CustomExtrinsicParamsBuilder::new().tip(1234).enable_foo();
// And provide them when submitting a transaction:
subxt/examples/setup_config_signed_extension.rs
+6 -7
View File
@@ -58,12 +58,11 @@ impl<T: Config> signed_extensions::SignedExtension<T> for CustomSignedExtension
// Gather together any params we need for our signed extension, here none.
impl<T: Config> ExtrinsicParams<T> for CustomSignedExtension {
type OtherParams = ();
type Params = ();
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
_client: Client,
_other_params: Self::OtherParams,
_params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(CustomSignedExtension)
}
@@ -80,13 +79,13 @@ impl ExtrinsicParamsEncoder for CustomSignedExtension {
}
// When composing a tuple of signed extensions, the user parameters we need must
// be able to convert `Into` a tuple of corresponding `OtherParams`. Here, we just
// "hijack" the default param builder, but add the `OtherParams` (`()`) for our
// be able to convert `Into` a tuple of corresponding `Params`. Here, we just
// "hijack" the default param builder, but add the `Params` (`()`) for our
// new signed extension at the end, to make the types line up. IN reality you may wish
// to construct an entirely new interface to provide the relevant `OtherParams`.
// to construct an entirely new interface to provide the relevant `Params`.
pub fn custom(
params: DefaultExtrinsicParamsBuilder<CustomConfig>,
) -> <<CustomConfig as Config>::ExtrinsicParams as ExtrinsicParams<CustomConfig>>::OtherParams {
) -> <<CustomConfig as Config>::ExtrinsicParams as ExtrinsicParams<CustomConfig>>::Params {
let (a, b, c, d, e, f, g) = params.build();
(a, b, c, d, e, f, g, ())
}
subxt/examples/storage_iterating.rs
+4 -3
View File
@@ -16,9 +16,10 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
// a time from the node, but we always iterate over one at a time).
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;
while let Some(Ok((key, value))) = results.next().await {
println!("Key: 0x{}", hex::encode(&key));
println!("Value: {:?}", value);
while let Some(Ok(kv)) = results.next().await {
println!("Keys decoded: {:?}", kv.keys);
println!("Key: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value);
}
Ok(())
subxt/examples/storage_iterating_dynamic.rs
+6 -5
View File
@@ -7,16 +7,17 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
let api = OnlineClient::<PolkadotConfig>::new().await?;
// Build a dynamic storage query to iterate account information.
// With a dynamic query, we can just provide an empty Vec as the keys to iterate over all entries.
let keys = Vec::<()>::new();
// With a dynamic query, we can just provide an empty vector as the keys to iterate over all entries.
let keys: Vec<scale_value::Value> = vec![];
let storage_query = subxt::dynamic::storage("System", "Account", keys);
// Use that query to return an iterator over the results.
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;
while let Some(Ok((key, value))) = results.next().await {
println!("Key: 0x{}", hex::encode(&key));
println!("Value: {:?}", value.to_value()?);
while let Some(Ok(kv)) = results.next().await {
println!("Keys decoded: {:?}", kv.keys);
println!("Key: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value.to_value()?);
}
Ok(())
subxt/examples/storage_iterating_partial.rs
+4 -4
View File
@@ -38,11 +38,11 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Get back an iterator of results.
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;
while let Some(Ok((key, value))) = results.next().await {
println!("Key: 0x{}", hex::encode(&key));
println!("Value: {:?}", value);
while let Some(Ok(kv)) = results.next().await {
println!("Keys decoded: {:?}", kv.keys);
println!("Key: 0x{}", hex::encode(&kv.key_bytes));
println!("Value: {:?}", kv.value);
}
Ok(())
}
subxt/src/backend/legacy/mod.rs
+49 -9
View File
@@ -22,18 +22,56 @@ use std::task::{Context, Poll};
// Expose the RPC methods.
pub use rpc_methods::LegacyRpcMethods;
/// Configure and build an [`LegacyBackend`].
pub struct LegacyBackendBuilder<T> {
storage_page_size: u32,
_marker: std::marker::PhantomData<T>,
}
impl<T: Config> Default for LegacyBackendBuilder<T> {
fn default() -> Self {
Self::new()
}
}
impl<T: Config> LegacyBackendBuilder<T> {
/// Create a new [`LegacyBackendBuilder`].
pub fn new() -> Self {
Self {
storage_page_size: 64,
_marker: std::marker::PhantomData,
}
}
/// Iterating over storage entries using the [`LegacyBackend`] requires
/// fetching entries in batches. This configures the number of entries that
/// we'll try to obtain in each batch (default: 64).
pub fn storage_page_size(mut self, storage_page_size: u32) -> Self {
self.storage_page_size = storage_page_size;
self
}
/// Given an [`RpcClient`] to use to make requests, this returns a [`LegacyBackend`],
/// which implements the [`Backend`] trait.
pub fn build(self, client: impl Into<RpcClient>) -> LegacyBackend<T> {
LegacyBackend {
storage_page_size: self.storage_page_size,
methods: LegacyRpcMethods::new(client.into()),
}
}
}
/// The legacy backend.
#[derive(Debug, Clone)]
pub struct LegacyBackend<T> {
storage_page_size: u32,
methods: LegacyRpcMethods<T>,
}
impl<T: Config> LegacyBackend<T> {
/// Instantiate a new backend which uses the legacy API methods.
pub fn new(client: RpcClient) -> Self {
Self {
methods: LegacyRpcMethods::new(client),
}
/// Configure and construct an [`LegacyBackend`].
pub fn builder() -> LegacyBackendBuilder<T> {
LegacyBackendBuilder::new()
}
}
@@ -74,6 +112,7 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for LegacyBackend<T> {
let keys = StorageFetchDescendantKeysStream {
at,
key,
storage_page_size: self.storage_page_size,
methods: self.methods.clone(),
done: Default::default(),
keys_fut: Default::default(),
@@ -104,6 +143,7 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for LegacyBackend<T> {
let keys_stream = StorageFetchDescendantKeysStream {
at,
key,
storage_page_size: self.storage_page_size,
methods: self.methods.clone(),
done: Default::default(),
keys_fut: Default::default(),
@@ -332,9 +372,6 @@ where
})
}
/// How many keys/values to fetch at once.
const STORAGE_PAGE_SIZE: u32 = 32;
/// This provides a stream of values given some prefix `key`. It
/// internally manages pagination and such.
#[allow(clippy::type_complexity)]
@@ -342,6 +379,8 @@ pub struct StorageFetchDescendantKeysStream<T: Config> {
methods: LegacyRpcMethods<T>,
key: Vec<u8>,
at: T::Hash,
// How many entries to ask for each time.
storage_page_size: u32,
// What key do we start paginating from? None = from the beginning.
pagination_start_key: Option<Vec<u8>>,
// Keys, future and cached:
@@ -392,12 +431,13 @@ impl<T: Config> Stream for StorageFetchDescendantKeysStream<T> {
let methods = this.methods.clone();
let key = this.key.clone();
let at = this.at;
let storage_page_size = this.storage_page_size;
let pagination_start_key = this.pagination_start_key.take();
let keys_fut = async move {
methods
.state_get_keys_paged(
&key,
STORAGE_PAGE_SIZE,
storage_page_size,
pagination_start_key.as_deref(),
Some(at),
)
subxt/src/backend/rpc/lightclient_impl.rs
+53
View File
@@ -0,0 +1,53 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use super::{RawRpcFuture, RawRpcSubscription, RpcClientT};
use crate::error::RpcError;
use futures::stream::{StreamExt, TryStreamExt};
use serde_json::value::RawValue;
use subxt_lightclient::{LightClientRpc, LightClientRpcError};
impl RpcClientT for LightClientRpc {
fn request_raw<'a>(
&'a self,
method: &'a str,
params: Option<Box<RawValue>>,
) -> RawRpcFuture<'a, Box<RawValue>> {
Box::pin(async move {
let res = self.request(method.to_owned(), params)
.await
.map_err(lc_err_to_rpc_err)?;
Ok(res)
})
}
fn subscribe_raw<'a>(
&'a self,
sub: &'a str,
params: Option<Box<RawValue>>,
unsub: &'a str,
) -> RawRpcFuture<'a, RawRpcSubscription> {
Box::pin(async move {
let sub = self.subscribe(sub.to_owned(), params, unsub.to_owned())
.await
.map_err(lc_err_to_rpc_err)?;
let id = Some(sub.id().to_owned());
let stream = sub
.map_err(|e| RpcError::ClientError(Box::new(e)))
.boxed();
Ok(RawRpcSubscription { id, stream })
})
}
}
fn lc_err_to_rpc_err(err: LightClientRpcError) -> RpcError {
match err {
LightClientRpcError::JsonRpcError(e) => RpcError::ClientError(Box::new(e)),
LightClientRpcError::SmoldotError(e) => RpcError::RequestRejected(e),
LightClientRpcError::BackgroundTaskDropped => RpcError::SubscriptionDropped,
}
}
subxt/src/backend/rpc/mod.rs
+4
View File
@@ -60,6 +60,10 @@ crate::macros::cfg_jsonrpsee! {
mod jsonrpsee_impl;
}
crate::macros::cfg_unstable_light_client! {
mod lightclient_impl;
}
crate::macros::cfg_reconnecting_rpc_client! {
mod reconnecting_jsonrpsee_impl;
pub use reconnecting_jsonrpsee_ws_client as reconnecting_rpc_client;
subxt/src/backend/rpc/rpc_client.rs
+6
View File
@@ -79,6 +79,12 @@ impl RpcClient {
}
}
impl<C: RpcClientT> From<C> for RpcClient {
fn from(client: C) -> Self {
RpcClient::new(client)
}
}
impl std::fmt::Debug for RpcClient {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_tuple("RpcClient").finish()
subxt/src/backend/unstable/follow_stream.rs
+1 -1
View File
@@ -243,7 +243,7 @@ pub(super) mod test_utils {
/// An initialized event
pub fn ev_initialized(n: u64) -> FollowEvent<H256> {
FollowEvent::Initialized(Initialized {
finalized_block_hash: H256::from_low_u64_le(n),
finalized_block_hashes: vec![H256::from_low_u64_le(n)],
finalized_block_runtime: None,
})
}
subxt/src/backend/unstable/follow_stream_driver.rs
+3 -3
View File
@@ -267,9 +267,9 @@ impl<Hash: BlockHash> Shared<Hash> {
shared.seen_runtime_events.clear();
if let Some(finalized) = finalized_ev.finalized_block_hashes.last() {
init_message.finalized_block_hash = finalized.clone();
}
init_message.finalized_block_hashes =
finalized_ev.finalized_block_hashes.clone();
if let Some(runtime_ev) = newest_runtime {
init_message.finalized_block_runtime = Some(runtime_ev);
}
subxt/src/backend/unstable/follow_stream_unpin.rs
+50 -35
View File
@@ -10,6 +10,7 @@ use crate::backend::unstable::rpc_methods::{
use crate::config::{BlockHash, Config};
use crate::error::Error;
use futures::stream::{FuturesUnordered, Stream, StreamExt};
use std::collections::{HashMap, HashSet};
use std::future::Future;
use std::pin::Pin;
@@ -34,9 +35,11 @@ pub struct FollowStreamUnpin<Hash: BlockHash> {
// Futures for sending unpin events that we'll poll to completion as
// part of polling the stream as a whole.
unpin_futs: FuturesUnordered<UnpinFut>,
// Each new finalized block increments this. Allows us to track
// the age of blocks so that we can unpin old ones.
rel_block_num: usize,
// Each time a new finalized block is seen, we give it an age of `next_rel_block_age`,
// and then increment this ready for the next finalized block. So, the first finalized
// block will have an age of 0, the next 1, 2, 3 and so on. We can then use `max_block_life`
// to say "unpin all blocks with an age < (next_rel_block_age-1) - max_block_life".
next_rel_block_age: usize,
// The latest ID of the FollowStream subscription, which we can use
// to unpin blocks.
subscription_id: Option<Arc<str>>,
@@ -113,15 +116,23 @@ impl<Hash: BlockHash> Stream for FollowStreamUnpin<Hash> {
FollowStreamMsg::Ready(subscription_id)
}
FollowStreamMsg::Event(FollowEvent::Initialized(details)) => {
// The first finalized block gets the starting block_num.
let rel_block_num = this.rel_block_num;
// Pin this block, but note that it can be unpinned any time since it won't show up again (except
// as a parent block, which we are ignoring at the moment).
let block_ref =
this.pin_unpinnable_block_at(rel_block_num, details.finalized_block_hash);
let mut finalized_block_hashes =
Vec::with_capacity(details.finalized_block_hashes.len());
// Pin each of the finalized blocks. None of them will show up again (except as a
// parent block), and so they can all be unpinned immediately at any time. Increment
// the block age for each one, so that older finalized blocks are pruned first.
for finalized_block in &details.finalized_block_hashes {
let rel_block_age = this.next_rel_block_age;
let block_ref =
this.pin_unpinnable_block_at(rel_block_age, *finalized_block);
finalized_block_hashes.push(block_ref);
this.next_rel_block_age += 1;
}
FollowStreamMsg::Event(FollowEvent::Initialized(Initialized {
finalized_block_hash: block_ref,
finalized_block_hashes,
finalized_block_runtime: details.finalized_block_runtime,
}))
}
@@ -129,15 +140,15 @@ impl<Hash: BlockHash> Stream for FollowStreamUnpin<Hash> {
// One bigger than our parent, and if no parent seen (maybe it was
// unpinned already), then one bigger than the last finalized block num
// as a best guess.
let parent_rel_block_num = this
let parent_rel_block_age = this
.pinned
.get(&details.parent_block_hash)
.map(|p| p.rel_block_num)
.unwrap_or(this.rel_block_num);
.map(|p| p.rel_block_age)
.unwrap_or(this.next_rel_block_age.saturating_sub(1));
let block_ref = this.pin_block_at(parent_rel_block_num + 1, details.block_hash);
let block_ref = this.pin_block_at(parent_rel_block_age + 1, details.block_hash);
let parent_block_ref =
this.pin_block_at(parent_rel_block_num, details.parent_block_hash);
this.pin_block_at(parent_rel_block_age, details.parent_block_hash);
FollowStreamMsg::Event(FollowEvent::NewBlock(NewBlock {
block_hash: block_ref,
@@ -148,8 +159,8 @@ impl<Hash: BlockHash> Stream for FollowStreamUnpin<Hash> {
FollowStreamMsg::Event(FollowEvent::BestBlockChanged(details)) => {
// We expect this block to already exist, so it'll keep its existing block_num,
// but worst case it'll just get the current finalized block_num + 1.
let rel_block_num = this.rel_block_num + 1;
let block_ref = this.pin_block_at(rel_block_num, details.best_block_hash);
let rel_block_age = this.next_rel_block_age;
let block_ref = this.pin_block_at(rel_block_age, details.best_block_hash);
FollowStreamMsg::Event(FollowEvent::BestBlockChanged(BestBlockChanged {
best_block_hash: block_ref,
@@ -167,14 +178,14 @@ impl<Hash: BlockHash> Stream for FollowStreamUnpin<Hash> {
//
// `pin_unpinnable_block_at` indicates that the block will not show up in future events
// (They will show up as a parent block, but we don't care about that right now).
let rel_block_num = this.rel_block_num + idx + 1;
this.pin_unpinnable_block_at(rel_block_num, hash)
let rel_block_age = this.next_rel_block_age + idx;
this.pin_unpinnable_block_at(rel_block_age, hash)
})
.collect();
// Our relative block height is increased by however many finalized
// blocks we've seen.
this.rel_block_num += finalized_block_refs.len();
this.next_rel_block_age += finalized_block_refs.len();
let pruned_block_refs: Vec<_> = details
.pruned_block_hashes
@@ -183,8 +194,8 @@ impl<Hash: BlockHash> Stream for FollowStreamUnpin<Hash> {
// We should know about these, too, and if not we set their age to last_finalized + 1.
//
// `pin_unpinnable_block_at` indicates that the block will not show up in future events.
let rel_block_num = this.rel_block_num + 1;
this.pin_unpinnable_block_at(rel_block_num, hash)
let rel_block_age = this.next_rel_block_age;
this.pin_unpinnable_block_at(rel_block_age, hash)
})
.collect();
@@ -208,7 +219,7 @@ impl<Hash: BlockHash> Stream for FollowStreamUnpin<Hash> {
this.pinned.clear();
this.unpin_futs.clear();
this.unpin_flags.lock().unwrap().clear();
this.rel_block_num = 0;
this.next_rel_block_age = 0;
FollowStreamMsg::Event(FollowEvent::Stop)
}
@@ -255,7 +266,7 @@ impl<Hash: BlockHash> FollowStreamUnpin<Hash> {
max_block_life,
pinned: Default::default(),
subscription_id: None,
rel_block_num: 0,
next_rel_block_age: 0,
unpin_flags: Default::default(),
unpin_futs: Default::default(),
}
@@ -287,21 +298,21 @@ impl<Hash: BlockHash> FollowStreamUnpin<Hash> {
/// Pin a block, or return the reference to an already-pinned block. If the block has been registered to
/// be unpinned, we'll clear those flags, so that it won't be unpinned. If the unpin request has already
/// been sent though, then the block will be unpinned.
fn pin_block_at(&mut self, rel_block_num: usize, hash: Hash) -> BlockRef<Hash> {
self.pin_block_at_setting_unpinnable_flag(rel_block_num, hash, false)
fn pin_block_at(&mut self, rel_block_age: usize, hash: Hash) -> BlockRef<Hash> {
self.pin_block_at_setting_unpinnable_flag(rel_block_age, hash, false)
}
/// Pin a block, or return the reference to an already-pinned block.
///
/// This is the same as [`Self::pin_block_at`], except that it also marks the block as being unpinnable now,
/// which should be done for any block that will no longer be seen in future events.
fn pin_unpinnable_block_at(&mut self, rel_block_num: usize, hash: Hash) -> BlockRef<Hash> {
self.pin_block_at_setting_unpinnable_flag(rel_block_num, hash, true)
fn pin_unpinnable_block_at(&mut self, rel_block_age: usize, hash: Hash) -> BlockRef<Hash> {
self.pin_block_at_setting_unpinnable_flag(rel_block_age, hash, true)
}
fn pin_block_at_setting_unpinnable_flag(
&mut self,
rel_block_num: usize,
rel_block_age: usize,
hash: Hash,
can_be_unpinned: bool,
) -> BlockRef<Hash> {
@@ -317,7 +328,7 @@ impl<Hash: BlockHash> FollowStreamUnpin<Hash> {
})
// If there's not an entry already, make one and return it.
.or_insert_with(|| PinnedDetails {
rel_block_num,
rel_block_age,
block_ref: BlockRef {
inner: Arc::new(BlockRefInner {
hash,
@@ -333,7 +344,9 @@ impl<Hash: BlockHash> FollowStreamUnpin<Hash> {
/// Unpin any blocks that are either too old, or have the unpin flag set and are old enough.
fn unpin_blocks(&mut self, waker: &Waker) {
let mut unpin_flags = self.unpin_flags.lock().unwrap();
let rel_block_num = self.rel_block_num;
// This gets the age of the last finalized block.
let rel_block_age = self.next_rel_block_age.saturating_sub(1);
// If we asked to unpin and there was no subscription_id, then there's nothing we can do,
// and nothing will need unpinning now anyway.
@@ -343,7 +356,7 @@ impl<Hash: BlockHash> FollowStreamUnpin<Hash> {
let mut blocks_to_unpin = vec![];
for (hash, details) in &self.pinned {
if rel_block_num.saturating_sub(details.rel_block_num) >= self.max_block_life
if rel_block_age.saturating_sub(details.rel_block_age) >= self.max_block_life
|| (unpin_flags.contains(hash) && details.can_be_unpinned)
{
// The block is too old, or it's been flagged to be unpinned and won't be in a future
@@ -381,8 +394,10 @@ type UnpinFlags<Hash> = Arc<Mutex<HashSet<Hash>>>;
#[derive(Debug)]
struct PinnedDetails<Hash: BlockHash> {
/// How old is the block?
rel_block_num: usize,
/// Realtively speaking, how old is the block? When we start following
/// blocks, the first finalized block gets an age of 0, the second an age
/// of 1 and so on.
rel_block_age: usize,
/// A block ref we can hand out to keep blocks pinned.
/// Because we store one here until it's unpinned, the live count
/// will only drop to 1 when no external refs are left.
@@ -502,7 +517,7 @@ pub(super) mod test_utils {
/// An initialized event containing a BlockRef (useful for comparisons)
pub fn ev_initialized_ref(n: u64) -> FollowEvent<BlockRef<H256>> {
FollowEvent::Initialized(Initialized {
finalized_block_hash: BlockRef::new(H256::from_low_u64_le(n)),
finalized_block_hashes: vec![BlockRef::new(H256::from_low_u64_le(n))],
finalized_block_runtime: None,
})
}
subxt/src/backend/unstable/mod.rs
+21 -13
View File
@@ -75,9 +75,12 @@ impl<T: Config> UnstableBackendBuilder<T> {
/// Given an [`RpcClient`] to use to make requests, this returns a tuple of an [`UnstableBackend`],
/// which implements the [`Backend`] trait, and an [`UnstableBackendDriver`] which must be polled in
/// order for the backend to make progress.
pub fn build(self, client: RpcClient) -> (UnstableBackend<T>, UnstableBackendDriver<T>) {
pub fn build(
self,
client: impl Into<RpcClient>,
) -> (UnstableBackend<T>, UnstableBackendDriver<T>) {
// Construct the underlying follow_stream layers:
let rpc_methods = UnstableRpcMethods::new(client);
let rpc_methods = UnstableRpcMethods::new(client.into());
let follow_stream =
follow_stream::FollowStream::<T::Hash>::from_methods(rpc_methods.clone());
let follow_stream_unpin = follow_stream_unpin::FollowStreamUnpin::<T::Hash>::from_methods(
@@ -321,7 +324,9 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for UnstableBackend<T> {
.events()
.filter_map(|ev| {
let out = match ev {
FollowEvent::Initialized(init) => Some(init.finalized_block_hash.into()),
FollowEvent::Initialized(init) => {
init.finalized_block_hashes.last().map(|b| b.clone().into())
}
_ => None,
};
std::future::ready(out)
@@ -353,7 +358,10 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for UnstableBackend<T> {
.filter_map(move |ev| {
let output = match ev {
FollowEvent::Initialized(ev) => {
runtimes.remove(&ev.finalized_block_hash.hash());
for finalized_block in ev.finalized_block_hashes {
runtimes.remove(&finalized_block.hash());
}
ev.finalized_block_runtime
}
FollowEvent::NewBlock(ev) => {
@@ -422,9 +430,11 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for UnstableBackend<T> {
&self,
) -> Result<StreamOfResults<(T::Header, BlockRef<T::Hash>)>, Error> {
self.stream_headers(|ev| match ev {
FollowEvent::Initialized(ev) => Some(ev.finalized_block_hash),
FollowEvent::NewBlock(ev) => Some(ev.block_hash),
_ => None,
FollowEvent::Initialized(init) => init.finalized_block_hashes,
FollowEvent::NewBlock(ev) => {
vec![ev.block_hash]
}
_ => vec![],
})
.await
}
@@ -433,9 +443,9 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for UnstableBackend<T> {
&self,
) -> Result<StreamOfResults<(T::Header, BlockRef<T::Hash>)>, Error> {
self.stream_headers(|ev| match ev {
FollowEvent::Initialized(ev) => Some(ev.finalized_block_hash),
FollowEvent::BestBlockChanged(ev) => Some(ev.best_block_hash),
_ => None,
FollowEvent::Initialized(init) => init.finalized_block_hashes,
FollowEvent::BestBlockChanged(ev) => vec![ev.best_block_hash],
_ => vec![],
})
.await
}
@@ -444,9 +454,7 @@ impl<T: Config + Send + Sync + 'static> Backend<T> for UnstableBackend<T> {
&self,
) -> Result<StreamOfResults<(T::Header, BlockRef<T::Hash>)>, Error> {
self.stream_headers(|ev| match ev {
FollowEvent::Initialized(ev) => {
vec![ev.finalized_block_hash]
}
FollowEvent::Initialized(init) => init.finalized_block_hashes,
FollowEvent::Finalized(ev) => ev.finalized_block_hashes,
_ => vec![],
})
subxt/src/backend/unstable/rpc_methods.rs
+24 -2
View File
@@ -288,6 +288,28 @@ impl<T: Config> UnstableRpcMethods<T> {
Ok(TransactionSubscription { sub, done: false })
}
/// Broadcast the transaction on the p2p network until the
/// [`Self::transaction_unstable_stop`] is called.
///
/// Returns an operation ID that can be used to stop the broadcasting process.
/// Returns `None` if the server cannot handle the request at the moment.
pub async fn transaction_unstable_broadcast(&self, tx: &[u8]) -> Result<Option<String>, Error> {
self.client
.request("transaction_unstable_broadcast", rpc_params![to_hex(tx)])
.await
}
/// Stop the broadcasting process of the transaction.
///
/// The operation ID is obtained from the [`Self::transaction_unstable_broadcast`] method.
///
/// Returns an error if the operation ID does not correspond to any active transaction for this connection.
pub async fn transaction_unstable_stop(&self, operation_id: &str) -> Result<(), Error> {
self.client
.request("transaction_unstable_stop", rpc_params![operation_id])
.await
}
}
/// This represents events generated by the `follow` method.
@@ -359,8 +381,8 @@ pub enum FollowEvent<Hash> {
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Initialized<Hash> {
/// The hash of the latest finalized block.
pub finalized_block_hash: Hash,
/// The hashes of the last finalized blocks.
pub finalized_block_hashes: Vec<Hash>,
/// The runtime version of the finalized block.
///
/// # Note
subxt/src/blocks/extrinsic_types.rs
+21 -21
View File
@@ -245,27 +245,27 @@ where
// Skip over the address, signature and extra fields.
scale_decode::visitor::decode_with_visitor(
cursor,
ids.address,
&ids.address,
metadata.types(),
scale_decode::visitor::IgnoreVisitor,
scale_decode::visitor::IgnoreVisitor::new(),
)
.map_err(scale_decode::Error::from)?;
let address_end_idx = bytes.len() - cursor.len();
scale_decode::visitor::decode_with_visitor(
cursor,
ids.signature,
&ids.signature,
metadata.types(),
scale_decode::visitor::IgnoreVisitor,
scale_decode::visitor::IgnoreVisitor::new(),
)
.map_err(scale_decode::Error::from)?;
let signature_end_idx = bytes.len() - cursor.len();
scale_decode::visitor::decode_with_visitor(
cursor,
ids.extra,
&ids.extra,
metadata.types(),
scale_decode::visitor::IgnoreVisitor,
scale_decode::visitor::IgnoreVisitor::new(),
)
.map_err(scale_decode::Error::from)?;
let extra_end_idx = bytes.len() - cursor.len();
@@ -420,9 +420,7 @@ where
/// Decode and provide the extrinsic fields back in the form of a [`scale_value::Composite`]
/// type which represents the named or unnamed fields that were present in the extrinsic.
pub fn field_values(
&self,
) -> Result<scale_value::Composite<scale_value::scale::TypeId>, Error> {
pub fn field_values(&self) -> Result<scale_value::Composite<u32>, Error> {
let bytes = &mut self.field_bytes();
let extrinsic_metadata = self.extrinsic_metadata()?;
@@ -430,12 +428,9 @@ where
.variant
.fields
.iter()
.map(|f| scale_decode::Field::new(f.ty.id, f.name.as_deref()));
let decoded = <scale_value::Composite<scale_value::scale::TypeId>>::decode_as_fields(
bytes,
&mut fields,
self.metadata.types(),
)?;
.map(|f| scale_decode::Field::new(&f.ty.id, f.name.as_deref()));
let decoded =
scale_value::scale::decode_as_fields(bytes, &mut fields, self.metadata.types())?;
Ok(decoded)
}
@@ -451,7 +446,7 @@ where
.variant
.fields
.iter()
.map(|f| scale_decode::Field::new(f.ty.id, f.name.as_deref()));
.map(|f| scale_decode::Field::new(&f.ty.id, f.name.as_deref()));
let decoded =
E::decode_as_fields(&mut self.field_bytes(), &mut fields, self.metadata.types())?;
Ok(Some(decoded))
@@ -466,7 +461,7 @@ where
pub fn as_root_extrinsic<E: DecodeAsType>(&self) -> Result<E, Error> {
let decoded = E::decode_as_type(
&mut &self.call_bytes()[..],
self.metadata.outer_enums().call_enum_ty(),
&self.metadata.outer_enums().call_enum_ty(),
self.metadata.types(),
)?;
@@ -651,9 +646,9 @@ impl<'a, T: Config> ExtrinsicSignedExtensions<'a, T> {
let cursor = &mut &bytes[byte_start_idx..];
if let Err(err) = scale_decode::visitor::decode_with_visitor(
cursor,
ty_id,
&ty_id,
metadata.types(),
scale_decode::visitor::IgnoreVisitor,
scale_decode::visitor::IgnoreVisitor::new(),
)
.map_err(|e| Error::Decode(e.into()))
{
@@ -748,7 +743,12 @@ impl<'a, T: Config> ExtrinsicSignedExtension<'a, T> {
/// Signed Extension as a [`scale_value::Value`]
pub fn value(&self) -> Result<DecodedValue, Error> {
self.as_type()
let value = scale_value::scale::decode_as_type(
&mut &self.bytes[..],
&self.ty_id,
self.metadata.types(),
)?;
Ok(value)
}
/// Decodes the bytes of this Signed Extension into its associated `Decoded` type.
@@ -762,7 +762,7 @@ impl<'a, T: Config> ExtrinsicSignedExtension<'a, T> {
}
fn as_type<E: DecodeAsType>(&self) -> Result<E, Error> {
let value = E::decode_as_type(&mut &self.bytes[..], self.ty_id, self.metadata.types())?;
let value = E::decode_as_type(&mut &self.bytes[..], &self.ty_id, self.metadata.types())?;
Ok(value)
}
}
subxt/src/book/setup/config.rs
+1 -1
View File
@@ -71,7 +71,7 @@
//!
//! The `ExtrinsicParams` config type expects to be given an implementation of the [`crate::config::ExtrinsicParams`] trait.
//! Implementations of the [`crate::config::ExtrinsicParams`] trait are handed some parameters from Subxt itself, and can
//! accept arbitrary `OtherParams` from users, and are then expected to provide this "extra" and "additional" data when asked
//! accept arbitrary other `Params` from users, and are then expected to provide this "extra" and "additional" data when asked
//! via the required [`crate::config::ExtrinsicParamsEncoder`] impl.
//!
//! **In most cases, the default [crate::config::DefaultExtrinsicParams] type will work**: it understands the "standard"
subxt/src/book/usage/light_client.rs
+16 -21
View File
@@ -8,25 +8,33 @@
//! node. This means that you don't have to trust a specific node when interacting with some chain.
//!
//! This feature is currently unstable. Use the `unstable-light-client` feature flag to enable it.
//! To use this in WASM environments, also enable the `web` feature flag.
//! To use this in WASM environments, enable the `web` feature flag and disable the "native" one.
//!
//! To connect to a blockchain network, the Light Client requires a trusted sync state of the network,
//! known as a _chain spec_. One way to obtain this is by making a `sync_state_genSyncSpec` RPC call to a
//! trusted node belonging to the chain that you wish to interact with.
//!
//! The following is an example of fetching the chain spec from a local running node on port 9933:
//! Subxt exposes a utility method to obtain the chain spec: [`crate::utils::fetch_chainspec_from_rpc_node()`].
//! Alternately, you can manually make an RPC call to `sync_state_genSyncSpec` like do (assuming a node running
//! locally on port 9933):
//!
//! ```bash
//! curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "sync_state_genSyncSpec", "params":[true]}' http://localhost:9933/ | jq .result > chain_spec.json
//! ```
//!
//! Alternately, you can have the `LightClient` download the chain spec from a trusted node when it
//! initializes, which is not recommended in production but is useful for examples and testing, as below.
//!
//! ## Examples
//!
//! ### Basic Example
//!
//! This basic example uses some already-known chain specs to connect to a relay chain and parachain
//! and stream information about their finalized blocks:
//!
//! ```rust,ignore
#![doc = include_str!("../../../examples/light_client_basic.rs")]
//! ```
//!
//! ### Connecting to a local node
//!
//! This example connects to a local chain and submits a transaction. To run this, you first need
//! to have a local polkadot node running using the following command:
//!
@@ -34,23 +42,10 @@
//! polkadot --dev --node-key 0000000000000000000000000000000000000000000000000000000000000001
//! ```
//!
//! Leave that running for a minute, and then you can run the example using the following command
//! in the `subxt` crate:
//!
//! ```bash
//! cargo run --example light_client_tx_basic --features=unstable-light-client
//! ```
//!
//! This is the code that will be executed:
//! Then, the following code will download a chain spec from this local node, alter the bootnodes
//! to point only to the local node, and then submit a transaction through it.
//!
//! ```rust,ignore
#![doc = include_str!("../../../examples/light_client_tx_basic.rs")]
#![doc = include_str!("../../../examples/light_client_local_node.rs")]
//! ```
//!
//! ### Connecting to a parachain
//!
//! This example connects to a parachain using the light client. Currently, it's quite verbose to do this.
//!
//! ```rust,ignore
#![doc = include_str!("../../../examples/light_client_parachains.rs")]
//! ```
subxt/src/book/usage/storage.rs
+1 -1
View File
@@ -49,7 +49,7 @@
//! // A static query capable of iterating over accounts:
//! let storage_query = polkadot::storage().system().account_iter();
//! // A dynamic query to do the same:
//! let storage_query = subxt::dynamic::storage("System", "Account", Vec::<u8>::new());
//! let storage_query = subxt::dynamic::storage("System", "Account", ());
//! ```
//!
//! Some storage entries are maps with multiple keys. As an example, we might end up with
subxt/src/book/usage/transactions.rs
+3 -4
View File
@@ -137,11 +137,10 @@
//! Value::from_bytes("Hello there")
//! ]);
//!
//! // Construct the tx but don't sign it. You need to provide the nonce
//! // here, or can use `create_partial_signed` to fetch the correct nonce.
//! let partial_tx = client.tx().create_partial_signed_with_nonce(
//! // Construct the tx but don't sign it. The account nonce here defaults to 0.
//! // You can use `create_partial_signed` to fetch the correct nonce.
//! let partial_tx = client.tx().create_partial_signed_offline(
//! &payload,
//! 0u64,
//! Default::default()
//! )?;
//!
subxt/src/client/light_client/builder.rs
-336
View File
@@ -1,336 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use super::{rpc::LightClientRpc, LightClient, LightClientError};
use crate::backend::rpc::RpcClient;
use crate::client::RawLightClient;
use crate::macros::{cfg_jsonrpsee_native, cfg_jsonrpsee_web};
use crate::{config::Config, error::Error, OnlineClient};
use std::num::NonZeroU32;
use subxt_lightclient::{smoldot, AddedChain};
#[cfg(feature = "jsonrpsee")]
use crate::utils::validate_url_is_secure;
/// Builder for [`LightClient`].
#[derive(Clone, Debug)]
pub struct LightClientBuilder<T: Config> {
max_pending_requests: NonZeroU32,
max_subscriptions: u32,
bootnodes: Option<Vec<serde_json::Value>>,
potential_relay_chains: Option<Vec<smoldot::ChainId>>,
_marker: std::marker::PhantomData<T>,
}
impl<T: Config> Default for LightClientBuilder<T> {
fn default() -> Self {
Self {
max_pending_requests: NonZeroU32::new(128)
.expect("Valid number is greater than zero; qed"),
max_subscriptions: 1024,
bootnodes: None,
potential_relay_chains: None,
_marker: std::marker::PhantomData,
}
}
}
impl<T: Config> LightClientBuilder<T> {
/// Create a new [`LightClientBuilder`].
pub fn new() -> LightClientBuilder<T> {
LightClientBuilder::default()
}
/// Overwrite the bootnodes of the chain specification.
///
/// Can be used to provide trusted entities to the chain spec, or for
/// testing environments.
pub fn bootnodes<'a>(mut self, bootnodes: impl IntoIterator<Item = &'a str>) -> Self {
self.bootnodes = Some(bootnodes.into_iter().map(Into::into).collect());
self
}
/// Maximum number of JSON-RPC in the queue of requests waiting to be processed.
/// This parameter is necessary for situations where the JSON-RPC clients aren't
/// trusted. If you control all the requests that are sent out and don't want them
/// to fail, feel free to pass `u32::max_value()`.
///
/// Default is 128.
pub fn max_pending_requests(mut self, max_pending_requests: NonZeroU32) -> Self {
self.max_pending_requests = max_pending_requests;
self
}
/// Maximum number of active subscriptions before new ones are automatically
/// rejected. Any JSON-RPC request that causes the server to generate notifications
/// counts as a subscription.
///
/// Default is 1024.
pub fn max_subscriptions(mut self, max_subscriptions: u32) -> Self {
self.max_subscriptions = max_subscriptions;
self
}
/// If the chain spec defines a parachain, contains the list of relay chains to choose
/// from. Ignored if not a parachain.
///
/// This field is necessary because multiple different chain can have the same identity.
///
/// For example: if user A adds a chain named "Kusama", then user B adds a different chain
/// also named "Kusama", then user B adds a parachain whose relay chain is "Kusama", it would
/// be wrong to connect to the "Kusama" created by user A.
pub fn potential_relay_chains(
mut self,
potential_relay_chains: impl IntoIterator<Item = smoldot::ChainId>,
) -> Self {
self.potential_relay_chains = Some(potential_relay_chains.into_iter().collect());
self
}
/// Build the light client with specified URL to connect to.
/// You must provide the port number in the URL.
///
/// ## Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ### Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ### Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
#[cfg(feature = "jsonrpsee")]
#[cfg_attr(docsrs, doc(cfg(feature = "jsonrpsee")))]
pub async fn build_from_url<Url: AsRef<str>>(self, url: Url) -> Result<LightClient<T>, Error> {
validate_url_is_secure(url.as_ref())?;
self.build_from_insecure_url(url).await
}
/// Build the light client with specified URL to connect to. Allows insecure URLs (no SSL, ws:// or http://).
///
/// For secure connections only, please use [`crate::LightClientBuilder::build_from_url`].
#[cfg(feature = "jsonrpsee")]
pub async fn build_from_insecure_url<Url: AsRef<str>>(
self,
url: Url,
) -> Result<LightClient<T>, Error> {
let chain_spec = fetch_url(url.as_ref()).await?;
self.build_client(chain_spec).await
}
/// Build the light client from chain spec.
///
/// The most important field of the configuration is the chain specification.
/// This is a JSON document containing all the information necessary for the client to
/// connect to said chain.
///
/// The chain spec must be obtained from a trusted entity.
///
/// It can be fetched from a trusted node with the following command:
/// ```bash
/// curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "sync_state_genSyncSpec", "params":[true]}' http://localhost:9944/ | jq .result > res.spec
/// ```
///
/// # Note
///
/// For testing environments, please populate the "bootNodes" if the not already provided.
/// See [`Self::bootnodes`] for more details.
///
/// ## Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ### Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ### Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
pub async fn build(self, chain_spec: &str) -> Result<LightClient<T>, Error> {
let chain_spec = serde_json::from_str(chain_spec)
.map_err(|_| Error::LightClient(LightClientError::InvalidChainSpec))?;
self.build_client(chain_spec).await
}
/// Build the light client.
async fn build_client(
self,
mut chain_spec: serde_json::Value,
) -> Result<LightClient<T>, Error> {
// Set custom bootnodes if provided.
if let Some(bootnodes) = self.bootnodes {
if let serde_json::Value::Object(map) = &mut chain_spec {
map.insert("bootNodes".to_string(), serde_json::Value::Array(bootnodes));
}
}
let config = smoldot::AddChainConfig {
specification: &chain_spec.to_string(),
json_rpc: smoldot::AddChainConfigJsonRpc::Enabled {
max_pending_requests: self.max_pending_requests,
max_subscriptions: self.max_subscriptions,
},
potential_relay_chains: self.potential_relay_chains.unwrap_or_default().into_iter(),
database_content: "",
user_data: (),
};
let raw_rpc = LightClientRpc::new(config)?;
build_client_from_rpc(raw_rpc).await
}
}
/// Raw builder for [`RawLightClient`].
#[derive(Default)]
pub struct RawLightClientBuilder {
chains: Vec<AddedChain>,
}
impl RawLightClientBuilder {
/// Create a new [`RawLightClientBuilder`].
pub fn new() -> RawLightClientBuilder {
RawLightClientBuilder::default()
}
/// Adds a new chain to the list of chains synchronized by the light client.
pub fn add_chain(
mut self,
chain_id: smoldot::ChainId,
rpc_responses: smoldot::JsonRpcResponses,
) -> Self {
self.chains.push(AddedChain {
chain_id,
rpc_responses,
});
self
}
/// Construct a [`RawLightClient`] from a raw smoldot client.
///
/// The provided `chain_id` is the chain with which the current instance of light client will interact.
/// To target a different chain call the [`LightClient::target_chain`] method.
pub async fn build<TPlatform: smoldot::PlatformRef>(
self,
client: smoldot::Client<TPlatform>,
) -> Result<RawLightClient, Error> {
// The raw subxt light client that spawns the smoldot background task.
let raw_rpc: subxt_lightclient::RawLightClientRpc =
subxt_lightclient::LightClientRpc::new_from_client(client, self.chains.into_iter());
// The crate implementation of `RpcClientT` over the raw subxt light client.
let raw_rpc = crate::client::light_client::rpc::RawLightClientRpc::from_inner(raw_rpc);
Ok(RawLightClient { raw_rpc })
}
}
/// Build the light client from a raw rpc client.
async fn build_client_from_rpc<T: Config>(
raw_rpc: LightClientRpc,
) -> Result<LightClient<T>, Error> {
let chain_id = raw_rpc.chain_id();
let rpc_client = RpcClient::new(raw_rpc);
let client = OnlineClient::<T>::from_rpc_client(rpc_client).await?;
Ok(LightClient { client, chain_id })
}
/// Fetch the chain spec from the URL.
#[cfg(feature = "jsonrpsee")]
async fn fetch_url(url: impl AsRef<str>) -> Result<serde_json::Value, Error> {
use jsonrpsee::core::client::{ClientT, SubscriptionClientT};
use jsonrpsee::rpc_params;
use serde_json::value::RawValue;
let client = jsonrpsee_helpers::client(url.as_ref()).await?;
let result = client
.request("sync_state_genSyncSpec", jsonrpsee::rpc_params![true])
.await
.map_err(|err| Error::Rpc(crate::error::RpcError::ClientError(Box::new(err))))?;
// Subscribe to the finalized heads of the chain.
let mut subscription = SubscriptionClientT::subscribe::<Box<RawValue>, _>(
&client,
"chain_subscribeFinalizedHeads",
rpc_params![],
"chain_unsubscribeFinalizedHeads",
)
.await
.map_err(|err| Error::Rpc(crate::error::RpcError::ClientError(Box::new(err))))?;
// We must ensure that the finalized block of the chain is not the block included
// in the chainSpec.
// This is a temporary workaround for: https://github.com/smol-dot/smoldot/issues/1562.
// The first finalized block that is received might by the finalized block could be the one
// included in the chainSpec. Decoding the chainSpec for this purpose is too complex.
let _ = subscription.next().await;
let _ = subscription.next().await;
Ok(result)
}
cfg_jsonrpsee_native! {
mod jsonrpsee_helpers {
use crate::error::{Error, LightClientError};
use tokio_util::compat::Compat;
pub use jsonrpsee::{
client_transport::ws::{self, EitherStream, Url, WsTransportClientBuilder},
core::client::Client,
};
pub type Sender = ws::Sender<Compat<EitherStream>>;
pub type Receiver = ws::Receiver<Compat<EitherStream>>;
/// Build WS RPC client from URL
pub async fn client(url: &str) -> Result<Client, Error> {
let url = Url::parse(url).map_err(|_| Error::LightClient(LightClientError::InvalidUrl))?;
if url.scheme() != "ws" && url.scheme() != "wss" {
return Err(Error::LightClient(LightClientError::InvalidScheme));
}
let (sender, receiver) = ws_transport(url).await?;
Ok(Client::builder()
.max_buffer_capacity_per_subscription(4096)
.build_with_tokio(sender, receiver))
}
async fn ws_transport(url: Url) -> Result<(Sender, Receiver), Error> {
WsTransportClientBuilder::default()
.build(url)
.await
.map_err(|_| Error::LightClient(LightClientError::Handshake))
}
}
}
cfg_jsonrpsee_web! {
mod jsonrpsee_helpers {
use crate::error::{Error, LightClientError};
pub use jsonrpsee::{
client_transport::web,
core::client::{Client, ClientBuilder},
};
/// Build web RPC client from URL
pub async fn client(url: &str) -> Result<Client, Error> {
let (sender, receiver) = web::connect(url)
.await
.map_err(|_| Error::LightClient(LightClientError::Handshake))?;
Ok(ClientBuilder::default()
.max_buffer_capacity_per_subscription(4096)
.build_with_wasm(sender, receiver))
}
}
}
subxt/src/client/light_client/rpc.rs
-162
View File
@@ -1,162 +0,0 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use super::{smoldot, LightClientError};
use crate::{
backend::rpc::{RawRpcFuture, RawRpcSubscription, RpcClientT},
error::{Error, RpcError},
};
use futures::StreamExt;
use serde_json::value::RawValue;
use tokio_stream::wrappers::UnboundedReceiverStream;
pub const LOG_TARGET: &str = "subxt-rpc-light-client";
/// The raw light-client RPC implementation that is used to connect with the chain.
#[derive(Clone)]
pub struct RawLightClientRpc(subxt_lightclient::RawLightClientRpc);
impl RawLightClientRpc {
/// Constructs a new [`RawLightClientRpc`] from a low level [`subxt_lightclient::RawLightClientRpc`].
pub fn from_inner(client: subxt_lightclient::RawLightClientRpc) -> RawLightClientRpc {
RawLightClientRpc(client)
}
/// Constructs a new [`LightClientRpc`] that communicates with the provided chain.
pub fn for_chain(&self, chain_id: smoldot::ChainId) -> LightClientRpc {
LightClientRpc(self.0.for_chain(chain_id))
}
}
/// The light-client RPC implementation that is used to connect with the chain.
#[derive(Clone)]
pub struct LightClientRpc(subxt_lightclient::LightClientRpc);
impl LightClientRpc {
/// Constructs a new [`LightClientRpc`], providing the chain specification.
///
/// The chain specification can be downloaded from a trusted network via
/// the `sync_state_genSyncSpec` RPC method. This parameter expects the
/// chain spec in text format (ie not in hex-encoded scale-encoded as RPC methods
/// will provide).
///
/// ## Panics
///
/// The panic behaviour depends on the feature flag being used:
///
/// ### Native
///
/// Panics when called outside of a `tokio` runtime context.
///
/// ### Web
///
/// If smoldot panics, then the promise created will be leaked. For more details, see
/// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
pub fn new(
config: smoldot::AddChainConfig<'_, (), impl Iterator<Item = smoldot::ChainId>>,
) -> Result<LightClientRpc, Error> {
let rpc = subxt_lightclient::LightClientRpc::new(config).map_err(LightClientError::Rpc)?;
Ok(LightClientRpc(rpc))
}
/// Returns the chain ID of the current light-client.
pub fn chain_id(&self) -> smoldot::ChainId {
self.0.chain_id()
}
}
impl RpcClientT for LightClientRpc {
fn request_raw<'a>(
&'a self,
method: &'a str,
params: Option<Box<RawValue>>,
) -> RawRpcFuture<'a, Box<RawValue>> {
let client = self.clone();
let chain_id = self.chain_id();
Box::pin(async move {
let params = match params {
Some(params) => serde_json::to_string(&params).map_err(|_| {
RpcError::ClientError(Box::new(LightClientError::InvalidParams))
})?,
None => "[]".into(),
};
// Fails if the background is closed.
let rx = client
.0
.method_request(method.to_string(), params)
.map_err(|_| RpcError::ClientError(Box::new(LightClientError::BackgroundClosed)))?;
// Fails if the background is closed.
let response = rx
.await
.map_err(|_| RpcError::ClientError(Box::new(LightClientError::BackgroundClosed)))?;
tracing::trace!(target: LOG_TARGET, "RPC response={:?} chain={:?}", response, chain_id);
response.map_err(|err| RpcError::ClientError(Box::new(err)))
})
}
fn subscribe_raw<'a>(
&'a self,
sub: &'a str,
params: Option<Box<RawValue>>,
unsub: &'a str,
) -> RawRpcFuture<'a, RawRpcSubscription> {
let client = self.clone();
let chain_id = self.chain_id();
Box::pin(async move {
tracing::trace!(
target: LOG_TARGET,
"Subscribe to {:?} with params {:?} chain={:?}",
sub,
params,
chain_id,
);
let params = match params {
Some(params) => serde_json::to_string(&params).map_err(|_| {
RpcError::ClientError(Box::new(LightClientError::InvalidParams))
})?,
None => "[]".into(),
};
// Fails if the background is closed.
let (sub_id, notif) = client
.0
.subscription_request(sub.to_string(), params, unsub.to_string())
.map_err(|_| RpcError::ClientError(Box::new(LightClientError::BackgroundClosed)))?;
// Fails if the background is closed.
let result = sub_id
.await
.map_err(|_| RpcError::ClientError(Box::new(LightClientError::BackgroundClosed)))?
.map_err(|err| {
RpcError::ClientError(Box::new(LightClientError::Rpc(
subxt_lightclient::LightClientRpcError::Request(err.to_string()),
)))
})?;
let sub_id = result
.get()
.trim_start_matches('"')
.trim_end_matches('"')
.to_string();
tracing::trace!(target: LOG_TARGET, "Received subscription={} chain={:?}", sub_id, chain_id);
let stream = UnboundedReceiverStream::new(notif);
let rpc_subscription = RawRpcSubscription {
stream: Box::pin(stream.map(Ok)),
id: Some(sub_id),
};
Ok(rpc_subscription)
})
}
}
subxt/src/client/mod.rs
-8
View File
@@ -11,14 +11,6 @@
mod offline_client;
mod online_client;
crate::macros::cfg_unstable_light_client! {
mod light_client;
pub use light_client::{
LightClient, LightClientBuilder, LightClientError, RawLightClient, RawLightClientBuilder,
};
}
pub use offline_client::{OfflineClient, OfflineClientT};
pub use online_client::{
ClientRuntimeUpdater, OnlineClient, OnlineClientT, RuntimeUpdaterStream, Update, UpgradeError,
subxt/src/client/online_client.rs
+9 -5
View File
@@ -76,7 +76,7 @@ impl<T: Config> OnlineClient<T> {
/// Allows insecure URLs without SSL encryption, e.g. (http:// and ws:// URLs).
pub async fn from_insecure_url(url: impl AsRef<str>) -> Result<OnlineClient<T>, Error> {
let client = RpcClient::from_insecure_url(url).await?;
let backend = LegacyBackend::new(client);
let backend = LegacyBackend::builder().build(client);
OnlineClient::from_backend(Arc::new(backend)).await
}
}
@@ -84,8 +84,11 @@ impl<T: Config> OnlineClient<T> {
impl<T: Config> OnlineClient<T> {
/// Construct a new [`OnlineClient`] by providing an [`RpcClient`] to drive the connection.
/// This will use the current default [`Backend`], which may change in future releases.
pub async fn from_rpc_client(rpc_client: RpcClient) -> Result<OnlineClient<T>, Error> {
let backend = Arc::new(LegacyBackend::new(rpc_client));
pub async fn from_rpc_client(
rpc_client: impl Into<RpcClient>,
) -> Result<OnlineClient<T>, Error> {
let rpc_client = rpc_client.into();
let backend = Arc::new(LegacyBackend::builder().build(rpc_client));
OnlineClient::from_backend(backend).await
}
@@ -106,9 +109,10 @@ impl<T: Config> OnlineClient<T> {
genesis_hash: T::Hash,
runtime_version: RuntimeVersion,
metadata: impl Into<Metadata>,
rpc_client: RpcClient,
rpc_client: impl Into<RpcClient>,
) -> Result<OnlineClient<T>, Error> {
let backend = Arc::new(LegacyBackend::new(rpc_client));
let rpc_client = rpc_client.into();
let backend = Arc::new(LegacyBackend::builder().build(rpc_client));
OnlineClient::from_backend_with(genesis_hash, runtime_version, metadata, backend)
}
subxt/src/config/default_extrinsic_params.rs
+15 -3
View File
@@ -2,6 +2,7 @@
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use super::signed_extensions::CheckNonceParams;
use super::{signed_extensions, ExtrinsicParams};
use super::{Config, Header};
@@ -20,12 +21,14 @@ pub type DefaultExtrinsicParams<T> = signed_extensions::AnyOf<
),
>;
/// A builder that outputs the set of [`super::ExtrinsicParams::OtherParams`] required for
/// A builder that outputs the set of [`super::ExtrinsicParams::Params`] required for
/// [`DefaultExtrinsicParams`]. This may expose methods that aren't applicable to the current
/// chain; such values will simply be ignored if so.
pub struct DefaultExtrinsicParamsBuilder<T: Config> {
/// `None` means the tx will be immortal.
mortality: Option<Mortality<T::Hash>>,
/// `None` means the nonce will be automatically set.
nonce: Option<u64>,
/// `None` means we'll use the native token.
tip_of_asset_id: Option<T::AssetId>,
tip: u128,
@@ -49,6 +52,7 @@ impl<T: Config> Default for DefaultExtrinsicParamsBuilder<T> {
tip: 0,
tip_of: 0,
tip_of_asset_id: None,
nonce: None,
}
}
}
@@ -72,6 +76,12 @@ impl<T: Config> DefaultExtrinsicParamsBuilder<T> {
self
}
/// Provide a specific nonce for the submitter of the extrinsic
pub fn nonce(mut self, nonce: u64) -> Self {
self.nonce = Some(nonce);
self
}
/// Make the transaction mortal, given a block number and block hash (which must both point to
/// the same block) that it should be mortal from, and the number of blocks (roughly; it'll be
/// rounded to a power of two) that it will be mortal for.
@@ -111,7 +121,7 @@ impl<T: Config> DefaultExtrinsicParamsBuilder<T> {
}
/// Build the extrinsic parameters.
pub fn build(self) -> <DefaultExtrinsicParams<T> as ExtrinsicParams<T>>::OtherParams {
pub fn build(self) -> <DefaultExtrinsicParams<T> as ExtrinsicParams<T>>::Params {
let check_mortality_params = if let Some(mortality) = self.mortality {
signed_extensions::CheckMortalityParams::mortal(
mortality.period,
@@ -131,10 +141,12 @@ impl<T: Config> DefaultExtrinsicParamsBuilder<T> {
let charge_transaction_params =
signed_extensions::ChargeTransactionPaymentParams::tip(self.tip);
let check_nonce_params = CheckNonceParams(self.nonce);
(
(),
(),
(),
check_nonce_params,
(),
check_mortality_params,
charge_asset_tx_params,
subxt/src/config/extrinsic_params.rs
+4 -3
View File
@@ -10,6 +10,8 @@
use crate::{client::OfflineClientT, Config};
use core::fmt::Debug;
use super::refine_params::RefineParams;
/// An error that can be emitted when trying to construct an instance of [`ExtrinsicParams`],
/// encode data from the instance, or match on signed extensions.
#[derive(thiserror::Error, Debug)]
@@ -53,13 +55,12 @@ pub trait ExtrinsicParams<T: Config>: ExtrinsicParamsEncoder + Sized + 'static {
/// These parameters can be provided to the constructor along with
/// some default parameters that `subxt` understands, in order to
/// help construct your [`ExtrinsicParams`] object.
type OtherParams;
type Params: RefineParams<T>;
/// Construct a new instance of our [`ExtrinsicParams`].
fn new<Client: OfflineClientT<T>>(
nonce: u64,
client: Client,
other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError>;
}
subxt/src/config/mod.rs
+3 -1
View File
@@ -10,6 +10,7 @@
mod default_extrinsic_params;
mod extrinsic_params;
mod refine_params;
pub mod polkadot;
pub mod signed_extensions;
@@ -25,6 +26,7 @@ use serde::{de::DeserializeOwned, Serialize};
pub use default_extrinsic_params::{DefaultExtrinsicParams, DefaultExtrinsicParamsBuilder};
pub use extrinsic_params::{ExtrinsicParams, ExtrinsicParamsEncoder, ExtrinsicParamsError};
pub use polkadot::{PolkadotConfig, PolkadotExtrinsicParams, PolkadotExtrinsicParamsBuilder};
pub use refine_params::{RefineParams, RefineParamsData};
pub use signed_extensions::SignedExtension;
pub use substrate::{SubstrateConfig, SubstrateExtrinsicParams, SubstrateExtrinsicParamsBuilder};
@@ -60,7 +62,7 @@ pub trait Config: Sized + Send + Sync + 'static {
}
/// given some [`Config`], this return the other params needed for its `ExtrinsicParams`.
pub type OtherParamsFor<T> = <<T as Config>::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams;
pub type ParamsFor<T> = <<T as Config>::ExtrinsicParams as ExtrinsicParams<T>>::Params;
/// Block hashes must conform to a bunch of things to be used in Subxt.
pub trait BlockHash:
subxt/src/config/refine_params.rs
+85
View File
@@ -0,0 +1,85 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
//! Refining params with values fetched from the chain
use crate::Config;
/// Data that can be used to refine the params of signed extensions.
pub struct RefineParamsData<T: Config> {
account_nonce: u64,
block_number: u64,
block_hash: T::Hash,
}
impl<T: Config> RefineParamsData<T> {
pub(crate) fn new(account_nonce: u64, block_number: u64, block_hash: T::Hash) -> Self {
RefineParamsData {
account_nonce,
block_number,
block_hash,
}
}
/// account nonce for extrinsic author
pub fn account_nonce(&self) -> u64 {
self.account_nonce
}
/// latest finalized block number
pub fn block_number(&self) -> u64 {
self.block_number
}
/// latest finalized block hash
pub fn block_hash(&self) -> T::Hash {
self.block_hash
}
}
/// Types implementing [`RefineParams`] can be modified to reflect live information from the chain.
pub trait RefineParams<T: Config> {
/// Refine params to an extrinsic. There is usually some notion of 'the param is already set/unset' in types implementing this trait.
/// The refinement should most likely not affect cases where a param is in a 'is already set by the user' state.
fn refine(&mut self, _data: &RefineParamsData<T>) {}
}
impl<T: Config> RefineParams<T> for () {}
macro_rules! impl_tuples {
($($ident:ident $index:tt),+) => {
impl <T: Config, $($ident : RefineParams<T>),+> RefineParams<T> for ($($ident,)+){
fn refine(&mut self, data: &RefineParamsData<T>) {
$(self.$index.refine(data);)+
}
}
}
}
#[rustfmt::skip]
const _: () = {
impl_tuples!(A 0);
impl_tuples!(A 0, B 1);
impl_tuples!(A 0, B 1, C 2);
impl_tuples!(A 0, B 1, C 2, D 3);
impl_tuples!(A 0, B 1, C 2, D 3, E 4);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14, P 15);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14, P 15, Q 16);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14, P 15, Q 16, R 17);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14, P 15, Q 16, R 17, S 18);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14, P 15, Q 16, R 17, S 18, U 19);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7, I 8, J 9, K 10, L 11, M 12, N 13, O 14, P 15, Q 16, R 17, S 18, U 19, V 20);
};
subxt/src/config/signed_extensions.rs
+69 -40
View File
@@ -8,6 +8,8 @@
//! when interacting with a chain.
use super::extrinsic_params::{ExtrinsicParams, ExtrinsicParamsEncoder, ExtrinsicParamsError};
use super::refine_params::RefineParamsData;
use super::RefineParams;
use crate::utils::Era;
use crate::{client::OfflineClientT, Config};
use codec::{Compact, Encode};
@@ -37,12 +39,11 @@ pub trait SignedExtension<T: Config>: ExtrinsicParams<T> {
pub struct CheckSpecVersion(u32);
impl<T: Config> ExtrinsicParams<T> for CheckSpecVersion {
type OtherParams = ();
type Params = ();
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
client: Client,
_other_params: Self::OtherParams,
_params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(CheckSpecVersion(client.runtime_version().spec_version))
}
@@ -65,13 +66,14 @@ impl<T: Config> SignedExtension<T> for CheckSpecVersion {
pub struct CheckNonce(Compact<u64>);
impl<T: Config> ExtrinsicParams<T> for CheckNonce {
type OtherParams = ();
type Params = CheckNonceParams;
fn new<Client: OfflineClientT<T>>(
nonce: u64,
_client: Client,
_other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
// If no nonce is set (nor by user nor refinement), use a nonce of 0.
let nonce = params.0.unwrap_or(0);
Ok(CheckNonce(Compact(nonce)))
}
}
@@ -89,16 +91,27 @@ impl<T: Config> SignedExtension<T> for CheckNonce {
}
}
/// Params for [`CheckNonce`]
#[derive(Debug, Clone, Default)]
pub struct CheckNonceParams(pub Option<u64>);
impl<T: Config> RefineParams<T> for CheckNonceParams {
fn refine(&mut self, data: &RefineParamsData<T>) {
if self.0.is_none() {
self.0 = Some(data.account_nonce());
}
}
}
/// The [`CheckTxVersion`] signed extension.
pub struct CheckTxVersion(u32);
impl<T: Config> ExtrinsicParams<T> for CheckTxVersion {
type OtherParams = ();
type Params = ();
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
client: Client,
_other_params: Self::OtherParams,
_params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(CheckTxVersion(client.runtime_version().transaction_version))
}
@@ -121,12 +134,11 @@ impl<T: Config> SignedExtension<T> for CheckTxVersion {
pub struct CheckGenesis<T: Config>(T::Hash);
impl<T: Config> ExtrinsicParams<T> for CheckGenesis<T> {
type OtherParams = ();
type Params = ();
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
client: Client,
_other_params: Self::OtherParams,
_params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(CheckGenesis(client.genesis_hash()))
}
@@ -152,16 +164,25 @@ pub struct CheckMortality<T: Config> {
}
/// Parameters to configure the [`CheckMortality`] signed extension.
pub struct CheckMortalityParams<T: Config> {
pub struct CheckMortalityParams<T: Config>(Option<CheckMortalityParamsInner<T>>);
struct CheckMortalityParamsInner<T: Config> {
era: Era,
checkpoint: Option<T::Hash>,
}
impl<T: Config> Default for CheckMortalityParams<T> {
fn default() -> Self {
Self {
era: Default::default(),
checkpoint: Default::default(),
CheckMortalityParams(None)
}
}
impl<T: Config> RefineParams<T> for CheckMortalityParams<T> {
fn refine(&mut self, data: &RefineParamsData<T>) {
if self.0.is_none() {
// By default we refine the params to have a mortal transaction valid for 32 blocks.
const TX_VALID_FOR: u64 = 32;
*self =
CheckMortalityParams::mortal(TX_VALID_FOR, data.block_number(), data.block_hash());
}
}
}
@@ -172,32 +193,39 @@ impl<T: Config> CheckMortalityParams<T> {
/// `block_hash` should both point to the same block, and are the block that
/// the transaction is mortal from.
pub fn mortal(period: u64, block_number: u64, block_hash: T::Hash) -> Self {
CheckMortalityParams {
Self(Some(CheckMortalityParamsInner {
era: Era::mortal(period, block_number),
checkpoint: Some(block_hash),
}
}))
}
/// An immortal transaction.
pub fn immortal() -> Self {
CheckMortalityParams {
Self(Some(CheckMortalityParamsInner {
era: Era::Immortal,
checkpoint: None,
}
}))
}
}
impl<T: Config> ExtrinsicParams<T> for CheckMortality<T> {
type OtherParams = CheckMortalityParams<T>;
type Params = CheckMortalityParams<T>;
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
client: Client,
other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(CheckMortality {
era: other_params.era,
checkpoint: other_params.checkpoint.unwrap_or(client.genesis_hash()),
})
let check_mortality = if let Some(params) = params.0 {
CheckMortality {
era: params.era,
checkpoint: params.checkpoint.unwrap_or(client.genesis_hash()),
}
} else {
CheckMortality {
era: Era::Immortal,
checkpoint: client.genesis_hash(),
}
};
Ok(check_mortality)
}
}
@@ -278,20 +306,21 @@ impl<T: Config> ChargeAssetTxPaymentParams<T> {
}
impl<T: Config> ExtrinsicParams<T> for ChargeAssetTxPayment<T> {
type OtherParams = ChargeAssetTxPaymentParams<T>;
type Params = ChargeAssetTxPaymentParams<T>;
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
_client: Client,
other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(ChargeAssetTxPayment {
tip: Compact(other_params.tip),
asset_id: other_params.asset_id,
tip: Compact(params.tip),
asset_id: params.asset_id,
})
}
}
impl<T: Config> RefineParams<T> for ChargeAssetTxPaymentParams<T> {}
impl<T: Config> ExtrinsicParamsEncoder for ChargeAssetTxPayment<T> {
fn encode_extra_to(&self, v: &mut Vec<u8>) {
(self.tip, &self.asset_id).encode_to(v);
@@ -336,19 +365,20 @@ impl ChargeTransactionPaymentParams {
}
impl<T: Config> ExtrinsicParams<T> for ChargeTransactionPayment {
type OtherParams = ChargeTransactionPaymentParams;
type Params = ChargeTransactionPaymentParams;
fn new<Client: OfflineClientT<T>>(
_nonce: u64,
_client: Client,
other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
Ok(ChargeTransactionPayment {
tip: Compact(other_params.tip),
tip: Compact(params.tip),
})
}
}
impl<T: Config> RefineParams<T> for ChargeTransactionPaymentParams {}
impl ExtrinsicParamsEncoder for ChargeTransactionPayment {
fn encode_extra_to(&self, v: &mut Vec<u8>) {
self.tip.encode_to(v);
@@ -380,12 +410,11 @@ macro_rules! impl_tuples {
T: Config,
$($ident: SignedExtension<T>,)+
{
type OtherParams = ($($ident::OtherParams,)+);
type Params = ($($ident::Params,)+);
fn new<Client: OfflineClientT<T>>(
nonce: u64,
client: Client,
other_params: Self::OtherParams,
params: Self::Params,
) -> Result<Self, ExtrinsicParamsError> {
let metadata = client.metadata();
let types = metadata.types();
@@ -401,7 +430,7 @@ macro_rules! impl_tuples {
}
// Break and record as soon as we find a match:
if $ident::matches(e.identifier(), e.extra_ty(), types) {
let ext = $ident::new(nonce, client.clone(), other_params.$index)?;
let ext = $ident::new(client.clone(), params.$index)?;
let boxed_ext: Box<dyn ExtrinsicParamsEncoder> = Box::new(ext);
exts_by_index.insert(idx, boxed_ext);
break
subxt/src/config/substrate.rs
+1 -1
View File
@@ -42,7 +42,7 @@ pub struct BlakeTwo256;
impl Hasher for BlakeTwo256 {
type Output = H256;
fn hash(s: &[u8]) -> Self::Output {
sp_core_hashing::blake2_256(s).into()
sp_crypto_hashing::blake2_256(s).into()
}
}
subxt/src/dynamic.rs
+4 -4
View File
@@ -17,7 +17,7 @@ pub use scale_value::{At, Value};
/// regarding what type was used to decode each part of it. This implements
/// [`crate::metadata::DecodeWithMetadata`], and is used as a return type
/// for dynamic requests.
pub type DecodedValue = scale_value::Value<scale_value::scale::TypeId>;
pub type DecodedValue = scale_value::Value<u32>;
// Submit dynamic transactions.
pub use crate::tx::dynamic as tx;
@@ -68,9 +68,9 @@ impl DecodedValueThunk {
}
/// Decode the SCALE encoded storage entry into a dynamic [`DecodedValue`] type.
pub fn to_value(&self) -> Result<DecodedValue, Error> {
let val = DecodedValue::decode_as_type(
let val = scale_value::scale::decode_as_type(
&mut &*self.scale_bytes,
self.type_id,
&self.type_id,
self.metadata.types(),
)?;
Ok(val)
@@ -79,7 +79,7 @@ impl DecodedValueThunk {
pub fn as_type<T: DecodeAsType>(&self) -> Result<T, scale_decode::Error> {
T::decode_as_type(
&mut &self.scale_bytes[..],
self.type_id,
&self.type_id,
self.metadata.types(),
)
}
subxt/src/error/dispatch_error.rs
+14 -10
View File
@@ -7,8 +7,9 @@
use crate::metadata::{DecodeWithMetadata, Metadata};
use core::fmt::Debug;
use scale_decode::{visitor::DecodeAsTypeResult, DecodeAsType};
use std::borrow::Cow;
use scale_decode::{visitor::DecodeAsTypeResult, DecodeAsType, TypeResolver};
use std::{borrow::Cow, marker::PhantomData};
use super::{Error, MetadataError};
@@ -209,7 +210,7 @@ impl ModuleError {
pub fn as_root_error<E: DecodeAsType>(&self) -> Result<E, Error> {
let decoded = E::decode_as_type(
&mut &self.bytes[..],
self.metadata.outer_enums().error_enum_ty(),
&self.metadata.outer_enums().error_enum_ty(),
self.metadata.types(),
)?;
@@ -262,24 +263,27 @@ impl DispatchError {
// a legacy format of 2 bytes, or a newer format of 5 bytes. So, just grab the bytes
// out when decoding to manually work with them.
struct DecodedModuleErrorBytes(Vec<u8>);
struct DecodedModuleErrorBytesVisitor;
impl scale_decode::Visitor for DecodedModuleErrorBytesVisitor {
struct DecodedModuleErrorBytesVisitor<R: TypeResolver>(PhantomData<R>);
impl<R: TypeResolver> scale_decode::Visitor for DecodedModuleErrorBytesVisitor<R> {
type Error = scale_decode::Error;
type Value<'scale, 'info> = DecodedModuleErrorBytes;
type TypeResolver = R;
fn unchecked_decode_as_type<'scale, 'info>(
self,
input: &mut &'scale [u8],
_type_id: scale_decode::visitor::TypeId,
_types: &'info scale_info::PortableRegistry,
_type_id: &R::TypeId,
_types: &'info R,
) -> DecodeAsTypeResult<Self, Result<Self::Value<'scale, 'info>, Self::Error>>
{
DecodeAsTypeResult::Decoded(Ok(DecodedModuleErrorBytes(input.to_vec())))
}
}
impl scale_decode::IntoVisitor for DecodedModuleErrorBytes {
type Visitor = DecodedModuleErrorBytesVisitor;
fn into_visitor() -> Self::Visitor {
DecodedModuleErrorBytesVisitor
type AnyVisitor<R: TypeResolver> = DecodedModuleErrorBytesVisitor<R>;
fn into_visitor<R: TypeResolver>() -> DecodedModuleErrorBytesVisitor<R> {
DecodedModuleErrorBytesVisitor(PhantomData)
}
}
subxt/src/error/mod.rs
+27 -8
View File
@@ -7,13 +7,14 @@
mod dispatch_error;
crate::macros::cfg_unstable_light_client! {
pub use crate::client::LightClientError;
pub use subxt_lightclient::LightClientError;
}
// Re-export dispatch error types:
pub use dispatch_error::{
ArithmeticError, DispatchError, ModuleError, TokenError, TransactionalError,
};
use subxt_metadata::StorageHasher;
// Re-expose the errors we use from other crates here:
pub use crate::config::ExtrinsicParamsError;
@@ -98,6 +99,12 @@ impl From<std::convert::Infallible> for Error {
}
}
impl From<scale_decode::visitor::DecodeError> for Error {
fn from(value: scale_decode::visitor::DecodeError) -> Self {
Error::Decode(value.into())
}
}
impl Error {
/// Checks whether the error was caused by a RPC re-connection.
pub fn is_disconnected_will_reconnect(&self) -> bool {
@@ -187,14 +194,9 @@ pub enum TransactionError {
#[derive(Clone, Debug, thiserror::Error)]
#[non_exhaustive]
pub enum StorageAddressError {
/// Storage map type must be a composite type.
#[error("Storage map type must be a composite type")]
MapTypeMustBeTuple,
/// Storage lookup does not have the expected number of keys.
#[error("Storage lookup requires {expected} keys but got {actual} keys")]
WrongNumberOfKeys {
/// The actual number of keys needed, based on the metadata.
actual: usize,
#[error("Storage lookup requires {expected} keys but more keys have been provided.")]
TooManyKeys {
/// The number of keys provided in the storage address.
expected: usize,
},
@@ -206,6 +208,23 @@ pub enum StorageAddressError {
/// The number of fields in the metadata for this storage entry.
fields: usize,
},
/// We weren't given enough bytes to decode the storage address/key.
#[error("Not enough remaining bytes to decode the storage address/key")]
NotEnoughBytes,
/// We have leftover bytes after decoding the storage address.
#[error("We have leftover bytes after decoding the storage address")]
TooManyBytes,
/// The bytes of a storage address are not the expected address for decoding the storage keys of the address.
#[error("Storage address bytes are not the expected format. Addresses need to be at least 16 bytes (pallet ++ entry) and follow a structure given by the hashers defined in the metadata")]
UnexpectedAddressBytes,
/// An invalid hasher was used to reconstruct a value from a chunk of bytes that is part of a storage address. Hashers where the hash does not contain the original value are invalid for this purpose.
#[error("An invalid hasher was used to reconstruct a value with type ID {ty_id} from a hash formed by a {hasher:?} hasher. This is only possible for concat-style hashers or the identity hasher")]
HasherCannotReconstructKey {
/// Type id of the key's type.
ty_id: u32,
/// The invalid hasher that caused this error.
hasher: StorageHasher,
},
}
/// Something went wrong trying to access details in the metadata.
subxt/src/events/events_client.rs
+2 -2
View File
@@ -76,8 +76,8 @@ where
// The storage key needed to access events.
fn system_events_key() -> [u8; 32] {
let a = sp_core_hashing::twox_128(b"System");
let b = sp_core_hashing::twox_128(b"Events");
let a = sp_crypto_hashing::twox_128(b"System");
let b = sp_crypto_hashing::twox_128(b"Events");
let mut res = [0; 32];
res[0..16].clone_from_slice(&a);
res[16..32].clone_from_slice(&b);
subxt/src/events/events_type.rs
+8 -14
View File
@@ -228,9 +228,9 @@ impl<T: Config> EventDetails<T> {
// Skip over the bytes for this field:
scale_decode::visitor::decode_with_visitor(
input,
field_metadata.ty.id,
&field_metadata.ty.id,
metadata.types(),
scale_decode::visitor::IgnoreVisitor,
scale_decode::visitor::IgnoreVisitor::new(),
)
.map_err(scale_decode::Error::from)?;
}
@@ -321,9 +321,7 @@ impl<T: Config> EventDetails<T> {
/// Decode and provide the event fields back in the form of a [`scale_value::Composite`]
/// type which represents the named or unnamed fields that were present in the event.
pub fn field_values(
&self,
) -> Result<scale_value::Composite<scale_value::scale::TypeId>, Error> {
pub fn field_values(&self) -> Result<scale_value::Composite<u32>, Error> {
let bytes = &mut self.field_bytes();
let event_metadata = self.event_metadata();
@@ -331,14 +329,10 @@ impl<T: Config> EventDetails<T> {
.variant
.fields
.iter()
.map(|f| scale_decode::Field::new(f.ty.id, f.name.as_deref()));
.map(|f| scale_decode::Field::new(&f.ty.id, f.name.as_deref()));
use scale_decode::DecodeAsFields;
let decoded = <scale_value::Composite<scale_value::scale::TypeId>>::decode_as_fields(
bytes,
&mut fields,
self.metadata.types(),
)?;
let decoded =
scale_value::scale::decode_as_fields(bytes, &mut fields, self.metadata.types())?;
Ok(decoded)
}
@@ -352,7 +346,7 @@ impl<T: Config> EventDetails<T> {
.variant
.fields
.iter()
.map(|f| scale_decode::Field::new(f.ty.id, f.name.as_deref()));
.map(|f| scale_decode::Field::new(&f.ty.id, f.name.as_deref()));
let decoded =
E::decode_as_fields(&mut self.field_bytes(), &mut fields, self.metadata.types())?;
Ok(Some(decoded))
@@ -369,7 +363,7 @@ impl<T: Config> EventDetails<T> {
let decoded = E::decode_as_type(
&mut &bytes[..],
self.metadata.outer_enums().event_enum_ty(),
&self.metadata.outer_enums().event_enum_ty(),
self.metadata.types(),
)?;
subxt/src/lib.rs
+5
View File
@@ -61,6 +61,11 @@ pub mod utils;
#[macro_use]
mod macros;
// Expose light client bits
cfg_unstable_light_client! {
pub use subxt_lightclient as lightclient;
}
// Expose a few of the most common types at root,
// but leave most types behind their respective modules.
pub use crate::{
subxt/src/metadata/decode_encode_traits.rs
+2 -2
View File
@@ -21,7 +21,7 @@ impl<T: scale_decode::DecodeAsType> DecodeWithMetadata for T {
type_id: u32,
metadata: &Metadata,
) -> Result<T, Error> {
let val = T::decode_as_type(bytes, type_id, metadata.types())?;
let val = T::decode_as_type(bytes, &type_id, metadata.types())?;
Ok(val)
}
}
@@ -45,7 +45,7 @@ impl<T: scale_encode::EncodeAsType> EncodeWithMetadata for T {
metadata: &Metadata,
bytes: &mut Vec<u8>,
) -> Result<(), Error> {
self.encode_as_type_to(type_id, metadata.types(), bytes)?;
self.encode_as_type_to(&type_id, metadata.types(), bytes)?;
Ok(())
}
}
subxt/src/runtime_api/runtime_payload.rs
+1 -1
View File
@@ -103,7 +103,7 @@ impl<ArgsData: EncodeAsFields, ReturnTy: DecodeWithMetadata> RuntimeApiPayload
.ok_or_else(|| MetadataError::RuntimeMethodNotFound((*self.method_name).to_owned()))?;
let mut fields = api_method
.inputs()
.map(|input| scale_encode::Field::named(input.ty, &input.name));
.map(|input| scale_encode::Field::named(&input.ty, &input.name));
self.args_data
.encode_as_fields_to(&mut fields, metadata.types(), out)?;
subxt/src/storage/mod.rs
+7 -7
View File
@@ -6,23 +6,23 @@
mod storage_address;
mod storage_client;
mod storage_key;
mod storage_type;
pub mod utils;
mod utils;
pub use storage_client::StorageClient;
pub use storage_type::Storage;
pub use storage_type::{Storage, StorageKeyValuePair};
/// Types representing an address which describes where a storage
/// entry lives and how to properly decode it.
pub mod address {
pub use super::storage_address::{
dynamic, make_static_storage_map_key, Address, DynamicAddress, StaticStorageMapKey,
StorageAddress, Yes,
};
pub use super::storage_address::{dynamic, Address, DynamicAddress, StorageAddress, Yes};
pub use super::storage_key::{StaticStorageKey, StorageKey};
}
pub use storage_key::StorageKey;
// For consistency with other modules, also expose
// the basic address stuff at the root of the module.
pub use storage_address::{dynamic, Address, DynamicAddress, StorageAddress};
subxt/src/storage/storage_address.rs
+47 -136
View File
@@ -4,20 +4,22 @@
use crate::{
dynamic::DecodedValueThunk,
error::{Error, MetadataError, StorageAddressError},
metadata::{DecodeWithMetadata, EncodeWithMetadata, Metadata},
utils::{Encoded, Static},
error::{Error, MetadataError},
metadata::{DecodeWithMetadata, Metadata},
};
use derivative::Derivative;
use scale_info::TypeDef;
use std::borrow::Cow;
use subxt_metadata::{StorageEntryType, StorageHasher};
use super::{storage_key::StorageHashers, StorageKey};
/// This represents a storage address. Anything implementing this trait
/// can be used to fetch and iterate over storage entries.
pub trait StorageAddress {
/// The target type of the value that lives at this address.
type Target: DecodeWithMetadata;
/// The keys type used to construct this address.
type Keys: StorageKey;
/// Can an entry be fetched from this address?
/// Set this type to [`Yes`] to enable the corresponding calls to be made.
type IsFetchable;
@@ -54,64 +56,69 @@ pub struct Yes;
/// via the `subxt` macro) or dynamic values via [`dynamic`].
#[derive(Derivative)]
#[derivative(
Clone(bound = "StorageKey: Clone"),
Debug(bound = "StorageKey: std::fmt::Debug"),
Eq(bound = "StorageKey: std::cmp::Eq"),
Ord(bound = "StorageKey: std::cmp::Ord"),
PartialEq(bound = "StorageKey: std::cmp::PartialEq"),
PartialOrd(bound = "StorageKey: std::cmp::PartialOrd")
Clone(bound = "Keys: Clone"),
Debug(bound = "Keys: std::fmt::Debug"),
Eq(bound = "Keys: std::cmp::Eq"),
Ord(bound = "Keys: std::cmp::Ord"),
PartialEq(bound = "Keys: std::cmp::PartialEq"),
PartialOrd(bound = "Keys: std::cmp::PartialOrd")
)]
pub struct Address<StorageKey, ReturnTy, Fetchable, Defaultable, Iterable> {
pub struct Address<Keys: StorageKey, ReturnTy, Fetchable, Defaultable, Iterable> {
pallet_name: Cow<'static, str>,
entry_name: Cow<'static, str>,
storage_entry_keys: Vec<StorageKey>,
keys: Keys,
validation_hash: Option<[u8; 32]>,
_marker: std::marker::PhantomData<(ReturnTy, Fetchable, Defaultable, Iterable)>,
}
/// A typical storage address constructed at runtime rather than via the `subxt` macro; this
/// has no restriction on what it can be used for (since we don't statically know).
pub type DynamicAddress<StorageKey> = Address<StorageKey, DecodedValueThunk, Yes, Yes, Yes>;
pub type DynamicAddress<Keys> = Address<Keys, DecodedValueThunk, Yes, Yes, Yes>;
impl<StorageKey, ReturnTy, Fetchable, Defaultable, Iterable>
Address<StorageKey, ReturnTy, Fetchable, Defaultable, Iterable>
where
StorageKey: EncodeWithMetadata,
ReturnTy: DecodeWithMetadata,
{
/// Create a new [`Address`] to use to access a storage entry.
pub fn new(
pallet_name: impl Into<String>,
entry_name: impl Into<String>,
storage_entry_keys: Vec<StorageKey>,
) -> Self {
impl<Keys: StorageKey> DynamicAddress<Keys> {
/// Creates a new dynamic address. As `Keys` you can use a `Vec<scale_value::Value>`
pub fn new(pallet_name: impl Into<String>, entry_name: impl Into<String>, keys: Keys) -> Self {
Self {
pallet_name: Cow::Owned(pallet_name.into()),
entry_name: Cow::Owned(entry_name.into()),
storage_entry_keys: storage_entry_keys.into_iter().collect(),
keys,
validation_hash: None,
_marker: std::marker::PhantomData,
}
}
}
impl<Keys, ReturnTy, Fetchable, Defaultable, Iterable>
Address<Keys, ReturnTy, Fetchable, Defaultable, Iterable>
where
Keys: StorageKey,
ReturnTy: DecodeWithMetadata,
{
/// Create a new [`Address`] using static strings for the pallet and call name.
/// This is only expected to be used from codegen.
#[doc(hidden)]
pub fn new_static(
pallet_name: &'static str,
entry_name: &'static str,
storage_entry_keys: Vec<StorageKey>,
keys: Keys,
hash: [u8; 32],
) -> Self {
Self {
pallet_name: Cow::Borrowed(pallet_name),
entry_name: Cow::Borrowed(entry_name),
storage_entry_keys: storage_entry_keys.into_iter().collect(),
keys,
validation_hash: Some(hash),
_marker: std::marker::PhantomData,
}
}
}
impl<Keys, ReturnTy, Fetchable, Defaultable, Iterable>
Address<Keys, ReturnTy, Fetchable, Defaultable, Iterable>
where
Keys: StorageKey,
ReturnTy: DecodeWithMetadata,
{
/// Do not validate this storage entry prior to accessing it.
pub fn unvalidated(self) -> Self {
Self {
@@ -128,13 +135,14 @@ where
}
}
impl<StorageKey, ReturnTy, Fetchable, Defaultable, Iterable> StorageAddress
for Address<StorageKey, ReturnTy, Fetchable, Defaultable, Iterable>
impl<Keys, ReturnTy, Fetchable, Defaultable, Iterable> StorageAddress
for Address<Keys, ReturnTy, Fetchable, Defaultable, Iterable>
where
StorageKey: EncodeWithMetadata,
Keys: StorageKey,
ReturnTy: DecodeWithMetadata,
{
type Target = ReturnTy;
type Keys = Keys;
type IsFetchable = Fetchable;
type IsDefaultable = Defaultable;
type IsIterable = Iterable;
@@ -156,78 +164,10 @@ where
.entry_by_name(self.entry_name())
.ok_or_else(|| MetadataError::StorageEntryNotFound(self.entry_name().to_owned()))?;
match entry.entry_type() {
StorageEntryType::Plain(_) => {
if !self.storage_entry_keys.is_empty() {
Err(StorageAddressError::WrongNumberOfKeys {
expected: 0,
actual: self.storage_entry_keys.len(),
}
.into())
} else {
Ok(())
}
}
StorageEntryType::Map {
hashers, key_ty, ..
} => {
let ty = metadata
.types()
.resolve(*key_ty)
.ok_or(MetadataError::TypeNotFound(*key_ty))?;
// If the provided keys are empty, the storage address must be
// equal to the storage root address.
if self.storage_entry_keys.is_empty() {
return Ok(());
}
// If the key is a tuple, we encode each value to the corresponding tuple type.
// If the key is not a tuple, encode a single value to the key type.
let type_ids = match &ty.type_def {
TypeDef::Tuple(tuple) => {
either::Either::Left(tuple.fields.iter().map(|f| f.id))
}
_other => either::Either::Right(std::iter::once(*key_ty)),
};
if type_ids.len() < self.storage_entry_keys.len() {
// Provided more keys than fields.
return Err(StorageAddressError::WrongNumberOfKeys {
expected: type_ids.len(),
actual: self.storage_entry_keys.len(),
}
.into());
}
if hashers.len() == 1 {
// One hasher; hash a tuple of all SCALE encoded bytes with the one hash function.
let mut input = Vec::new();
let iter = self.storage_entry_keys.iter().zip(type_ids);
for (key, type_id) in iter {
key.encode_with_metadata(type_id, metadata, &mut input)?;
}
hash_bytes(&input, &hashers[0], bytes);
Ok(())
} else if hashers.len() >= type_ids.len() {
let iter = self.storage_entry_keys.iter().zip(type_ids).zip(hashers);
// A hasher per field; encode and hash each field independently.
for ((key, type_id), hasher) in iter {
let mut input = Vec::new();
key.encode_with_metadata(type_id, metadata, &mut input)?;
hash_bytes(&input, hasher, bytes);
}
Ok(())
} else {
// Provided more fields than hashers.
Err(StorageAddressError::WrongNumberOfHashers {
hashers: hashers.len(),
fields: type_ids.len(),
}
.into())
}
}
}
let hashers = StorageHashers::new(entry.entry_type(), metadata.types())?;
self.keys
.encode_storage_key(bytes, &mut hashers.iter(), metadata.types())?;
Ok(())
}
fn validation_hash(&self) -> Option<[u8; 32]> {
@@ -235,40 +175,11 @@ where
}
}
/// A static storage key; this is some pre-encoded bytes
/// likely provided by the generated interface.
pub type StaticStorageMapKey = Static<Encoded>;
// Used in codegen to construct the above.
#[doc(hidden)]
pub fn make_static_storage_map_key<T: codec::Encode>(t: T) -> StaticStorageMapKey {
Static(Encoded(t.encode()))
}
/// Construct a new dynamic storage lookup.
pub fn dynamic<StorageKey: EncodeWithMetadata>(
pub fn dynamic<Keys: StorageKey>(
pallet_name: impl Into<String>,
entry_name: impl Into<String>,
storage_entry_keys: Vec<StorageKey>,
) -> DynamicAddress<StorageKey> {
storage_entry_keys: Keys,
) -> DynamicAddress<Keys> {
DynamicAddress::new(pallet_name, entry_name, storage_entry_keys)
}
/// Take some SCALE encoded bytes and a [`StorageHasher`] and hash the bytes accordingly.
fn hash_bytes(input: &[u8], hasher: &StorageHasher, bytes: &mut Vec<u8>) {
match hasher {
StorageHasher::Identity => bytes.extend(input),
StorageHasher::Blake2_128 => bytes.extend(sp_core_hashing::blake2_128(input)),
StorageHasher::Blake2_128Concat => {
bytes.extend(sp_core_hashing::blake2_128(input));
bytes.extend(input);
}
StorageHasher::Blake2_256 => bytes.extend(sp_core_hashing::blake2_256(input)),
StorageHasher::Twox128 => bytes.extend(sp_core_hashing::twox_128(input)),
StorageHasher::Twox256 => bytes.extend(sp_core_hashing::twox_256(input)),
StorageHasher::Twox64Concat => {
bytes.extend(sp_core_hashing::twox_64(input));
bytes.extend(input);
}
}
}
subxt/src/storage/storage_key.rs
+472
View File
@@ -0,0 +1,472 @@
use crate::{
error::{Error, MetadataError, StorageAddressError},
utils::{Encoded, Static},
};
use scale_decode::visitor::IgnoreVisitor;
use scale_encode::EncodeAsType;
use scale_info::{PortableRegistry, TypeDef};
use scale_value::Value;
use subxt_metadata::{StorageEntryType, StorageHasher};
use derivative::Derivative;
use super::utils::hash_bytes;
/// A collection of storage hashers paired with the type ids of the types they should hash.
/// Can be created for each storage entry in the metadata via [`StorageHashers::new()`].
#[derive(Debug)]
pub(crate) struct StorageHashers {
hashers_and_ty_ids: Vec<(StorageHasher, u32)>,
}
impl StorageHashers {
/// Creates new [`StorageHashers`] from a storage entry. Looks at the [`StorageEntryType`] and
/// assigns a hasher to each type id that makes up the key.
pub fn new(storage_entry: &StorageEntryType, types: &PortableRegistry) -> Result<Self, Error> {
let mut hashers_and_ty_ids = vec![];
if let StorageEntryType::Map {
hashers, key_ty, ..
} = storage_entry
{
let ty = types
.resolve(*key_ty)
.ok_or(MetadataError::TypeNotFound(*key_ty))?;
if let TypeDef::Tuple(tuple) = &ty.type_def {
if hashers.len() == 1 {
// use the same hasher for all fields, if only 1 hasher present:
let hasher = hashers[0];
for f in tuple.fields.iter() {
hashers_and_ty_ids.push((hasher, f.id));
}
} else if hashers.len() < tuple.fields.len() {
return Err(StorageAddressError::WrongNumberOfHashers {
hashers: hashers.len(),
fields: tuple.fields.len(),
}
.into());
} else {
for (i, f) in tuple.fields.iter().enumerate() {
hashers_and_ty_ids.push((hashers[i], f.id));
}
}
} else {
if hashers.len() != 1 {
return Err(StorageAddressError::WrongNumberOfHashers {
hashers: hashers.len(),
fields: 1,
}
.into());
}
hashers_and_ty_ids.push((hashers[0], *key_ty));
};
}
Ok(Self { hashers_and_ty_ids })
}
/// Creates an iterator over the storage hashers and type ids.
pub fn iter(&self) -> StorageHashersIter<'_> {
StorageHashersIter {
hashers: self,
idx: 0,
}
}
}
/// An iterator over all type ids of the key and the respective hashers.
/// See [`StorageHashers::iter()`].
#[derive(Debug)]
pub struct StorageHashersIter<'a> {
hashers: &'a StorageHashers,
idx: usize,
}
impl<'a> StorageHashersIter<'a> {
fn next_or_err(&mut self) -> Result<(StorageHasher, u32), Error> {
self.next().ok_or_else(|| {
StorageAddressError::TooManyKeys {
expected: self.hashers.hashers_and_ty_ids.len(),
}
.into()
})
}
}
impl<'a> Iterator for StorageHashersIter<'a> {
type Item = (StorageHasher, u32);
fn next(&mut self) -> Option<Self::Item> {
let item = self.hashers.hashers_and_ty_ids.get(self.idx).copied()?;
self.idx += 1;
Some(item)
}
}
impl<'a> ExactSizeIterator for StorageHashersIter<'a> {
fn len(&self) -> usize {
self.hashers.hashers_and_ty_ids.len() - self.idx
}
}
/// This trait should be implemented by anything that can be used as one or multiple storage keys.
pub trait StorageKey {
/// Encodes the storage key into some bytes
fn encode_storage_key(
&self,
bytes: &mut Vec<u8>,
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<(), Error>;
/// Attempts to decode the StorageKey given some bytes and a set of hashers and type IDs that they are meant to represent.
/// The bytes passed to `decode` should start with:
/// - 1. some fixed size hash (for all hashers except `Identity`)
/// - 2. the plain key value itself (for `Identity`, `Blake2_128Concat` and `Twox64Concat` hashers)
fn decode_storage_key(
bytes: &mut &[u8],
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<Self, Error>
where
Self: Sized + 'static;
}
/// Implement `StorageKey` for `()` which can be used for keyless storage entries,
/// or to otherwise just ignore some entry.
impl StorageKey for () {
fn encode_storage_key(
&self,
_bytes: &mut Vec<u8>,
hashers: &mut StorageHashersIter,
_types: &PortableRegistry,
) -> Result<(), Error> {
_ = hashers.next_or_err();
Ok(())
}
fn decode_storage_key(
bytes: &mut &[u8],
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<Self, Error> {
let (hasher, ty_id) = match hashers.next_or_err() {
Ok((hasher, ty_id)) => (hasher, ty_id),
Err(_) if bytes.is_empty() => return Ok(()),
Err(err) => return Err(err),
};
consume_hash_returning_key_bytes(bytes, hasher, ty_id, types)?;
Ok(())
}
}
/// A storage key for static encoded values.
/// The original value is only present at construction, but can be decoded from the contained bytes.
#[derive(Derivative)]
#[derivative(Clone(bound = ""), Debug(bound = ""))]
pub struct StaticStorageKey<K: ?Sized> {
bytes: Static<Encoded>,
_marker: std::marker::PhantomData<K>,
}
impl<K: codec::Encode + ?Sized> StaticStorageKey<K> {
/// Creates a new static storage key
pub fn new(key: &K) -> Self {
StaticStorageKey {
bytes: Static(Encoded(key.encode())),
_marker: std::marker::PhantomData,
}
}
}
impl<K: codec::Decode + ?Sized> StaticStorageKey<K> {
/// Decodes the encoded inner bytes into the type `K`.
pub fn decoded(&self) -> Result<K, Error> {
let decoded = K::decode(&mut self.bytes())?;
Ok(decoded)
}
}
impl<K: ?Sized> StaticStorageKey<K> {
/// Returns the scale-encoded bytes that make up this key
pub fn bytes(&self) -> &[u8] {
&self.bytes.0 .0
}
}
// Note: The ?Sized bound is necessary to support e.g. `StorageKey<[u8]>`.
impl<K: ?Sized> StorageKey for StaticStorageKey<K> {
fn encode_storage_key(
&self,
bytes: &mut Vec<u8>,
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<(), Error> {
let (hasher, ty_id) = hashers.next_or_err()?;
let encoded_value = self.bytes.encode_as_type(&ty_id, types)?;
hash_bytes(&encoded_value, hasher, bytes);
Ok(())
}
fn decode_storage_key(
bytes: &mut &[u8],
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<Self, Error>
where
Self: Sized + 'static,
{
let (hasher, ty_id) = hashers.next_or_err()?;
let key_bytes = consume_hash_returning_key_bytes(bytes, hasher, ty_id, types)?;
// if the hasher had no key appended, we can't decode it into a `StaticStorageKey`.
let Some(key_bytes) = key_bytes else {
return Err(StorageAddressError::HasherCannotReconstructKey { ty_id, hasher }.into());
};
// Return the key bytes.
let key = StaticStorageKey {
bytes: Static(Encoded(key_bytes.to_vec())),
_marker: std::marker::PhantomData::<K>,
};
Ok(key)
}
}
impl StorageKey for Vec<scale_value::Value> {
fn encode_storage_key(
&self,
bytes: &mut Vec<u8>,
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<(), Error> {
for value in self.iter() {
let (hasher, ty_id) = hashers.next_or_err()?;
let encoded_value = value.encode_as_type(&ty_id, types)?;
hash_bytes(&encoded_value, hasher, bytes);
}
Ok(())
}
fn decode_storage_key(
bytes: &mut &[u8],
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<Self, Error>
where
Self: Sized + 'static,
{
let mut result: Vec<scale_value::Value> = vec![];
for (hasher, ty_id) in hashers.by_ref() {
match consume_hash_returning_key_bytes(bytes, hasher, ty_id, types)? {
Some(value_bytes) => {
let value =
scale_value::scale::decode_as_type(&mut &*value_bytes, &ty_id, types)?;
result.push(value.remove_context());
}
None => {
result.push(Value::unnamed_composite([]));
}
}
}
// We've consumed all of the hashers, so we expect to also consume all of the bytes:
if !bytes.is_empty() {
return Err(StorageAddressError::TooManyBytes.into());
}
Ok(result)
}
}
// Skip over the hash bytes (including any key at the end), returning bytes
// representing the key if one exists, or None if the hasher has no key appended.
fn consume_hash_returning_key_bytes<'a>(
bytes: &mut &'a [u8],
hasher: StorageHasher,
ty_id: u32,
types: &PortableRegistry,
) -> Result<Option<&'a [u8]>, Error> {
// Strip the bytes off for the actual hash, consuming them.
let bytes_to_strip = hasher.len_excluding_key();
if bytes.len() < bytes_to_strip {
return Err(StorageAddressError::NotEnoughBytes.into());
}
*bytes = &bytes[bytes_to_strip..];
// Now, find the bytes representing the key, consuming them.
let before_key = *bytes;
if hasher.ends_with_key() {
scale_decode::visitor::decode_with_visitor(
bytes,
&ty_id,
types,
IgnoreVisitor::<PortableRegistry>::new(),
)
.map_err(|err| Error::Decode(err.into()))?;
// Return the key bytes, having advanced the input cursor past them.
let key_bytes = &before_key[..before_key.len() - bytes.len()];
Ok(Some(key_bytes))
} else {
// There are no key bytes, so return None.
Ok(None)
}
}
/// Generates StorageKey implementations for tuples
macro_rules! impl_tuples {
($($ty:ident $n:tt),+) => {{
impl<$($ty: StorageKey),+> StorageKey for ($( $ty ),+) {
fn encode_storage_key(
&self,
bytes: &mut Vec<u8>,
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<(), Error> {
$( self.$n.encode_storage_key(bytes, hashers, types)?; )+
Ok(())
}
fn decode_storage_key(
bytes: &mut &[u8],
hashers: &mut StorageHashersIter,
types: &PortableRegistry,
) -> Result<Self, Error>
where
Self: Sized + 'static,
{
Ok( ( $( $ty::decode_storage_key(bytes, hashers, types)?, )+ ) )
}
}
}};
}
#[rustfmt::skip]
const _: () = {
impl_tuples!(A 0, B 1);
impl_tuples!(A 0, B 1, C 2);
impl_tuples!(A 0, B 1, C 2, D 3);
impl_tuples!(A 0, B 1, C 2, D 3, E 4);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6);
impl_tuples!(A 0, B 1, C 2, D 3, E 4, F 5, G 6, H 7);
};
#[cfg(test)]
mod tests {
use codec::Encode;
use scale_info::{meta_type, PortableRegistry, Registry, TypeInfo};
use subxt_metadata::StorageHasher;
use crate::utils::Era;
use super::{StaticStorageKey, StorageKey};
struct KeyBuilder {
registry: Registry,
bytes: Vec<u8>,
hashers_and_ty_ids: Vec<(StorageHasher, u32)>,
}
impl KeyBuilder {
fn new() -> KeyBuilder {
KeyBuilder {
registry: Registry::new(),
bytes: vec![],
hashers_and_ty_ids: vec![],
}
}
fn add<T: TypeInfo + Encode + 'static>(mut self, value: T, hasher: StorageHasher) -> Self {
let id = self.registry.register_type(&meta_type::<T>()).id;
self.hashers_and_ty_ids.push((hasher, id));
for _i in 0..hasher.len_excluding_key() {
self.bytes.push(0);
}
value.encode_to(&mut self.bytes);
self
}
fn build(self) -> (PortableRegistry, Vec<u8>, Vec<(StorageHasher, u32)>) {
(self.registry.into(), self.bytes, self.hashers_and_ty_ids)
}
}
#[test]
fn storage_key_decoding_fuzz() {
let hashers = [
StorageHasher::Blake2_128,
StorageHasher::Blake2_128Concat,
StorageHasher::Blake2_256,
StorageHasher::Identity,
StorageHasher::Twox128,
StorageHasher::Twox256,
StorageHasher::Twox64Concat,
];
let key_preserving_hashers = [
StorageHasher::Blake2_128Concat,
StorageHasher::Identity,
StorageHasher::Twox64Concat,
];
type T4A = (
(),
StaticStorageKey<u32>,
StaticStorageKey<String>,
StaticStorageKey<Era>,
);
type T4B = (
(),
(StaticStorageKey<u32>, StaticStorageKey<String>),
StaticStorageKey<Era>,
);
type T4C = (
((), StaticStorageKey<u32>),
(StaticStorageKey<String>, StaticStorageKey<Era>),
);
let era = Era::Immortal;
for h0 in hashers {
for h1 in key_preserving_hashers {
for h2 in key_preserving_hashers {
for h3 in key_preserving_hashers {
let (types, bytes, hashers_and_ty_ids) = KeyBuilder::new()
.add((), h0)
.add(13u32, h1)
.add("Hello", h2)
.add(era, h3)
.build();
let hashers = super::StorageHashers { hashers_and_ty_ids };
let keys_a =
T4A::decode_storage_key(&mut &bytes[..], &mut hashers.iter(), &types)
.unwrap();
let keys_b =
T4B::decode_storage_key(&mut &bytes[..], &mut hashers.iter(), &types)
.unwrap();
let keys_c =
T4C::decode_storage_key(&mut &bytes[..], &mut hashers.iter(), &types)
.unwrap();
assert_eq!(keys_a.1.decoded().unwrap(), 13);
assert_eq!(keys_b.1 .0.decoded().unwrap(), 13);
assert_eq!(keys_c.0 .1.decoded().unwrap(), 13);
assert_eq!(keys_a.2.decoded().unwrap(), "Hello");
assert_eq!(keys_b.1 .1.decoded().unwrap(), "Hello");
assert_eq!(keys_c.1 .0.decoded().unwrap(), "Hello");
assert_eq!(keys_a.3.decoded().unwrap(), era);
assert_eq!(keys_b.2.decoded().unwrap(), era);
assert_eq!(keys_c.1 .1.decoded().unwrap(), era);
}
}
}
}
}
}
subxt/src/storage/storage_type.rs
+57 -11
View File
@@ -3,18 +3,22 @@
// see LICENSE for license details.
use super::storage_address::{StorageAddress, Yes};
use super::storage_key::StorageHashers;
use super::StorageKey;
use crate::{
backend::{BackendExt, BlockRef},
client::OnlineClientT,
error::{Error, MetadataError},
error::{Error, MetadataError, StorageAddressError},
metadata::{DecodeWithMetadata, Metadata},
Config,
};
use codec::Decode;
use derivative::Derivative;
use futures::StreamExt;
use std::{future::Future, marker::PhantomData};
use subxt_metadata::{PalletMetadata, StorageEntryMetadata, StorageEntryType};
/// This is returned from a couple of storage functions.
@@ -197,18 +201,19 @@ where
/// .await
/// .unwrap();
///
/// while let Some(Ok((key, value))) = iter.next().await {
/// println!("Key: 0x{}", hex::encode(&key));
/// println!("Value: {}", value);
/// while let Some(Ok(kv)) = iter.next().await {
/// println!("Key bytes: 0x{}", hex::encode(&kv.key_bytes));
/// println!("Value: {}", kv.value);
/// }
/// # }
/// ```
pub fn iter<Address>(
&self,
address: Address,
) -> impl Future<Output = Result<StreamOfResults<(Vec<u8>, Address::Target)>, Error>> + 'static
) -> impl Future<Output = Result<StreamOfResults<StorageKeyValuePair<Address>>, Error>> + 'static
where
Address: StorageAddress<IsIterable = Yes> + 'static,
Address::Keys: 'static + Sized,
{
let client = self.client.clone();
let block_ref = self.block_ref.clone();
@@ -226,11 +231,13 @@ where
// Look up the return type for flexible decoding. Do this once here to avoid
// potentially doing it every iteration if we used `decode_storage_with_metadata`
// in the iterator.
let return_type_id = return_type_from_storage_entry_type(entry.entry_type());
let entry = entry.entry_type();
let return_type_id = entry.value_ty();
let hashers = StorageHashers::new(entry, metadata.types())?;
// The address bytes of this entry:
let address_bytes = super::utils::storage_address_bytes(&address, &metadata)?;
let s = client
.backend()
.storage_fetch_descendant_values(address_bytes, block_ref.hash())
@@ -240,12 +247,27 @@ where
Ok(kv) => kv,
Err(e) => return Err(e),
};
let val = Address::Target::decode_with_metadata(
let value = Address::Target::decode_with_metadata(
&mut &*kv.value,
return_type_id,
&metadata,
)?;
Ok((kv.key, val))
let key_bytes = kv.key;
let cursor = &mut &key_bytes[..];
strip_storage_addess_root_bytes(cursor)?;
let keys = <Address::Keys as StorageKey>::decode_storage_key(
cursor,
&mut hashers.iter(),
metadata.types(),
)?;
Ok(StorageKeyValuePair::<Address> {
keys,
key_bytes,
value,
})
});
let s = StreamOfResults::new(Box::pin(s));
@@ -265,8 +287,10 @@ where
// construct the storage key. This is done similarly in `frame_support::traits::metadata::StorageVersion::storage_key()`.
pub const STORAGE_VERSION_STORAGE_KEY_POSTFIX: &[u8] = b":__STORAGE_VERSION__:";
let mut key_bytes: Vec<u8> = vec![];
key_bytes.extend(&sp_core_hashing::twox_128(pallet_name.as_ref().as_bytes()));
key_bytes.extend(&sp_core_hashing::twox_128(
key_bytes.extend(&sp_crypto_hashing::twox_128(
pallet_name.as_ref().as_bytes(),
));
key_bytes.extend(&sp_crypto_hashing::twox_128(
STORAGE_VERSION_STORAGE_KEY_POSTFIX,
));
@@ -290,6 +314,28 @@ where
}
}
/// Strips the first 16 bytes (8 for the pallet hash, 8 for the entry hash) off some storage address bytes.
fn strip_storage_addess_root_bytes(address_bytes: &mut &[u8]) -> Result<(), StorageAddressError> {
if address_bytes.len() >= 16 {
*address_bytes = &address_bytes[16..];
Ok(())
} else {
Err(StorageAddressError::UnexpectedAddressBytes)
}
}
/// A pair of keys and values together with all the bytes that make up the storage address.
/// `keys` is `None` if non-concat hashers are used. In this case the keys could not be extracted back from the key_bytes.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd)]
pub struct StorageKeyValuePair<T: StorageAddress> {
/// The bytes that make up the address of the storage entry.
pub key_bytes: Vec<u8>,
/// The keys that can be used to construct the address of this storage entry.
pub keys: T::Keys,
/// The value of the storage entry.
pub value: T::Target,
}
/// Validate a storage address against the metadata.
pub(crate) fn validate_storage_address<Address: StorageAddress>(
address: &Address,
subxt/src/storage/utils.rs
+26 -5
View File
@@ -6,22 +6,24 @@
//! aren't things that should ever be overridden, and so don't exist on
//! the trait itself.
use subxt_metadata::StorageHasher;
use super::StorageAddress;
use crate::{error::Error, metadata::Metadata};
/// Return the root of a given [`StorageAddress`]: hash the pallet name and entry name
/// and append those bytes to the output.
pub(crate) fn write_storage_address_root_bytes<Address: StorageAddress>(
pub fn write_storage_address_root_bytes<Address: StorageAddress>(
addr: &Address,
out: &mut Vec<u8>,
) {
out.extend(sp_core_hashing::twox_128(addr.pallet_name().as_bytes()));
out.extend(sp_core_hashing::twox_128(addr.entry_name().as_bytes()));
out.extend(sp_crypto_hashing::twox_128(addr.pallet_name().as_bytes()));
out.extend(sp_crypto_hashing::twox_128(addr.entry_name().as_bytes()));
}
/// Outputs the [`storage_address_root_bytes`] as well as any additional bytes that represent
/// a lookup in a storage map at that location.
pub(crate) fn storage_address_bytes<Address: StorageAddress>(
pub fn storage_address_bytes<Address: StorageAddress>(
addr: &Address,
metadata: &Metadata,
) -> Result<Vec<u8>, Error> {
@@ -32,8 +34,27 @@ pub(crate) fn storage_address_bytes<Address: StorageAddress>(
}
/// Outputs a vector containing the bytes written by [`write_storage_address_root_bytes`].
pub(crate) fn storage_address_root_bytes<Address: StorageAddress>(addr: &Address) -> Vec<u8> {
pub fn storage_address_root_bytes<Address: StorageAddress>(addr: &Address) -> Vec<u8> {
let mut bytes = Vec::new();
write_storage_address_root_bytes(addr, &mut bytes);
bytes
}
/// Take some SCALE encoded bytes and a [`StorageHasher`] and hash the bytes accordingly.
pub fn hash_bytes(input: &[u8], hasher: StorageHasher, bytes: &mut Vec<u8>) {
match hasher {
StorageHasher::Identity => bytes.extend(input),
StorageHasher::Blake2_128 => bytes.extend(sp_crypto_hashing::blake2_128(input)),
StorageHasher::Blake2_128Concat => {
bytes.extend(sp_crypto_hashing::blake2_128(input));
bytes.extend(input);
}
StorageHasher::Blake2_256 => bytes.extend(sp_crypto_hashing::blake2_256(input)),
StorageHasher::Twox128 => bytes.extend(sp_crypto_hashing::twox_128(input)),
StorageHasher::Twox256 => bytes.extend(sp_crypto_hashing::twox_256(input)),
StorageHasher::Twox64Concat => {
bytes.extend(sp_crypto_hashing::twox_64(input));
bytes.extend(input);
}
}
}
subxt/src/tx/mod.rs
+1 -1
View File
@@ -28,6 +28,6 @@ pub use self::{
PartialExtrinsic, SubmittableExtrinsic, TransactionInvalid, TransactionUnknown, TxClient,
ValidationResult,
},
tx_payload::{dynamic, BoxedPayload, DynamicPayload, Payload, TxPayload},
tx_payload::{dynamic, DynamicPayload, Payload, TxPayload},
tx_progress::{TxInBlock, TxProgress, TxStatus},
};
subxt/src/tx/tx_client.rs
+67 -28
View File
@@ -7,14 +7,17 @@ use std::borrow::Cow;
use crate::{
backend::{BackendExt, BlockRef, TransactionStatus},
client::{OfflineClientT, OnlineClientT},
config::{Config, ExtrinsicParams, ExtrinsicParamsEncoder, Hasher},
error::{Error, MetadataError},
config::{
Config, ExtrinsicParams, ExtrinsicParamsEncoder, Hasher, Header, RefineParams,
RefineParamsData,
},
error::{BlockError, Error, MetadataError},
tx::{Signer as SignerT, TxPayload, TxProgress},
utils::{Encoded, PhantomDataSendSync},
};
use codec::{Compact, Decode, Encode};
use derivative::Derivative;
use sp_core_hashing::blake2_256;
use sp_crypto_hashing::blake2_256;
/// A client for working with transactions.
#[derive(Derivative)]
@@ -103,11 +106,13 @@ impl<T: Config, C: OfflineClientT<T>> TxClient<T, C> {
}
/// Create a partial extrinsic.
pub fn create_partial_signed_with_nonce<Call>(
///
/// Note: if not provided, the default account nonce will be set to 0 and the default mortality will be _immortal_.
/// This is because this method runs offline, and so is unable to fetch the data needed for more appropriate values.
pub fn create_partial_signed_offline<Call>(
&self,
call: &Call,
account_nonce: u64,
other_params: <T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams,
params: <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<PartialExtrinsic<T, C>, Error>
where
Call: TxPayload,
@@ -120,11 +125,8 @@ impl<T: Config, C: OfflineClientT<T>> TxClient<T, C> {
let call_data = self.call_data(call)?;
// 3. Construct our custom additional/extra params.
let additional_and_extra_params = <T::ExtrinsicParams as ExtrinsicParams<T>>::new(
account_nonce,
self.client.clone(),
other_params,
)?;
let additional_and_extra_params =
<T::ExtrinsicParams as ExtrinsicParams<T>>::new(self.client.clone(), params)?;
// Return these details, ready to construct a signed extrinsic from.
Ok(PartialExtrinsic {
@@ -135,12 +137,14 @@ impl<T: Config, C: OfflineClientT<T>> TxClient<T, C> {
}
/// Creates a signed extrinsic without submitting it.
pub fn create_signed_with_nonce<Call, Signer>(
///
/// Note: if not provided, the default account nonce will be set to 0 and the default mortality will be _immortal_.
/// This is because this method runs offline, and so is unable to fetch the data needed for more appropriate values.
pub fn create_signed_offline<Call, Signer>(
&self,
call: &Call,
signer: &Signer,
account_nonce: u64,
other_params: <T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams,
params: <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<SubmittableExtrinsic<T, C>, Error>
where
Call: TxPayload,
@@ -152,8 +156,7 @@ impl<T: Config, C: OfflineClientT<T>> TxClient<T, C> {
// 2. Gather the "additional" and "extra" params along with the encoded call data,
// ready to be signed.
let partial_signed =
self.create_partial_signed_with_nonce(call, account_nonce, other_params)?;
let partial_signed = self.create_partial_signed_offline(call, params)?;
// 3. Sign and construct an extrinsic from these details.
Ok(partial_signed.sign(signer))
@@ -165,6 +168,30 @@ where
T: Config,
C: OnlineClientT<T>,
{
/// Fetch the latest block header and account nonce from the backend and use them to refine [`ExtrinsicParams::Params`].
async fn refine_params(
&self,
account_id: &T::AccountId,
params: &mut <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<(), Error> {
let block_ref = self.client.backend().latest_finalized_block_ref().await?;
let block_header = self
.client
.backend()
.block_header(block_ref.hash())
.await?
.ok_or_else(|| Error::Block(BlockError::not_found(block_ref.hash())))?;
let account_nonce =
crate::blocks::get_account_nonce(&self.client, account_id, block_ref.hash()).await?;
params.refine(&RefineParamsData::new(
account_nonce,
block_header.number().into(),
block_header.hash(),
));
Ok(())
}
/// Get the account nonce for a given account ID.
pub async fn account_nonce(&self, account_id: &T::AccountId) -> Result<u64, Error> {
let block_ref = self.client.backend().latest_finalized_block_ref().await?;
@@ -176,13 +203,15 @@ where
&self,
call: &Call,
account_id: &T::AccountId,
other_params: <T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams,
mut params: <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<PartialExtrinsic<T, C>, Error>
where
Call: TxPayload,
{
let account_nonce = self.account_nonce(account_id).await?;
self.create_partial_signed_with_nonce(call, account_nonce, other_params)
// Refine the params by adding account nonce and latest block information:
self.refine_params(account_id, &mut params).await?;
// Create the partial extrinsic with the refined params:
self.create_partial_signed_offline(call, params)
}
/// Creates a signed extrinsic, without submitting it.
@@ -190,14 +219,24 @@ where
&self,
call: &Call,
signer: &Signer,
other_params: <T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams,
params: <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<SubmittableExtrinsic<T, C>, Error>
where
Call: TxPayload,
Signer: SignerT<T>,
{
let account_nonce = self.account_nonce(&signer.account_id()).await?;
self.create_signed_with_nonce(call, signer, account_nonce, other_params)
// 1. Validate this call against the current node metadata if the call comes
// with a hash allowing us to do so.
self.validate(call)?;
// 2. Gather the "additional" and "extra" params along with the encoded call data,
// ready to be signed.
let partial_signed = self
.create_partial_signed(call, &signer.account_id(), params)
.await?;
// 3. Sign and construct an extrinsic from these details.
Ok(partial_signed.sign(signer))
}
/// Creates and signs an extrinsic and submits it to the chain. Passes default parameters
@@ -213,7 +252,7 @@ where
where
Call: TxPayload,
Signer: SignerT<T>,
<T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams: Default,
<T::ExtrinsicParams as ExtrinsicParams<T>>::Params: Default,
{
self.sign_and_submit_then_watch(call, signer, Default::default())
.await
@@ -227,13 +266,13 @@ where
&self,
call: &Call,
signer: &Signer,
other_params: <T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams,
params: <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<TxProgress<T, C>, Error>
where
Call: TxPayload,
Signer: SignerT<T>,
{
self.create_signed(call, signer, other_params)
self.create_signed(call, signer, params)
.await?
.submit_and_watch()
.await
@@ -257,7 +296,7 @@ where
where
Call: TxPayload,
Signer: SignerT<T>,
<T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams: Default,
<T::ExtrinsicParams as ExtrinsicParams<T>>::Params: Default,
{
self.sign_and_submit(call, signer, Default::default()).await
}
@@ -274,13 +313,13 @@ where
&self,
call: &Call,
signer: &Signer,
other_params: <T::ExtrinsicParams as ExtrinsicParams<T>>::OtherParams,
params: <T::ExtrinsicParams as ExtrinsicParams<T>>::Params,
) -> Result<T::Hash, Error>
where
Call: TxPayload,
Signer: SignerT<T>,
{
self.create_signed(call, signer, other_params)
self.create_signed(call, signer, params)
.await?
.submit()
.await
subxt/src/tx/tx_payload.rs
+2 -19
View File
@@ -14,7 +14,7 @@ use codec::Encode;
use derivative::Derivative;
use scale_encode::EncodeAsFields;
use scale_value::{Composite, ValueDef, Variant};
use std::{borrow::Cow, sync::Arc};
use std::borrow::Cow;
/// This represents a transaction payload that can be submitted
/// to a node.
@@ -65,10 +65,6 @@ pub struct Payload<CallData> {
validation_hash: Option<[u8; 32]>,
}
/// A boxed transaction payload.
// Dev Note: Arc used to enable easy cloning (given that we can't have dyn Clone).
pub type BoxedPayload = Payload<Arc<dyn EncodeAsFields + Send + Sync + 'static>>;
/// The type of a payload typically used for dynamic transaction payloads.
pub type DynamicPayload = Payload<Composite<()>>;
@@ -104,19 +100,6 @@ impl<CallData> Payload<CallData> {
}
}
/// Box the payload.
pub fn boxed(self) -> BoxedPayload
where
CallData: EncodeAsFields + Send + Sync + 'static,
{
BoxedPayload {
pallet_name: self.pallet_name,
call_name: self.call_name,
call_data: Arc::new(self.call_data),
validation_hash: self.validation_hash,
}
}
/// Do not validate this call prior to submitting it.
pub fn unvalidated(self) -> Self {
Self {
@@ -174,7 +157,7 @@ impl<CallData: EncodeAsFields> TxPayload for Payload<CallData> {
let mut fields = call
.fields
.iter()
.map(|f| scale_encode::Field::new(f.ty.id, f.name.as_deref()));
.map(|f| scale_encode::Field::new(&f.ty.id, f.name.as_deref()));
self.call_data
.encode_as_fields_to(&mut fields, metadata.types(), out)?;
subxt/src/utils/bits.rs
+19 -20
View File
@@ -9,7 +9,8 @@ use scale_bits::{
scale::format::{Format, OrderFormat, StoreFormat},
Bits,
};
use scale_decode::IntoVisitor;
use scale_decode::{IntoVisitor, TypeResolver};
use std::marker::PhantomData;
/// Associates `bitvec::store::BitStore` trait with corresponding, type-erased `scale_bits::StoreFormat` enum.
@@ -144,45 +145,43 @@ impl<Store: BitStore, Order: BitOrder> codec::Encode for DecodedBits<Store, Orde
}
#[doc(hidden)]
pub struct DecodedBitsVisitor<S, O>(std::marker::PhantomData<(S, O)>);
impl<Store, Order> scale_decode::Visitor for DecodedBitsVisitor<Store, Order> {
pub struct DecodedBitsVisitor<S, O, R: TypeResolver>(std::marker::PhantomData<(S, O, R)>);
impl<Store, Order, R: TypeResolver> scale_decode::Visitor for DecodedBitsVisitor<Store, Order, R> {
type Value<'scale, 'info> = DecodedBits<Store, Order>;
type Error = scale_decode::Error;
type TypeResolver = R;
fn unchecked_decode_as_type<'scale, 'info>(
self,
input: &mut &'scale [u8],
type_id: scale_decode::visitor::TypeId,
types: &'info scale_info::PortableRegistry,
type_id: &R::TypeId,
types: &'info R,
) -> scale_decode::visitor::DecodeAsTypeResult<
Self,
Result<Self::Value<'scale, 'info>, Self::Error>,
> {
let res = scale_decode::visitor::decode_with_visitor(
input,
type_id.0,
types,
Bits::into_visitor(),
)
.map(|bits| DecodedBits {
bits,
_marker: PhantomData,
});
let res =
scale_decode::visitor::decode_with_visitor(input, type_id, types, Bits::into_visitor())
.map(|bits| DecodedBits {
bits,
_marker: PhantomData,
});
scale_decode::visitor::DecodeAsTypeResult::Decoded(res)
}
}
impl<Store, Order> scale_decode::IntoVisitor for DecodedBits<Store, Order> {
type Visitor = DecodedBitsVisitor<Store, Order>;
fn into_visitor() -> Self::Visitor {
type AnyVisitor<R: scale_decode::TypeResolver> = DecodedBitsVisitor<Store, Order, R>;
fn into_visitor<R: TypeResolver>() -> DecodedBitsVisitor<Store, Order, R> {
DecodedBitsVisitor(PhantomData)
}
}
impl<Store, Order> scale_encode::EncodeAsType for DecodedBits<Store, Order> {
fn encode_as_type_to(
fn encode_as_type_to<R: TypeResolver>(
&self,
type_id: u32,
types: &scale_info::PortableRegistry,
type_id: &R::TypeId,
types: &R,
out: &mut Vec<u8>,
) -> Result<(), scale_encode::Error> {
self.bits.encode_as_type_to(type_id, types, out)
subxt/src/utils/fetch_chain_spec.rs
+113
View File
@@ -0,0 +1,113 @@
// Copyright 2019-2024 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::macros::{cfg_jsonrpsee_native, cfg_jsonrpsee_web};
use serde_json::value::RawValue;
/// Possible errors encountered trying to fetch a chain spec from an RPC node.
#[derive(thiserror::Error, Debug)]
#[allow(missing_docs)]
pub enum FetchChainspecError {
#[error("Cannot fetch chain spec: RPC error: {0}.")]
RpcError(String),
#[error("Cannot fetch chain spec: Invalid URL.")]
InvalidUrl,
#[error("Cannot fetch chain spec: Invalid URL scheme.")]
InvalidScheme,
#[error("Cannot fetch chain spec: Handshake error establishing WS connection.")]
HandshakeError,
}
/// Fetch a chain spec from an RPC node at the given URL.
pub async fn fetch_chainspec_from_rpc_node(
url: impl AsRef<str>,
) -> Result<Box<RawValue>, FetchChainspecError> {
use jsonrpsee::core::client::{ClientT, SubscriptionClientT};
use jsonrpsee::rpc_params;
let client = jsonrpsee_helpers::client(url.as_ref()).await?;
let result = client
.request("sync_state_genSyncSpec", jsonrpsee::rpc_params![true])
.await
.map_err(|err| FetchChainspecError::RpcError(err.to_string()))?;
// Subscribe to the finalized heads of the chain.
let mut subscription = SubscriptionClientT::subscribe::<Box<RawValue>, _>(
&client,
"chain_subscribeFinalizedHeads",
rpc_params![],
"chain_unsubscribeFinalizedHeads",
)
.await
.map_err(|err| FetchChainspecError::RpcError(err.to_string()))?;
// We must ensure that the finalized block of the chain is not the block included
// in the chainSpec.
// This is a temporary workaround for: https://github.com/smol-dot/smoldot/issues/1562.
// The first finalized block that is received might by the finalized block could be the one
// included in the chainSpec. Decoding the chainSpec for this purpose is too complex.
let _ = subscription.next().await;
let _ = subscription.next().await;
Ok(result)
}
cfg_jsonrpsee_native! {
mod jsonrpsee_helpers {
use super::FetchChainspecError;
use tokio_util::compat::Compat;
pub use jsonrpsee::{
client_transport::ws::{self, EitherStream, Url, WsTransportClientBuilder},
core::client::Client,
};
pub type Sender = ws::Sender<Compat<EitherStream>>;
pub type Receiver = ws::Receiver<Compat<EitherStream>>;
/// Build WS RPC client from URL
pub async fn client(url: &str) -> Result<Client, FetchChainspecError> {
let url = Url::parse(url).map_err(|_| FetchChainspecError::InvalidUrl)?;
if url.scheme() != "ws" && url.scheme() != "wss" {
return Err(FetchChainspecError::InvalidScheme);
}
let (sender, receiver) = ws_transport(url).await?;
Ok(Client::builder()
.max_buffer_capacity_per_subscription(4096)
.build_with_tokio(sender, receiver))
}
async fn ws_transport(url: Url) -> Result<(Sender, Receiver), FetchChainspecError> {
WsTransportClientBuilder::default()
.build(url)
.await
.map_err(|_| FetchChainspecError::HandshakeError)
}
}
}
cfg_jsonrpsee_web! {
mod jsonrpsee_helpers {
use super::FetchChainspecError;
pub use jsonrpsee::{
client_transport::web,
core::client::{Client, ClientBuilder},
};
/// Build web RPC client from URL
pub async fn client(url: &str) -> Result<Client, FetchChainspecError> {
let (sender, receiver) = web::connect(url)
.await
.map_err(|_| FetchChainspecError::HandshakeError)?;
Ok(ClientBuilder::default()
.max_buffer_capacity_per_subscription(4096)
.build_with_wasm(sender, receiver))
}
}
}
subxt/src/utils/mod.rs
+6
View File
@@ -14,6 +14,7 @@ mod unchecked_extrinsic;
mod wrapper_opaque;
use crate::error::RpcError;
use crate::macros::cfg_jsonrpsee;
use crate::Error;
use codec::{Compact, Decode, Encode};
use derivative::Derivative;
@@ -27,6 +28,11 @@ pub use static_type::Static;
pub use unchecked_extrinsic::UncheckedExtrinsic;
pub use wrapper_opaque::WrapperKeepOpaque;
cfg_jsonrpsee! {
mod fetch_chain_spec;
pub use fetch_chain_spec::{fetch_chainspec_from_rpc_node, FetchChainspecError};
}
// Used in codegen
#[doc(hidden)]
pub use primitive_types::{H160, H256, H512};
subxt/src/utils/static_type.rs
+11 -10
View File
@@ -3,7 +3,7 @@
// see LICENSE for license details.
use codec::{Decode, Encode};
use scale_decode::{visitor::DecodeAsTypeResult, IntoVisitor, Visitor};
use scale_decode::{visitor::DecodeAsTypeResult, IntoVisitor, TypeResolver, Visitor};
use scale_encode::EncodeAsType;
/// If the type inside this implements [`Encode`], this will implement [`scale_encode::EncodeAsType`].
@@ -18,10 +18,10 @@ use scale_encode::EncodeAsType;
pub struct Static<T>(pub T);
impl<T: Encode> EncodeAsType for Static<T> {
fn encode_as_type_to(
fn encode_as_type_to<R: TypeResolver>(
&self,
_type_id: u32,
_types: &scale_decode::PortableRegistry,
_type_id: &R::TypeId,
_types: &R,
out: &mut Vec<u8>,
) -> Result<(), scale_encode::Error> {
self.0.encode_to(out);
@@ -29,17 +29,18 @@ impl<T: Encode> EncodeAsType for Static<T> {
}
}
pub struct StaticDecodeAsTypeVisitor<T>(std::marker::PhantomData<T>);
pub struct StaticDecodeAsTypeVisitor<T, R>(std::marker::PhantomData<(T, R)>);
impl<T: Decode> Visitor for StaticDecodeAsTypeVisitor<T> {
impl<T: Decode, R: TypeResolver> Visitor for StaticDecodeAsTypeVisitor<T, R> {
type Value<'scale, 'info> = Static<T>;
type Error = scale_decode::Error;
type TypeResolver = R;
fn unchecked_decode_as_type<'scale, 'info>(
self,
input: &mut &'scale [u8],
_type_id: scale_decode::visitor::TypeId,
_types: &'info scale_info::PortableRegistry,
_type_id: &R::TypeId,
_types: &'info R,
) -> DecodeAsTypeResult<Self, Result<Self::Value<'scale, 'info>, Self::Error>> {
use scale_decode::{visitor::DecodeError, Error};
let decoded = T::decode(input)
@@ -50,8 +51,8 @@ impl<T: Decode> Visitor for StaticDecodeAsTypeVisitor<T> {
}
impl<T: Decode> IntoVisitor for Static<T> {
type Visitor = StaticDecodeAsTypeVisitor<T>;
fn into_visitor() -> Self::Visitor {
type AnyVisitor<R: TypeResolver> = StaticDecodeAsTypeVisitor<T, R>;
fn into_visitor<R: TypeResolver>() -> StaticDecodeAsTypeVisitor<T, R> {
StaticDecodeAsTypeVisitor(std::marker::PhantomData)
}
}
subxt/src/utils/unchecked_extrinsic.rs
+16 -13
View File
@@ -12,7 +12,7 @@
use std::marker::PhantomData;
use codec::{Decode, Encode};
use scale_decode::{visitor::DecodeAsTypeResult, DecodeAsType, IntoVisitor, Visitor};
use scale_decode::{visitor::DecodeAsTypeResult, DecodeAsType, IntoVisitor, TypeResolver, Visitor};
use super::{Encoded, Static};
@@ -52,10 +52,10 @@ impl<Address, Call, Signature, Extra> Decode
impl<Address, Call, Signature, Extra> scale_encode::EncodeAsType
for UncheckedExtrinsic<Address, Call, Signature, Extra>
{
fn encode_as_type_to(
fn encode_as_type_to<R: TypeResolver>(
&self,
type_id: u32,
types: &scale_info::PortableRegistry,
type_id: &R::TypeId,
types: &R,
out: &mut Vec<u8>,
) -> Result<(), scale_encode::Error> {
self.0.encode_as_type_to(type_id, types, out)
@@ -78,32 +78,35 @@ impl<Address, Call, Signature, Extra> From<UncheckedExtrinsic<Address, Call, Sig
}
}
pub struct UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra>(
PhantomData<(Address, Call, Signature, Extra)>,
pub struct UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra, R: TypeResolver>(
PhantomData<(Address, Call, Signature, Extra, R)>,
);
impl<Address, Call, Signature, Extra> Visitor
for UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra>
impl<Address, Call, Signature, Extra, R: TypeResolver> Visitor
for UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra, R>
{
type Value<'scale, 'info> = UncheckedExtrinsic<Address, Call, Signature, Extra>;
type Error = scale_decode::Error;
type TypeResolver = R;
fn unchecked_decode_as_type<'scale, 'info>(
self,
input: &mut &'scale [u8],
type_id: scale_decode::visitor::TypeId,
types: &'info scale_info::PortableRegistry,
type_id: &R::TypeId,
types: &'info R,
) -> DecodeAsTypeResult<Self, Result<Self::Value<'scale, 'info>, Self::Error>> {
DecodeAsTypeResult::Decoded(Self::Value::decode_as_type(input, type_id.0, types))
DecodeAsTypeResult::Decoded(Self::Value::decode_as_type(input, type_id, types))
}
}
impl<Address, Call, Signature, Extra> IntoVisitor
for UncheckedExtrinsic<Address, Call, Signature, Extra>
{
type Visitor = UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra>;
type AnyVisitor<R: TypeResolver> =
UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra, R>;
fn into_visitor() -> Self::Visitor {
fn into_visitor<R: TypeResolver>(
) -> UncheckedExtrinsicDecodeAsTypeVisitor<Address, Call, Signature, Extra, R> {
UncheckedExtrinsicDecodeAsTypeVisitor(PhantomData)
}
}
subxt/src/utils/wrapper_opaque.rs
+28 -38
View File
@@ -5,7 +5,7 @@
use super::PhantomDataSendSync;
use codec::{Compact, Decode, DecodeAll, Encode};
use derivative::Derivative;
use scale_decode::{IntoVisitor, Visitor};
use scale_decode::{ext::scale_type_resolver::visitor, IntoVisitor, TypeResolver, Visitor};
use scale_encode::EncodeAsType;
/// A wrapper for any type `T` which implement encode/decode in a way compatible with `Vec<u8>`.
@@ -74,57 +74,47 @@ impl<T> WrapperKeepOpaque<T> {
}
impl<T> EncodeAsType for WrapperKeepOpaque<T> {
fn encode_as_type_to(
fn encode_as_type_to<R: TypeResolver>(
&self,
type_id: u32,
types: &scale_info::PortableRegistry,
type_id: &R::TypeId,
types: &R,
out: &mut Vec<u8>,
) -> Result<(), scale_encode::Error> {
use scale_encode::error::{Error, ErrorKind, Kind};
let Some(ty) = types.resolve(type_id) else {
return Err(Error::new(ErrorKind::TypeNotFound(type_id)));
};
// Do a basic check that the target shape lines up.
let scale_info::TypeDef::Composite(_) = &ty.type_def else {
return Err(Error::new(ErrorKind::WrongShape {
let visitor = visitor::new(out, |_, _| {
// Check that the target shape lines up: any other shape but the composite is wrong.
Err(Error::new(ErrorKind::WrongShape {
actual: Kind::Struct,
expected: type_id,
}));
};
expected_id: format!("{:?}", type_id),
}))
})
.visit_composite(|out, _fields| {
self.data.encode_to(out);
Ok(())
});
// Check that the name also lines up.
if ty.path.ident().as_deref() != Some("WrapperKeepOpaque") {
return Err(Error::new(ErrorKind::WrongShape {
actual: Kind::Struct,
expected: type_id,
}));
}
// Just blat the bytes out.
self.data.encode_to(out);
Ok(())
types
.resolve_type(type_id, visitor)
.map_err(|_| Error::new(ErrorKind::TypeNotFound(format!("{:?}", type_id))))?
}
}
pub struct WrapperKeepOpaqueVisitor<T>(std::marker::PhantomData<T>);
impl<T> Visitor for WrapperKeepOpaqueVisitor<T> {
pub struct WrapperKeepOpaqueVisitor<T, R>(std::marker::PhantomData<(T, R)>);
impl<T, R: TypeResolver> Visitor for WrapperKeepOpaqueVisitor<T, R> {
type Value<'scale, 'info> = WrapperKeepOpaque<T>;
type Error = scale_decode::Error;
type TypeResolver = R;
fn visit_composite<'scale, 'info>(
self,
value: &mut scale_decode::visitor::types::Composite<'scale, 'info>,
_type_id: scale_decode::visitor::TypeId,
value: &mut scale_decode::visitor::types::Composite<'scale, 'info, R>,
_type_id: &R::TypeId,
) -> Result<Self::Value<'scale, 'info>, Self::Error> {
use scale_decode::error::{Error, ErrorKind};
if value.path().ident().as_deref() != Some("WrapperKeepOpaque") {
return Err(Error::custom_str(
"Type to decode is not 'WrapperTypeKeepOpaque'",
));
}
// TODO: When `scale-type-resolver` [provides struct names](https://github.com/paritytech/scale-type-resolver/issues/4), check that this struct name is `WrapperKeepOpaque`
if value.remaining() != 2 {
return Err(Error::new(ErrorKind::WrongLength {
actual_len: value.remaining(),
@@ -151,8 +141,8 @@ impl<T> Visitor for WrapperKeepOpaqueVisitor<T> {
}
impl<T> IntoVisitor for WrapperKeepOpaque<T> {
type Visitor = WrapperKeepOpaqueVisitor<T>;
fn into_visitor() -> Self::Visitor {
type AnyVisitor<R: TypeResolver> = WrapperKeepOpaqueVisitor<T, R>;
fn into_visitor<R: TypeResolver>() -> WrapperKeepOpaqueVisitor<T, R> {
WrapperKeepOpaqueVisitor(std::marker::PhantomData)
}
}
@@ -205,7 +195,7 @@ mod test {
let (type_id, types) = make_type::<T>();
let scale_codec_encoded = t.encode();
let encode_as_type_encoded = t.encode_as_type(type_id, &types).unwrap();
let encode_as_type_encoded = t.encode_as_type(&type_id, &types).unwrap();
assert_eq!(
scale_codec_encoded, encode_as_type_encoded,
@@ -213,7 +203,7 @@ mod test {
);
let decode_as_type_bytes = &mut &*scale_codec_encoded;
let decoded_as_type = T::decode_as_type(decode_as_type_bytes, type_id, &types)
let decoded_as_type = T::decode_as_type(decode_as_type_bytes, &type_id, &types)
.expect("decode-as-type decodes");
let decode_scale_codec_bytes = &mut &*scale_codec_encoded;
testing/integration-tests/Cargo.toml
+1 -1
View File
@@ -37,7 +37,7 @@ scale-info = { workspace = true, features = ["bit-vec"] }
sp-core = { workspace = true }
syn = { workspace = true }
subxt = { workspace = true, features = ["unstable-metadata", "native", "jsonrpsee", "substrate-compat"] }
subxt-signer = { workspace = true }
subxt-signer = { workspace = true, features = ["default"] }
subxt-codegen = { workspace = true }
subxt-metadata = { workspace = true }
test-runtime = { workspace = true }
testing/integration-tests/src/full_client/client/unstable_rpcs.rs
+116 -9
View File
@@ -14,7 +14,9 @@ use subxt::{
FollowEvent, Initialized, MethodResponse, RuntimeEvent, RuntimeVersionEvent, StorageQuery,
StorageQueryType,
},
utils::AccountId32,
config::Hasher,
utils::{AccountId32, MultiAddress},
SubstrateConfig,
};
#[cfg(lightclient)]
@@ -36,7 +38,7 @@ async fn chainhead_unstable_follow() {
assert_eq!(
event,
FollowEvent::Initialized(Initialized {
finalized_block_hash,
finalized_block_hashes: vec![finalized_block_hash],
finalized_block_runtime: None,
})
);
@@ -51,7 +53,7 @@ async fn chainhead_unstable_follow() {
assert_matches!(
event,
FollowEvent::Initialized(init) => {
assert_eq!(init.finalized_block_hash, finalized_block_hash);
assert_eq!(init.finalized_block_hashes, vec![finalized_block_hash]);
if let Some(RuntimeEvent::Valid(RuntimeVersionEvent { spec })) = init.finalized_block_runtime {
assert_eq!(spec.spec_version, runtime_version.spec_version);
assert_eq!(spec.transaction_version, runtime_version.transaction_version);
@@ -70,7 +72,7 @@ async fn chainhead_unstable_body() {
let mut blocks = rpc.chainhead_unstable_follow(false).await.unwrap();
let event = blocks.next().await.unwrap().unwrap();
let hash = match event {
FollowEvent::Initialized(init) => init.finalized_block_hash,
FollowEvent::Initialized(init) => init.finalized_block_hashes.last().unwrap().clone(),
_ => panic!("Unexpected event"),
};
let sub_id = blocks.subscription_id().unwrap();
@@ -99,7 +101,7 @@ async fn chainhead_unstable_header() {
let mut blocks = rpc.chainhead_unstable_follow(false).await.unwrap();
let event = blocks.next().await.unwrap().unwrap();
let hash = match event {
FollowEvent::Initialized(init) => init.finalized_block_hash,
FollowEvent::Initialized(init) => init.finalized_block_hashes.last().unwrap().clone(),
_ => panic!("Unexpected event"),
};
let sub_id = blocks.subscription_id().unwrap();
@@ -127,7 +129,7 @@ async fn chainhead_unstable_storage() {
let mut blocks = rpc.chainhead_unstable_follow(false).await.unwrap();
let event = blocks.next().await.unwrap().unwrap();
let hash = match event {
FollowEvent::Initialized(init) => init.finalized_block_hash,
FollowEvent::Initialized(init) => init.finalized_block_hashes.last().unwrap().clone(),
_ => panic!("Unexpected event"),
};
let sub_id = blocks.subscription_id().unwrap();
@@ -172,7 +174,7 @@ async fn chainhead_unstable_call() {
let mut blocks = rpc.chainhead_unstable_follow(true).await.unwrap();
let event = blocks.next().await.unwrap().unwrap();
let hash = match event {
FollowEvent::Initialized(init) => init.finalized_block_hash,
FollowEvent::Initialized(init) => init.finalized_block_hashes.last().unwrap().clone(),
_ => panic!("Unexpected event"),
};
let sub_id = blocks.subscription_id().unwrap();
@@ -209,7 +211,7 @@ async fn chainhead_unstable_unpin() {
let mut blocks = rpc.chainhead_unstable_follow(true).await.unwrap();
let event = blocks.next().await.unwrap().unwrap();
let hash = match event {
FollowEvent::Initialized(init) => init.finalized_block_hash,
FollowEvent::Initialized(init) => init.finalized_block_hashes.last().unwrap().clone(),
_ => panic!("Unexpected event"),
};
let sub_id = blocks.subscription_id().unwrap();
@@ -269,7 +271,7 @@ async fn transaction_unstable_submit_and_watch() {
let tx_bytes = ctx
.client()
.tx()
.create_signed_with_nonce(&payload, &dev::alice(), 0, Default::default())
.create_signed_offline(&payload, &dev::alice(), Default::default())
.unwrap()
.into_encoded();
@@ -317,3 +319,108 @@ async fn next_operation_event<
panic!("Cannot find operation related event after {NUM_EVENTS} produced events");
}
#[tokio::test]
async fn transaction_unstable_broadcast() {
let bob = dev::bob();
let bob_address: MultiAddress<AccountId32, u32> = bob.public_key().into();
let ctx = test_context().await;
let api = ctx.client();
let rpc = ctx.unstable_rpc_methods().await;
let tx = node_runtime::tx()
.balances()
.transfer_allow_death(bob_address.clone(), 10_001);
let tx_bytes = ctx
.client()
.tx()
.create_signed_offline(&tx, &dev::alice(), Default::default())
.unwrap()
.into_encoded();
let tx_hash = <SubstrateConfig as subxt::Config>::Hasher::hash(&tx_bytes[2..]);
// Subscribe to finalized blocks.
let mut finalized_sub = api.blocks().subscribe_finalized().await.unwrap();
// Expect the tx to be encountered in a maximum number of blocks.
let mut num_blocks: usize = 10;
// Submit the transaction.
let _operation_id = rpc
.transaction_unstable_broadcast(&tx_bytes)
.await
.unwrap()
.expect("Server is not overloaded by 1 tx; qed");
while let Some(finalized) = finalized_sub.next().await {
let finalized = finalized.unwrap();
// Started with positive, should not overflow.
num_blocks = num_blocks.saturating_sub(1);
if num_blocks == 0 {
panic!("Did not find the tx in due time");
}
let extrinsics = finalized.extrinsics().await.unwrap();
let block_extrinsics = extrinsics
.iter()
.map(|res| res.unwrap())
.collect::<Vec<_>>();
let Some(ext) = block_extrinsics
.iter()
.find(|ext| <SubstrateConfig as subxt::Config>::Hasher::hash(ext.bytes()) == tx_hash)
else {
continue;
};
let ext = ext
.as_extrinsic::<node_runtime::balances::calls::types::TransferAllowDeath>()
.unwrap()
.unwrap();
assert_eq!(ext.value, 10_001);
return;
}
}
#[tokio::test]
async fn transaction_unstable_stop() {
let bob = dev::bob();
let bob_address: MultiAddress<AccountId32, u32> = bob.public_key().into();
let ctx = test_context().await;
let rpc = ctx.unstable_rpc_methods().await;
// Cannot stop an operation that was not started.
let _err = rpc
.transaction_unstable_stop("non-existent-operation-id")
.await
.unwrap_err();
// Submit a transaction and stop it.
let tx = node_runtime::tx()
.balances()
.transfer_allow_death(bob_address.clone(), 10_001);
let tx_bytes = ctx
.client()
.tx()
.create_signed_offline(&tx, &dev::alice(), Default::default())
.unwrap()
.into_encoded();
// Submit the transaction.
let operation_id = rpc
.transaction_unstable_broadcast(&tx_bytes)
.await
.unwrap()
.expect("Server is not overloaded by 1 tx; qed");
let _ = rpc.transaction_unstable_stop(&operation_id).await.unwrap();
// Cannot stop it twice.
let _err = rpc
.transaction_unstable_stop(&operation_id)
.await
.unwrap_err();
}
testing/integration-tests/src/full_client/codegen/polkadot.rs
+8763 -2363
View File
File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More