light-client: Add experimental light-client support (#965)

* rpc/types: Decode `SubstrateTxStatus` for substrate and smoldot

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Add light client Error

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Add background task to manage RPC responses

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Implement the light client RPC in subxt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Expose light client under experimental feature-flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Add development chain spec for local nodes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Add light client example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update sp-* crates and smoldot to use git with branch / rev

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply cargo fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Import hashmap entry

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Fetch spec only if jsonrpsee feature is enabled

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update subxt/src/rpc/lightclient/background.rs

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Fix typo

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Update dev chain spec

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* types: Handle storage replies from chainHead_storage

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Add polkadot spec

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Handle RPC error responses

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Tx basic with light client for local nodes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* example: Light client coprehensive example for live chains

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Remove prior light client example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* feature: Rename experimental to unstable

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* book: Add light client section

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Ignore validated events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust tests for light-clients and normal clients

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Keep lightclient variant

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove support for chainHead_storage for light client

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update light client to point to crates.io

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update sp-crates from crates.io

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Replace Atomic with u64

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add LightClientBuilder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust chainspec with provided bootnodes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add potential_relay_chains to light client builder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Move the light-client to the background task

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust tracing logs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update book and example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply cargo fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove dev_spec.json artifact

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Examples fix duplicate Cargo.toml

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use tracing_subscriber crate

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy for different features

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add comment about bootNodes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add comment about tracing-sub dependency

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Run integration-tests with light-client

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Feature guard some incompatible tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ci: Enable light-client tests under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ci: Fix git step name

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust flags for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust warnings

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Rename feature flag jsonrpsee-ws to jsonrpsee

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix cargo check

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ci: Run tests on just 2 threads

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Move light-client to subxt/src/client

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust LightClientBuilder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use ws_url to construct light client for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Refactor background

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Address feedback

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove polkadot.spec and trim sub_id

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Wait for substrate to produce block before connecting light client

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust builder and tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ci: Use release for light client testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add single test for light-client

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Wait for more blocks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use polkadot endpoint for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust cargo check

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Remove light client chain connection example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust cargo.toml section for the old example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust background task to use usize for subscription Id

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Build bootnodes with serde_json::Value directly

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Make channel between subxt user and subxt background unbounded

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update subxt/src/client/lightclient/builder.rs

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Switch to smoldot 0.6.0 from 0.5.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Move testing to `full_client` and `light_client` higher modules

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove subscriptionID type

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove subxt/integration-testing feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust wait_for_blocks documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust utils import for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove into_iter from builder construction

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
This commit is contained in:
Alexandru Vasile
2023-06-26 12:10:57 +03:00
committed by GitHub
parent 8413c4d2dd
commit ef89752904
42 changed files with 2352 additions and 147 deletions
@@ -0,0 +1,127 @@
// Copyright 2019-2023 Parity Technologies (UK) Ltd.
// This file is dual-licensed as Apache-2.0 or GPL-3.0.
// see LICENSE for license details.
use crate::{node_runtime, test_context, utils::wait_for_blocks};
use subxt::utils::AccountId32;
use subxt_signer::sr25519::dev;
#[tokio::test]
async fn storage_plain_lookup() -> Result<(), subxt::Error> {
let ctx = test_context().await;
let api = ctx.client();
// Look up a plain value. Wait long enough that we don't get the genesis block data,
// because it may have no storage associated with it.
wait_for_blocks(&api).await;
let addr = node_runtime::storage().timestamp().now();
let entry = api
.storage()
.at_latest()
.await?
.fetch_or_default(&addr)
.await?;
assert!(entry > 0);
Ok(())
}
#[tokio::test]
async fn storage_map_lookup() -> Result<(), subxt::Error> {
let ctx = test_context().await;
let api = ctx.client();
let signer = dev::alice();
let alice: AccountId32 = dev::alice().public_key().into();
// Do some transaction to bump the Alice nonce to 1:
let remark_tx = node_runtime::tx().system().remark(vec![1, 2, 3, 4, 5]);
api.tx()
.sign_and_submit_then_watch_default(&remark_tx, &signer)
.await?
.wait_for_finalized_success()
.await?;
// Look up the nonce for the user (we expect it to be 1).
let nonce_addr = node_runtime::storage().system().account(alice);
let entry = api
.storage()
.at_latest()
.await?
.fetch_or_default(&nonce_addr)
.await?;
assert_eq!(entry.nonce, 1);
Ok(())
}
// This fails until the fix in https://github.com/paritytech/subxt/pull/458 is introduced.
// Here we create a key that looks a bit like a StorageNMap key, but should in fact be
// treated as a StorageKey (ie we should hash both values together with one hasher, rather
// than hash both values separately, or ignore the second value).
#[tokio::test]
async fn storage_n_mapish_key_is_properly_created() -> Result<(), subxt::Error> {
use codec::Encode;
use node_runtime::runtime_types::sp_core::crypto::KeyTypeId;
let ctx = test_context().await;
let api = ctx.client();
// This is what the generated code hashes a `session().key_owner(..)` key into:
let actual_key = node_runtime::storage()
.session()
.key_owner(KeyTypeId([1, 2, 3, 4]), [5u8, 6, 7, 8]);
let actual_key_bytes = api.storage().address_bytes(&actual_key)?;
// Let's manually hash to what we assume it should be and compare:
let expected_key_bytes = {
// Hash the prefix to the storage entry:
let mut bytes = sp_core::twox_128("Session".as_bytes()).to_vec();
bytes.extend(&sp_core::twox_128("KeyOwner".as_bytes())[..]);
// twox64_concat a *tuple* of the args expected:
let suffix = (KeyTypeId([1, 2, 3, 4]), vec![5u8, 6, 7, 8]).encode();
bytes.extend(sp_core::twox_64(&suffix));
bytes.extend(&suffix);
bytes
};
assert_eq!(actual_key_bytes, expected_key_bytes);
Ok(())
}
#[tokio::test]
async fn storage_n_map_storage_lookup() -> Result<(), subxt::Error> {
let ctx = test_context().await;
let api = ctx.client();
// Boilerplate; we create a new asset class with ID 99, and then
// we "approveTransfer" of some of this asset class. This gives us an
// entry in the `Approvals` StorageNMap that we can try to look up.
let signer = dev::alice();
let alice: AccountId32 = dev::alice().public_key().into();
let bob: AccountId32 = dev::bob().public_key().into();
let tx1 = node_runtime::tx()
.assets()
.create(99, alice.clone().into(), 1);
let tx2 = node_runtime::tx()
.assets()
.approve_transfer(99, bob.clone().into(), 123);
api.tx()
.sign_and_submit_then_watch_default(&tx1, &signer)
.await?
.wait_for_finalized_success()
.await?;
api.tx()
.sign_and_submit_then_watch_default(&tx2, &signer)
.await?
.wait_for_finalized_success()
.await?;
// The actual test; look up this approval in storage:
let addr = node_runtime::storage().assets().approvals(99, alice, bob);
let entry = api.storage().at_latest().await?.fetch(&addr).await?;
assert_eq!(entry.map(|a| a.amount), Some(123));
Ok(())
}