Files
pezkuwi-subxt/substrate/test-utils/client/src/lib.rs
T
David 29c0c6a4a8 jsonrpsee integration (#8783)
* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-05-10 08:52:19 +00:00

463 lines
13 KiB
Rust

// This file is part of Substrate.
// Copyright (C) 2019-2022 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Client testing utilities.
#![warn(missing_docs)]
pub mod client_ext;
pub use self::client_ext::{ClientBlockImportExt, ClientExt};
pub use sc_client_api::{
execution_extensions::{ExecutionExtensions, ExecutionStrategies},
BadBlocks, ForkBlocks,
};
pub use sc_client_db::{self, Backend};
pub use sc_executor::{self, NativeElseWasmExecutor, WasmExecutionMethod};
pub use sc_service::{client, RpcHandlers};
pub use sp_consensus;
pub use sp_keyring::{
ed25519::Keyring as Ed25519Keyring, sr25519::Keyring as Sr25519Keyring, AccountKeyring,
};
pub use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
pub use sp_runtime::{Storage, StorageChild};
pub use sp_state_machine::ExecutionStrategy;
use futures::{future::Future, stream::StreamExt};
use sc_client_api::BlockchainEvents;
use sc_service::client::{ClientConfig, LocalCallExecutor};
use serde::Deserialize;
use sp_core::storage::ChildInfo;
use sp_runtime::{codec::Encode, traits::Block as BlockT, OpaqueExtrinsic};
use std::{
collections::{HashMap, HashSet},
pin::Pin,
sync::Arc,
};
/// A genesis storage initialization trait.
pub trait GenesisInit: Default {
/// Construct genesis storage.
fn genesis_storage(&self) -> Storage;
}
impl GenesisInit for () {
fn genesis_storage(&self) -> Storage {
Default::default()
}
}
/// A builder for creating a test client instance.
pub struct TestClientBuilder<Block: BlockT, ExecutorDispatch, Backend, G: GenesisInit> {
execution_strategies: ExecutionStrategies,
genesis_init: G,
/// The key is an unprefixed storage key, this only contains
/// default child trie content.
child_storage_extension: HashMap<Vec<u8>, StorageChild>,
backend: Arc<Backend>,
_executor: std::marker::PhantomData<ExecutorDispatch>,
keystore: Option<SyncCryptoStorePtr>,
fork_blocks: ForkBlocks<Block>,
bad_blocks: BadBlocks<Block>,
enable_offchain_indexing_api: bool,
no_genesis: bool,
}
impl<Block: BlockT, ExecutorDispatch, G: GenesisInit> Default
for TestClientBuilder<Block, ExecutorDispatch, Backend<Block>, G>
{
fn default() -> Self {
Self::with_default_backend()
}
}
impl<Block: BlockT, ExecutorDispatch, G: GenesisInit>
TestClientBuilder<Block, ExecutorDispatch, Backend<Block>, G>
{
/// Create new `TestClientBuilder` with default backend.
pub fn with_default_backend() -> Self {
let backend = Arc::new(Backend::new_test(std::u32::MAX, std::u64::MAX));
Self::with_backend(backend)
}
/// Create new `TestClientBuilder` with default backend and pruning window size
pub fn with_pruning_window(keep_blocks: u32) -> Self {
let backend = Arc::new(Backend::new_test(keep_blocks, 0));
Self::with_backend(backend)
}
/// Create new `TestClientBuilder` with default backend and storage chain mode
pub fn with_tx_storage(keep_blocks: u32) -> Self {
let backend = Arc::new(Backend::new_test_with_tx_storage(keep_blocks, 0));
Self::with_backend(backend)
}
}
impl<Block: BlockT, ExecutorDispatch, Backend, G: GenesisInit>
TestClientBuilder<Block, ExecutorDispatch, Backend, G>
{
/// Create a new instance of the test client builder.
pub fn with_backend(backend: Arc<Backend>) -> Self {
TestClientBuilder {
backend,
execution_strategies: ExecutionStrategies::default(),
child_storage_extension: Default::default(),
genesis_init: Default::default(),
_executor: Default::default(),
keystore: None,
fork_blocks: None,
bad_blocks: None,
enable_offchain_indexing_api: false,
no_genesis: false,
}
}
/// Set the keystore that should be used by the externalities.
pub fn set_keystore(mut self, keystore: SyncCryptoStorePtr) -> Self {
self.keystore = Some(keystore);
self
}
/// Alter the genesis storage parameters.
pub fn genesis_init_mut(&mut self) -> &mut G {
&mut self.genesis_init
}
/// Give access to the underlying backend of these clients
pub fn backend(&self) -> Arc<Backend> {
self.backend.clone()
}
/// Extend child storage
pub fn add_child_storage(
mut self,
child_info: &ChildInfo,
key: impl AsRef<[u8]>,
value: impl AsRef<[u8]>,
) -> Self {
let storage_key = child_info.storage_key();
let entry = self.child_storage_extension.entry(storage_key.to_vec()).or_insert_with(|| {
StorageChild { data: Default::default(), child_info: child_info.clone() }
});
entry.data.insert(key.as_ref().to_vec(), value.as_ref().to_vec());
self
}
/// Set the execution strategy that should be used by all contexts.
pub fn set_execution_strategy(mut self, execution_strategy: ExecutionStrategy) -> Self {
self.execution_strategies = ExecutionStrategies {
syncing: execution_strategy,
importing: execution_strategy,
block_construction: execution_strategy,
offchain_worker: execution_strategy,
other: execution_strategy,
};
self
}
/// Sets custom block rules.
pub fn set_block_rules(
mut self,
fork_blocks: ForkBlocks<Block>,
bad_blocks: BadBlocks<Block>,
) -> Self {
self.fork_blocks = fork_blocks;
self.bad_blocks = bad_blocks;
self
}
/// Enable the offchain indexing api.
pub fn enable_offchain_indexing_api(mut self) -> Self {
self.enable_offchain_indexing_api = true;
self
}
/// Disable writing genesis.
pub fn set_no_genesis(mut self) -> Self {
self.no_genesis = true;
self
}
/// Build the test client with the given native executor.
pub fn build_with_executor<RuntimeApi>(
self,
executor: ExecutorDispatch,
) -> (
client::Client<Backend, ExecutorDispatch, Block, RuntimeApi>,
sc_consensus::LongestChain<Backend, Block>,
)
where
ExecutorDispatch:
sc_client_api::CallExecutor<Block> + sc_executor::RuntimeVersionOf + 'static,
Backend: sc_client_api::backend::Backend<Block>,
<Backend as sc_client_api::backend::Backend<Block>>::OffchainStorage: 'static,
{
let storage = {
let mut storage = self.genesis_init.genesis_storage();
// Add some child storage keys.
for (key, child_content) in self.child_storage_extension {
storage.children_default.insert(
key,
StorageChild {
data: child_content.data.into_iter().collect(),
child_info: child_content.child_info,
},
);
}
storage
};
let client = client::Client::new(
self.backend.clone(),
executor,
&storage,
self.fork_blocks,
self.bad_blocks,
ExecutionExtensions::new(
self.execution_strategies,
self.keystore,
sc_offchain::OffchainDb::factory_from_backend(&*self.backend),
),
None,
None,
ClientConfig {
offchain_indexing_api: self.enable_offchain_indexing_api,
no_genesis: self.no_genesis,
..Default::default()
},
)
.expect("Creates new client");
let longest_chain = sc_consensus::LongestChain::new(self.backend);
(client, longest_chain)
}
}
impl<Block: BlockT, D, Backend, G: GenesisInit>
TestClientBuilder<
Block,
client::LocalCallExecutor<Block, Backend, NativeElseWasmExecutor<D>>,
Backend,
G,
> where
D: sc_executor::NativeExecutionDispatch,
{
/// Build the test client with the given native executor.
pub fn build_with_native_executor<RuntimeApi, I>(
self,
executor: I,
) -> (
client::Client<
Backend,
client::LocalCallExecutor<Block, Backend, NativeElseWasmExecutor<D>>,
Block,
RuntimeApi,
>,
sc_consensus::LongestChain<Backend, Block>,
)
where
I: Into<Option<NativeElseWasmExecutor<D>>>,
D: sc_executor::NativeExecutionDispatch + 'static,
Backend: sc_client_api::backend::Backend<Block> + 'static,
{
let executor = executor.into().unwrap_or_else(|| {
NativeElseWasmExecutor::new(WasmExecutionMethod::Interpreted, None, 8, 2)
});
let executor = LocalCallExecutor::new(
self.backend.clone(),
executor,
Box::new(sp_core::testing::TaskExecutor::new()),
Default::default(),
)
.expect("Creates LocalCallExecutor");
self.build_with_executor(executor)
}
}
/// The output of an RPC transaction.
pub struct RpcTransactionOutput {
/// The output string of the transaction if any.
pub result: String,
/// An async receiver if data will be returned via a callback.
pub receiver: futures::channel::mpsc::UnboundedReceiver<String>,
}
impl std::fmt::Debug for RpcTransactionOutput {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "RpcTransactionOutput {{ result: {:?}, receiver }}", self.result)
}
}
/// An error for when the RPC call fails.
#[derive(Deserialize, Debug)]
pub struct RpcTransactionError {
/// A Number that indicates the error type that occurred.
pub code: i64,
/// A String providing a short description of the error.
pub message: String,
/// A Primitive or Structured value that contains additional information about the error.
pub data: Option<serde_json::Value>,
}
impl std::fmt::Display for RpcTransactionError {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
std::fmt::Debug::fmt(self, f)
}
}
/// An extension trait for `RpcHandlers`.
#[async_trait::async_trait]
pub trait RpcHandlersExt {
/// Send a transaction through the RpcHandlers.
async fn send_transaction(
&self,
extrinsic: OpaqueExtrinsic,
) -> Result<RpcTransactionOutput, RpcTransactionError>;
}
#[async_trait::async_trait]
impl RpcHandlersExt for RpcHandlers {
async fn send_transaction(
&self,
extrinsic: OpaqueExtrinsic,
) -> Result<RpcTransactionOutput, RpcTransactionError> {
let (result, rx) = self
.rpc_query(&format!(
r#"{{
"jsonrpc": "2.0",
"method": "author_submitExtrinsic",
"params": ["0x{}"],
"id": 0
}}"#,
hex::encode(extrinsic.encode())
))
.await
.expect("valid JSON-RPC request object; qed");
parse_rpc_result(result, rx)
}
}
pub(crate) fn parse_rpc_result(
result: String,
receiver: futures::channel::mpsc::UnboundedReceiver<String>,
) -> Result<RpcTransactionOutput, RpcTransactionError> {
let json: serde_json::Value =
serde_json::from_str(&result).expect("the result can only be a JSONRPC string; qed");
let error = json.as_object().expect("JSON result is always an object; qed").get("error");
if let Some(error) = error {
return Err(serde_json::from_value(error.clone())
.expect("the JSONRPC result's error is always valid; qed"))
}
Ok(RpcTransactionOutput { result, receiver })
}
/// An extension trait for `BlockchainEvents`.
pub trait BlockchainEventsExt<C, B>
where
C: BlockchainEvents<B>,
B: BlockT,
{
/// Wait for `count` blocks to be imported in the node and then exit. This function will not
/// return if no blocks are ever created, thus you should restrict the maximum amount of time of
/// the test execution.
fn wait_for_blocks(&self, count: usize) -> Pin<Box<dyn Future<Output = ()> + Send>>;
}
impl<C, B> BlockchainEventsExt<C, B> for C
where
C: BlockchainEvents<B>,
B: BlockT,
{
fn wait_for_blocks(&self, count: usize) -> Pin<Box<dyn Future<Output = ()> + Send>> {
assert!(count > 0, "'count' argument must be greater than 0");
let mut import_notification_stream = self.import_notification_stream();
let mut blocks = HashSet::new();
Box::pin(async move {
while let Some(notification) = import_notification_stream.next().await {
if notification.is_new_best {
blocks.insert(notification.hash);
if blocks.len() == count {
break
}
}
}
})
}
}
#[cfg(test)]
mod tests {
#[test]
fn parses_error_properly() {
let (_, rx) = futures::channel::mpsc::unbounded();
assert!(super::parse_rpc_result(
r#"{
"jsonrpc": "2.0",
"result": 19,
"id": 1
}"#
.to_string(),
rx
)
.is_ok());
let (_, rx) = futures::channel::mpsc::unbounded();
let error = super::parse_rpc_result(
r#"{
"jsonrpc": "2.0",
"error": {
"code": -32601,
"message": "Method not found"
},
"id": 1
}"#
.to_string(),
rx,
)
.unwrap_err();
assert_eq!(error.code, -32601);
assert_eq!(error.message, "Method not found");
assert!(error.data.is_none());
let (_, rx) = futures::channel::mpsc::unbounded();
let error = super::parse_rpc_result(
r#"{
"jsonrpc": "2.0",
"error": {
"code": -32601,
"message": "Method not found",
"data": 42
},
"id": 1
}"#
.to_string(),
rx,
)
.unwrap_err();
assert_eq!(error.code, -32601);
assert_eq!(error.message, "Method not found");
assert!(error.data.is_some());
}
}