jsonrpsee integration (#8783)

* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This commit is contained in:
David
2022-05-10 10:52:19 +02:00
committed by GitHub
parent e45d53552d
commit 29c0c6a4a8
93 changed files with 3813 additions and 5094 deletions
+109 -114
View File
@@ -17,12 +17,15 @@
//! System FRAME specific RPC methods.
use std::sync::Arc;
use std::{fmt::Display, sync::Arc};
use codec::{self, Codec, Decode, Encode};
use jsonrpsee::{
core::{async_trait, RpcResult},
proc_macros::rpc,
types::error::{CallError, ErrorObject},
};
use codec::{Codec, Decode, Encode};
use futures::FutureExt;
use jsonrpc_core::{Error as RpcError, ErrorCode};
use jsonrpc_derive::rpc;
use sc_rpc_api::DenyUnsafe;
use sc_transaction_pool_api::{InPoolTransaction, TransactionPool};
use sp_api::ApiExt;
@@ -31,26 +34,22 @@ use sp_blockchain::HeaderBackend;
use sp_core::{hexdisplay::HexDisplay, Bytes};
use sp_runtime::{generic::BlockId, legacy, traits};
pub use self::gen_client::Client as SystemClient;
pub use frame_system_rpc_runtime_api::AccountNonceApi;
/// Future that resolves to account nonce.
type FutureResult<T> = jsonrpc_core::BoxFuture<Result<T, RpcError>>;
/// System RPC methods.
#[rpc]
#[rpc(client, server)]
pub trait SystemApi<BlockHash, AccountId, Index> {
/// Returns the next valid index (aka nonce) for given account.
///
/// This method takes into consideration all pending transactions
/// currently in the pool and if no transactions are found in the pool
/// it fallbacks to query the index from the runtime (aka. state nonce).
#[rpc(name = "system_accountNextIndex", alias("account_nextIndex"))]
fn nonce(&self, account: AccountId) -> FutureResult<Index>;
#[method(name = "system_accountNextIndex", aliases = ["account_nextIndex"])]
async fn nonce(&self, account: AccountId) -> RpcResult<Index>;
/// Dry run an extrinsic at a given block. Return SCALE encoded ApplyExtrinsicResult.
#[rpc(name = "system_dryRun", alias("system_dryRunAt"))]
fn dry_run(&self, extrinsic: Bytes, at: Option<BlockHash>) -> FutureResult<Bytes>;
#[method(name = "system_dryRun", aliases = ["system_dryRunAt"])]
async fn dry_run(&self, extrinsic: Bytes, at: Option<BlockHash>) -> RpcResult<Bytes>;
}
/// Error type of this RPC api.
@@ -61,8 +60,8 @@ pub enum Error {
RuntimeError,
}
impl From<Error> for i64 {
fn from(e: Error) -> i64 {
impl From<Error> for i32 {
fn from(e: Error) -> i32 {
match e {
Error::RuntimeError => 1,
Error::DecodeError => 2,
@@ -71,22 +70,23 @@ impl From<Error> for i64 {
}
/// An implementation of System-specific RPC methods on full client.
pub struct FullSystem<P: TransactionPool, C, B> {
pub struct SystemRpc<P: TransactionPool, C, B> {
client: Arc<C>,
pool: Arc<P>,
deny_unsafe: DenyUnsafe,
_marker: std::marker::PhantomData<B>,
}
impl<P: TransactionPool, C, B> FullSystem<P, C, B> {
impl<P: TransactionPool, C, B> SystemRpc<P, C, B> {
/// Create new `FullSystem` given client and transaction pool.
pub fn new(client: Arc<C>, pool: Arc<P>, deny_unsafe: DenyUnsafe) -> Self {
FullSystem { client, pool, deny_unsafe, _marker: Default::default() }
Self { client, pool, deny_unsafe, _marker: Default::default() }
}
}
impl<P, C, Block, AccountId, Index> SystemApi<<Block as traits::Block>::Hash, AccountId, Index>
for FullSystem<P, C, Block>
#[async_trait]
impl<P, C, Block, AccountId, Index>
SystemApiServer<<Block as traits::Block>::Hash, AccountId, Index> for SystemRpc<P, C, Block>
where
C: sp_api::ProvideRuntimeApi<Block>,
C: HeaderBackend<Block>,
@@ -95,88 +95,83 @@ where
C::Api: BlockBuilder<Block>,
P: TransactionPool + 'static,
Block: traits::Block,
AccountId: Clone + std::fmt::Display + Codec,
Index: Clone + std::fmt::Display + Codec + Send + traits::AtLeast32Bit + 'static,
AccountId: Clone + Display + Codec + Send + 'static,
Index: Clone + Display + Codec + Send + traits::AtLeast32Bit + 'static,
{
fn nonce(&self, account: AccountId) -> FutureResult<Index> {
let get_nonce = || {
let api = self.client.runtime_api();
let best = self.client.info().best_hash;
let at = BlockId::hash(best);
async fn nonce(&self, account: AccountId) -> RpcResult<Index> {
let api = self.client.runtime_api();
let best = self.client.info().best_hash;
let at = BlockId::hash(best);
let nonce = api.account_nonce(&at, account.clone()).map_err(|e| RpcError {
code: ErrorCode::ServerError(Error::RuntimeError.into()),
message: "Unable to query nonce.".into(),
data: Some(e.to_string().into()),
})?;
Ok(adjust_nonce(&*self.pool, account, nonce))
};
let res = get_nonce();
async move { res }.boxed()
let nonce = api.account_nonce(&at, account.clone()).map_err(|e| {
CallError::Custom(ErrorObject::owned(
Error::RuntimeError.into(),
"Unable to query nonce.",
Some(e.to_string()),
))
})?;
Ok(adjust_nonce(&*self.pool, account, nonce))
}
fn dry_run(
async fn dry_run(
&self,
extrinsic: Bytes,
at: Option<<Block as traits::Block>::Hash>,
) -> FutureResult<Bytes> {
if let Err(err) = self.deny_unsafe.check_if_safe() {
return async move { Err(err.into()) }.boxed()
}
) -> RpcResult<Bytes> {
self.deny_unsafe.check_if_safe()?;
let api = self.client.runtime_api();
let at = BlockId::<Block>::hash(at.unwrap_or_else(||
// If the block hash is not supplied assume the best block.
self.client.info().best_hash));
let dry_run = || {
let api = self.client.runtime_api();
let at = BlockId::<Block>::hash(at.unwrap_or_else(||
// If the block hash is not supplied assume the best block.
self.client.info().best_hash));
let uxt: <Block as traits::Block>::Extrinsic =
Decode::decode(&mut &*extrinsic).map_err(|e| {
CallError::Custom(ErrorObject::owned(
Error::DecodeError.into(),
"Unable to dry run extrinsic",
Some(e.to_string()),
))
})?;
let uxt: <Block as traits::Block>::Extrinsic = Decode::decode(&mut &*extrinsic)
.map_err(|e| RpcError {
code: ErrorCode::ServerError(Error::DecodeError.into()),
message: "Unable to dry run extrinsic.".into(),
data: Some(e.to_string().into()),
})?;
let api_version = api
.api_version::<dyn BlockBuilder<Block>>(&at)
.map_err(|e| {
CallError::Custom(ErrorObject::owned(
Error::RuntimeError.into(),
"Unable to dry run extrinsic.",
Some(e.to_string()),
))
})?
.ok_or_else(|| {
CallError::Custom(ErrorObject::owned(
Error::RuntimeError.into(),
"Unable to dry run extrinsic.",
Some(format!("Could not find `BlockBuilder` api for block `{:?}`.", at)),
))
})?;
let api_version = api
.api_version::<dyn BlockBuilder<Block>>(&at)
.map_err(|e| RpcError {
code: ErrorCode::ServerError(Error::RuntimeError.into()),
message: "Unable to dry run extrinsic.".into(),
data: Some(e.to_string().into()),
let result = if api_version < 6 {
#[allow(deprecated)]
api.apply_extrinsic_before_version_6(&at, uxt)
.map(legacy::byte_sized_error::convert_to_latest)
.map_err(|e| {
CallError::Custom(ErrorObject::owned(
Error::RuntimeError.into(),
"Unable to dry run extrinsic.",
Some(e.to_string()),
))
})?
.ok_or_else(|| RpcError {
code: ErrorCode::ServerError(Error::RuntimeError.into()),
message: "Unable to dry run extrinsic.".into(),
data: Some(
format!("Could not find `BlockBuilder` api for block `{:?}`.", at).into(),
),
})?;
let result = if api_version < 6 {
#[allow(deprecated)]
api.apply_extrinsic_before_version_6(&at, uxt)
.map(legacy::byte_sized_error::convert_to_latest)
.map_err(|e| RpcError {
code: ErrorCode::ServerError(Error::RuntimeError.into()),
message: "Unable to dry run extrinsic.".into(),
data: Some(e.to_string().into()),
})?
} else {
api.apply_extrinsic(&at, uxt).map_err(|e| RpcError {
code: ErrorCode::ServerError(Error::RuntimeError.into()),
message: "Unable to dry run extrinsic.".into(),
data: Some(e.to_string().into()),
})?
};
Ok(Encode::encode(&result).into())
} else {
api.apply_extrinsic(&at, uxt).map_err(|e| {
CallError::Custom(ErrorObject::owned(
Error::RuntimeError.into(),
"Unable to dry run extrinsic.",
Some(e.to_string()),
))
})?
};
let res = dry_run();
async move { res }.boxed()
Ok(Encode::encode(&result).into())
}
}
@@ -220,7 +215,9 @@ where
mod tests {
use super::*;
use assert_matches::assert_matches;
use futures::executor::block_on;
use jsonrpsee::{core::Error as JsonRpseeError, types::error::CallError};
use sc_transaction_pool::BasicPool;
use sp_runtime::{
transaction_validity::{InvalidTransaction, TransactionValidityError},
@@ -228,8 +225,8 @@ mod tests {
};
use substrate_test_runtime_client::{runtime::Transfer, AccountKeyring};
#[test]
fn should_return_next_nonce_for_some_account() {
#[tokio::test]
async fn should_return_next_nonce_for_some_account() {
sp_tracing::try_init_simple();
// given
@@ -254,17 +251,17 @@ mod tests {
let ext1 = new_transaction(1);
block_on(pool.submit_one(&BlockId::number(0), source, ext1)).unwrap();
let accounts = FullSystem::new(client, pool, DenyUnsafe::Yes);
let accounts = SystemRpc::new(client, pool, DenyUnsafe::Yes);
// when
let nonce = accounts.nonce(AccountKeyring::Alice.into());
let nonce = accounts.nonce(AccountKeyring::Alice.into()).await;
// then
assert_eq!(block_on(nonce).unwrap(), 2);
assert_eq!(nonce.unwrap(), 2);
}
#[test]
fn dry_run_should_deny_unsafe() {
#[tokio::test]
async fn dry_run_should_deny_unsafe() {
sp_tracing::try_init_simple();
// given
@@ -273,17 +270,17 @@ mod tests {
let pool =
BasicPool::new_full(Default::default(), true.into(), None, spawner, client.clone());
let accounts = FullSystem::new(client, pool, DenyUnsafe::Yes);
let accounts = SystemRpc::new(client, pool, DenyUnsafe::Yes);
// when
let res = accounts.dry_run(vec![].into(), None);
// then
assert_eq!(block_on(res), Err(sc_rpc_api::UnsafeRpcError.into()));
let res = accounts.dry_run(vec![].into(), None).await;
assert_matches!(res, Err(JsonRpseeError::Call(CallError::Custom(e))) => {
assert!(e.message().contains("RPC call is unsafe to be called externally"));
});
}
#[test]
fn dry_run_should_work() {
#[tokio::test]
async fn dry_run_should_work() {
sp_tracing::try_init_simple();
// given
@@ -292,7 +289,7 @@ mod tests {
let pool =
BasicPool::new_full(Default::default(), true.into(), None, spawner, client.clone());
let accounts = FullSystem::new(client, pool, DenyUnsafe::No);
let accounts = SystemRpc::new(client, pool, DenyUnsafe::No);
let tx = Transfer {
from: AccountKeyring::Alice.into(),
@@ -303,16 +300,15 @@ mod tests {
.into_signed_tx();
// when
let res = accounts.dry_run(tx.encode().into(), None);
let bytes = accounts.dry_run(tx.encode().into(), None).await.expect("Call is successful");
// then
let bytes = block_on(res).unwrap().0;
let apply_res: ApplyExtrinsicResult = Decode::decode(&mut bytes.as_slice()).unwrap();
let apply_res: ApplyExtrinsicResult = Decode::decode(&mut bytes.as_ref()).unwrap();
assert_eq!(apply_res, Ok(Ok(())));
}
#[test]
fn dry_run_should_indicate_error() {
#[tokio::test]
async fn dry_run_should_indicate_error() {
sp_tracing::try_init_simple();
// given
@@ -321,7 +317,7 @@ mod tests {
let pool =
BasicPool::new_full(Default::default(), true.into(), None, spawner, client.clone());
let accounts = FullSystem::new(client, pool, DenyUnsafe::No);
let accounts = SystemRpc::new(client, pool, DenyUnsafe::No);
let tx = Transfer {
from: AccountKeyring::Alice.into(),
@@ -332,11 +328,10 @@ mod tests {
.into_signed_tx();
// when
let res = accounts.dry_run(tx.encode().into(), None);
let bytes = accounts.dry_run(tx.encode().into(), None).await.expect("Call is successful");
// then
let bytes = block_on(res).unwrap().0;
let apply_res: ApplyExtrinsicResult = Decode::decode(&mut bytes.as_slice()).unwrap();
let apply_res: ApplyExtrinsicResult = Decode::decode(&mut bytes.as_ref()).unwrap();
assert_eq!(apply_res, Err(TransactionValidityError::Invalid(InvalidTransaction::Stale)));
}
}