jsonrpsee integration (#8783)

* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This commit is contained in:
David
2022-05-10 10:52:19 +02:00
committed by GitHub
parent e45d53552d
commit 29c0c6a4a8
93 changed files with 3813 additions and 5094 deletions
+104 -76
View File
@@ -18,28 +18,27 @@
//! Authoring RPC module errors.
use crate::errors;
use jsonrpc_core as rpc;
use jsonrpsee::{
core::Error as JsonRpseeError,
types::error::{CallError, ErrorObject},
};
use sp_runtime::transaction_validity::InvalidTransaction;
/// Author RPC Result type.
pub type Result<T> = std::result::Result<T, Error>;
/// Author RPC future Result type.
pub type FutureResult<T> = jsonrpc_core::BoxFuture<Result<T>>;
/// Author RPC errors.
#[derive(Debug, thiserror::Error)]
pub enum Error {
/// Client error.
#[error("Client error: {}", .0)]
Client(Box<dyn std::error::Error + Send>),
Client(Box<dyn std::error::Error + Send + Sync>),
/// Transaction pool error,
#[error("Transaction pool error: {}", .0)]
Pool(#[from] sc_transaction_pool_api::error::Error),
/// Verification error
#[error("Extrinsic verification error: {}", .0)]
Verification(Box<dyn std::error::Error + Send>),
Verification(Box<dyn std::error::Error + Send + Sync>),
/// Incorrect extrinsic format.
#[error("Invalid extrinsic format: {}", .0)]
BadFormat(#[from] codec::Error),
@@ -58,98 +57,127 @@ pub enum Error {
}
/// Base code for all authorship errors.
const BASE_ERROR: i64 = 1000;
const BASE_ERROR: i32 = 1000;
/// Extrinsic has an invalid format.
const BAD_FORMAT: i64 = BASE_ERROR + 1;
const BAD_FORMAT: i32 = BASE_ERROR + 1;
/// Error during transaction verification in runtime.
const VERIFICATION_ERROR: i64 = BASE_ERROR + 2;
const VERIFICATION_ERROR: i32 = BASE_ERROR + 2;
/// Pool rejected the transaction as invalid
const POOL_INVALID_TX: i64 = BASE_ERROR + 10;
const POOL_INVALID_TX: i32 = BASE_ERROR + 10;
/// Cannot determine transaction validity.
const POOL_UNKNOWN_VALIDITY: i64 = POOL_INVALID_TX + 1;
const POOL_UNKNOWN_VALIDITY: i32 = POOL_INVALID_TX + 1;
/// The transaction is temporarily banned.
const POOL_TEMPORARILY_BANNED: i64 = POOL_INVALID_TX + 2;
const POOL_TEMPORARILY_BANNED: i32 = POOL_INVALID_TX + 2;
/// The transaction is already in the pool
const POOL_ALREADY_IMPORTED: i64 = POOL_INVALID_TX + 3;
const POOL_ALREADY_IMPORTED: i32 = POOL_INVALID_TX + 3;
/// Transaction has too low priority to replace existing one in the pool.
const POOL_TOO_LOW_PRIORITY: i64 = POOL_INVALID_TX + 4;
const POOL_TOO_LOW_PRIORITY: i32 = POOL_INVALID_TX + 4;
/// Including this transaction would cause a dependency cycle.
const POOL_CYCLE_DETECTED: i64 = POOL_INVALID_TX + 5;
const POOL_CYCLE_DETECTED: i32 = POOL_INVALID_TX + 5;
/// The transaction was not included to the pool because of the limits.
const POOL_IMMEDIATELY_DROPPED: i64 = POOL_INVALID_TX + 6;
const POOL_IMMEDIATELY_DROPPED: i32 = POOL_INVALID_TX + 6;
/// The transaction was not included to the pool since it is unactionable,
/// it is not propagable and the local node does not author blocks.
const POOL_UNACTIONABLE: i64 = POOL_INVALID_TX + 8;
const POOL_UNACTIONABLE: i32 = POOL_INVALID_TX + 8;
/// Transaction does not provide any tags, so the pool can't identify it.
const POOL_NO_TAGS: i32 = POOL_INVALID_TX + 9;
/// Invalid block ID.
const POOL_INVALID_BLOCK_ID: i32 = POOL_INVALID_TX + 10;
/// The pool is not accepting future transactions.
const POOL_FUTURE_TX: i32 = POOL_INVALID_TX + 11;
impl From<Error> for rpc::Error {
impl From<Error> for JsonRpseeError {
fn from(e: Error) -> Self {
use sc_transaction_pool_api::error::Error as PoolError;
match e {
Error::BadFormat(e) => rpc::Error {
code: rpc::ErrorCode::ServerError(BAD_FORMAT),
message: format!("Extrinsic has invalid format: {}", e),
data: None,
},
Error::Verification(e) => rpc::Error {
code: rpc::ErrorCode::ServerError(VERIFICATION_ERROR),
message: format!("Verification Error: {}", e),
data: Some(e.to_string().into()),
},
Error::Pool(PoolError::InvalidTransaction(InvalidTransaction::Custom(e))) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_INVALID_TX),
message: "Invalid Transaction".into(),
data: Some(format!("Custom error: {}", e).into()),
Error::BadFormat(e) => CallError::Custom(ErrorObject::owned(
BAD_FORMAT,
format!("Extrinsic has invalid format: {}", e),
None::<()>,
)),
Error::Verification(e) => CallError::Custom(ErrorObject::owned(
VERIFICATION_ERROR,
format!("Verification Error: {}", e),
Some(format!("{:?}", e)),
)),
Error::Pool(PoolError::InvalidTransaction(InvalidTransaction::Custom(e))) => {
CallError::Custom(ErrorObject::owned(
POOL_INVALID_TX,
"Invalid Transaction",
Some(format!("Custom error: {}", e)),
))
},
Error::Pool(PoolError::InvalidTransaction(e)) => {
let msg: &str = e.into();
rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_INVALID_TX),
message: "Invalid Transaction".into(),
data: Some(msg.into()),
}
CallError::Custom(ErrorObject::owned(
POOL_INVALID_TX,
"Invalid Transaction",
Some(msg),
))
},
Error::Pool(PoolError::UnknownTransaction(e)) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_UNKNOWN_VALIDITY),
message: "Unknown Transaction Validity".into(),
data: serde_json::to_value(e).ok(),
Error::Pool(PoolError::UnknownTransaction(e)) => {
CallError::Custom(ErrorObject::owned(
POOL_UNKNOWN_VALIDITY,
"Unknown Transaction Validity",
Some(format!("{:?}", e)),
))
},
Error::Pool(PoolError::TemporarilyBanned) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_TEMPORARILY_BANNED),
message: "Transaction is temporarily banned".into(),
data: None,
},
Error::Pool(PoolError::AlreadyImported(hash)) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_ALREADY_IMPORTED),
message: "Transaction Already Imported".into(),
data: Some(format!("{:?}", hash).into()),
},
Error::Pool(PoolError::TooLowPriority { old, new }) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_TOO_LOW_PRIORITY),
message: format!("Priority is too low: ({} vs {})", old, new),
data: Some("The transaction has too low priority to replace another transaction already in the pool.".into()),
},
Error::Pool(PoolError::CycleDetected) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_CYCLE_DETECTED),
message: "Cycle Detected".into(),
data: None,
},
Error::Pool(PoolError::ImmediatelyDropped) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_IMMEDIATELY_DROPPED),
message: "Immediately Dropped".into(),
data: Some("The transaction couldn't enter the pool because of the limit".into()),
},
Error::Pool(PoolError::Unactionable) => rpc::Error {
code: rpc::ErrorCode::ServerError(POOL_UNACTIONABLE),
message: "Unactionable".into(),
data: Some(
"The transaction is unactionable since it is not propagable and \
the local node does not author blocks".into(),
),
Error::Pool(PoolError::TemporarilyBanned) =>
CallError::Custom(ErrorObject::owned(
POOL_TEMPORARILY_BANNED,
"Transaction is temporarily banned",
None::<()>,
)),
Error::Pool(PoolError::AlreadyImported(hash)) =>
CallError::Custom(ErrorObject::owned(
POOL_ALREADY_IMPORTED,
"Transaction Already Imported",
Some(format!("{:?}", hash)),
)),
Error::Pool(PoolError::TooLowPriority { old, new }) => CallError::Custom(ErrorObject::owned(
POOL_TOO_LOW_PRIORITY,
format!("Priority is too low: ({} vs {})", old, new),
Some("The transaction has too low priority to replace another transaction already in the pool.")
)),
Error::Pool(PoolError::CycleDetected) =>
CallError::Custom(ErrorObject::owned(
POOL_CYCLE_DETECTED,
"Cycle Detected",
None::<()>
)),
Error::Pool(PoolError::ImmediatelyDropped) => CallError::Custom(ErrorObject::owned(
POOL_IMMEDIATELY_DROPPED,
"Immediately Dropped",
Some("The transaction couldn't enter the pool because of the limit"),
)),
Error::Pool(PoolError::Unactionable) => CallError::Custom(ErrorObject::owned(
POOL_UNACTIONABLE,
"Unactionable",
Some("The transaction is unactionable since it is not propagable and \
the local node does not author blocks")
)),
Error::Pool(PoolError::NoTagsProvided) => CallError::Custom(ErrorObject::owned(
POOL_NO_TAGS,
"No tags provided",
Some("Transaction does not provide any tags, so the pool can't identify it")
)),
Error::Pool(PoolError::InvalidBlockId(_)) =>
CallError::Custom(ErrorObject::owned(
POOL_INVALID_BLOCK_ID,
"The provided block ID is not valid",
None::<()>
)),
Error::Pool(PoolError::RejectedFutureTransaction) => {
CallError::Custom(ErrorObject::owned(
POOL_FUTURE_TX,
"The pool is not accepting future transactions",
None::<()>,
))
},
Error::UnsafeRpcCalled(e) => e.into(),
e => errors::internal(e),
}
e => CallError::Failed(e.into()),
}.into()
}
}
+23 -47
View File
@@ -18,85 +18,61 @@
//! Substrate block-author/full-node API.
pub mod error;
pub mod hash;
use self::error::{FutureResult, Result};
use jsonrpc_derive::rpc;
use jsonrpc_pubsub::{typed::Subscriber, SubscriptionId};
use jsonrpsee::{core::RpcResult, proc_macros::rpc};
use sc_transaction_pool_api::TransactionStatus;
use sp_core::Bytes;
pub use self::gen_client::Client as AuthorClient;
pub mod error;
pub mod hash;
/// Substrate authoring RPC API
#[rpc]
#[rpc(client, server)]
pub trait AuthorApi<Hash, BlockHash> {
/// RPC metadata
type Metadata;
/// Submit hex-encoded extrinsic for inclusion in block.
#[rpc(name = "author_submitExtrinsic")]
fn submit_extrinsic(&self, extrinsic: Bytes) -> FutureResult<Hash>;
#[method(name = "author_submitExtrinsic")]
async fn submit_extrinsic(&self, extrinsic: Bytes) -> RpcResult<Hash>;
/// Insert a key into the keystore.
#[rpc(name = "author_insertKey")]
fn insert_key(&self, key_type: String, suri: String, public: Bytes) -> Result<()>;
#[method(name = "author_insertKey")]
fn insert_key(&self, key_type: String, suri: String, public: Bytes) -> RpcResult<()>;
/// Generate new session keys and returns the corresponding public keys.
#[rpc(name = "author_rotateKeys")]
fn rotate_keys(&self) -> Result<Bytes>;
#[method(name = "author_rotateKeys")]
fn rotate_keys(&self) -> RpcResult<Bytes>;
/// Checks if the keystore has private keys for the given session public keys.
///
/// `session_keys` is the SCALE encoded session keys object from the runtime.
///
/// Returns `true` iff all private keys could be found.
#[rpc(name = "author_hasSessionKeys")]
fn has_session_keys(&self, session_keys: Bytes) -> Result<bool>;
#[method(name = "author_hasSessionKeys")]
fn has_session_keys(&self, session_keys: Bytes) -> RpcResult<bool>;
/// Checks if the keystore has private keys for the given public key and key type.
///
/// Returns `true` if a private key could be found.
#[rpc(name = "author_hasKey")]
fn has_key(&self, public_key: Bytes, key_type: String) -> Result<bool>;
#[method(name = "author_hasKey")]
fn has_key(&self, public_key: Bytes, key_type: String) -> RpcResult<bool>;
/// Returns all pending extrinsics, potentially grouped by sender.
#[rpc(name = "author_pendingExtrinsics")]
fn pending_extrinsics(&self) -> Result<Vec<Bytes>>;
#[method(name = "author_pendingExtrinsics")]
fn pending_extrinsics(&self) -> RpcResult<Vec<Bytes>>;
/// Remove given extrinsic from the pool and temporarily ban it to prevent reimporting.
#[rpc(name = "author_removeExtrinsic")]
#[method(name = "author_removeExtrinsic")]
fn remove_extrinsic(
&self,
bytes_or_hash: Vec<hash::ExtrinsicOrHash<Hash>>,
) -> Result<Vec<Hash>>;
) -> RpcResult<Vec<Hash>>;
/// Submit an extrinsic to watch.
///
/// See [`TransactionStatus`](sc_transaction_pool_api::TransactionStatus) for details on
/// transaction life cycle.
#[pubsub(
subscription = "author_extrinsicUpdate",
subscribe,
name = "author_submitAndWatchExtrinsic"
#[subscription(
name = "author_submitAndWatchExtrinsic" => "author_extrinsicUpdate",
unsubscribe = "author_unwatchExtrinsic",
item = TransactionStatus<Hash, BlockHash>,
)]
fn watch_extrinsic(
&self,
metadata: Self::Metadata,
subscriber: Subscriber<TransactionStatus<Hash, BlockHash>>,
bytes: Bytes,
);
/// Unsubscribe from extrinsic watching.
#[pubsub(
subscription = "author_extrinsicUpdate",
unsubscribe,
name = "author_unwatchExtrinsic"
)]
fn unwatch_extrinsic(
&self,
metadata: Option<Self::Metadata>,
id: SubscriptionId,
) -> Result<bool>;
fn watch_extrinsic(&self, bytes: Bytes);
}