jsonrpsee integration (#8783)

* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

substrate/client/rpc-api/src/system/error.rs

@@ -19,7 +19,10 @@
 //! System RPC module errors.
 
 use crate::system::helpers::Health;
-use jsonrpc_core as rpc;
+use jsonrpsee::{
+	core::Error as JsonRpseeError,
+	types::error::{CallError, ErrorObject},
+};
 
 /// System RPC Result type.
 pub type Result<T> = std::result::Result<T, Error>;
@@ -35,22 +38,24 @@ pub enum Error {
 	MalformattedPeerArg(String),
 }
 
-/// Base code for all system errors.
-const BASE_ERROR: i64 = 2000;
+// Base code for all system errors.
+const BASE_ERROR: i32 = 2000;
+// Provided block range couldn't be resolved to a list of blocks.
+const NOT_HEALTHY_ERROR: i32 = BASE_ERROR + 1;
+// Peer argument is malformatted.
+const MALFORMATTED_PEER_ARG_ERROR: i32 = BASE_ERROR + 2;
 
-impl From<Error> for rpc::Error {
+impl From<Error> for JsonRpseeError {
 	fn from(e: Error) -> Self {
 		match e {
-			Error::NotHealthy(ref h) => rpc::Error {
-				code: rpc::ErrorCode::ServerError(BASE_ERROR + 1),
-				message: format!("{}", e),
-				data: serde_json::to_value(h).ok(),
-			},
-			Error::MalformattedPeerArg(ref e) => rpc::Error {
-				code: rpc::ErrorCode::ServerError(BASE_ERROR + 2),
-				message: e.clone(),
-				data: None,
-			},
+			Error::NotHealthy(ref h) =>
+				CallError::Custom(ErrorObject::owned(NOT_HEALTHY_ERROR, e.to_string(), Some(h))),
+			Error::MalformattedPeerArg(e) => CallError::Custom(ErrorObject::owned(
+				MALFORMATTED_PEER_ARG_ERROR + 2,
+				e,
+				None::<()>,
+			)),
 		}
+		.into()
 	}
 }

substrate/client/rpc-api/src/system/mod.rs

@@ -18,65 +18,61 @@
 
 //! Substrate system API.
 
+use jsonrpsee::{
+	core::{JsonValue, RpcResult},
+	proc_macros::rpc,
+};
+
+pub use self::helpers::{Health, NodeRole, PeerInfo, SyncState, SystemInfo};
+
 pub mod error;
 pub mod helpers;
 
-use crate::helpers::Receiver;
-use jsonrpc_core::BoxFuture;
-use jsonrpc_derive::rpc;
-
-use self::error::Result as SystemResult;
-
-pub use self::{
-	gen_client::Client as SystemClient,
-	helpers::{Health, NodeRole, PeerInfo, SyncState, SystemInfo},
-};
-
 /// Substrate system RPC API
-#[rpc]
+#[rpc(client, server)]
 pub trait SystemApi<Hash, Number> {
 	/// Get the node's implementation name. Plain old string.
-	#[rpc(name = "system_name")]
-	fn system_name(&self) -> SystemResult<String>;
+	#[method(name = "system_name")]
+	fn system_name(&self) -> RpcResult<String>;
 
 	/// Get the node implementation's version. Should be a semver string.
-	#[rpc(name = "system_version")]
-	fn system_version(&self) -> SystemResult<String>;
+	#[method(name = "system_version")]
+	fn system_version(&self) -> RpcResult<String>;
 
 	/// Get the chain's name. Given as a string identifier.
-	#[rpc(name = "system_chain")]
-	fn system_chain(&self) -> SystemResult<String>;
+	#[method(name = "system_chain")]
+	fn system_chain(&self) -> RpcResult<String>;
 
 	/// Get the chain's type.
-	#[rpc(name = "system_chainType")]
-	fn system_type(&self) -> SystemResult<sc_chain_spec::ChainType>;
+	#[method(name = "system_chainType")]
+	fn system_type(&self) -> RpcResult<sc_chain_spec::ChainType>;
 
 	/// Get a custom set of properties as a JSON object, defined in the chain spec.
-	#[rpc(name = "system_properties")]
-	fn system_properties(&self) -> SystemResult<sc_chain_spec::Properties>;
+	#[method(name = "system_properties")]
+	fn system_properties(&self) -> RpcResult<sc_chain_spec::Properties>;
 
 	/// Return health status of the node.
 	///
 	/// Node is considered healthy if it is:
 	/// - connected to some peers (unless running in dev mode)
 	/// - not performing a major sync
-	#[rpc(name = "system_health", returns = "Health")]
-	fn system_health(&self) -> Receiver<Health>;
+	#[method(name = "system_health")]
+	async fn system_health(&self) -> RpcResult<Health>;
 
 	/// Returns the base58-encoded PeerId of the node.
-	#[rpc(name = "system_localPeerId", returns = "String")]
-	fn system_local_peer_id(&self) -> Receiver<String>;
+	#[method(name = "system_localPeerId")]
+	async fn system_local_peer_id(&self) -> RpcResult<String>;
 
-	/// Returns the multiaddresses that the local node is listening on
+	/// Returns the multi-addresses that the local node is listening on
 	///
 	/// The addresses include a trailing `/p2p/` with the local PeerId, and are thus suitable to
-	/// be passed to `system_addReservedPeer` or as a bootnode address for example.
-	#[rpc(name = "system_localListenAddresses", returns = "Vec<String>")]
-	fn system_local_listen_addresses(&self) -> Receiver<Vec<String>>;
+	/// be passed to `addReservedPeer` or as a bootnode address for example.
+	#[method(name = "system_localListenAddresses")]
+	async fn system_local_listen_addresses(&self) -> RpcResult<Vec<String>>;
 
 	/// Returns currently connected peers
-	#[rpc(name = "system_peers", returns = "Vec<PeerInfo<Hash, Number>>")]
-	fn system_peers(&self) -> BoxFuture<jsonrpc_core::Result<Vec<PeerInfo<Hash, Number>>>>;
+	#[method(name = "system_peers")]
+	async fn system_peers(&self) -> RpcResult<Vec<PeerInfo<Hash, Number>>>;
 
 	/// Returns current state of the network.
 	///
@@ -84,47 +80,44 @@ pub trait SystemApi<Hash, Number> {
 	/// as its format might change at any time.
 	// TODO: the future of this call is uncertain: https://github.com/paritytech/substrate/issues/1890
 	// https://github.com/paritytech/substrate/issues/5541
-	#[rpc(name = "system_unstable_networkState", returns = "jsonrpc_core::Value")]
-	fn system_network_state(&self) -> BoxFuture<jsonrpc_core::Result<jsonrpc_core::Value>>;
+	#[method(name = "system_unstable_networkState")]
+	async fn system_network_state(&self) -> RpcResult<JsonValue>;
 
 	/// Adds a reserved peer. Returns the empty string or an error. The string
 	/// parameter should encode a `p2p` multiaddr.
 	///
 	/// `/ip4/198.51.100.19/tcp/30333/p2p/QmSk5HQbn6LhUwDiNMseVUjuRYhEtYj4aUZ6WfWoGURpdV`
 	/// is an example of a valid, passing multiaddr with PeerId attached.
-	#[rpc(name = "system_addReservedPeer", returns = "()")]
-	fn system_add_reserved_peer(&self, peer: String) -> BoxFuture<Result<(), jsonrpc_core::Error>>;
+	#[method(name = "system_addReservedPeer")]
+	async fn system_add_reserved_peer(&self, peer: String) -> RpcResult<()>;
 
 	/// Remove a reserved peer. Returns the empty string or an error. The string
 	/// should encode only the PeerId e.g. `QmSk5HQbn6LhUwDiNMseVUjuRYhEtYj4aUZ6WfWoGURpdV`.
-	#[rpc(name = "system_removeReservedPeer", returns = "()")]
-	fn system_remove_reserved_peer(
-		&self,
-		peer_id: String,
-	) -> BoxFuture<Result<(), jsonrpc_core::Error>>;
+	#[method(name = "system_removeReservedPeer")]
+	async fn system_remove_reserved_peer(&self, peer_id: String) -> RpcResult<()>;
 
 	/// Returns the list of reserved peers
-	#[rpc(name = "system_reservedPeers", returns = "Vec<String>")]
-	fn system_reserved_peers(&self) -> Receiver<Vec<String>>;
+	#[method(name = "system_reservedPeers")]
+	async fn system_reserved_peers(&self) -> RpcResult<Vec<String>>;
 
 	/// Returns the roles the node is running as.
-	#[rpc(name = "system_nodeRoles", returns = "Vec<NodeRole>")]
-	fn system_node_roles(&self) -> Receiver<Vec<NodeRole>>;
+	#[method(name = "system_nodeRoles")]
+	async fn system_node_roles(&self) -> RpcResult<Vec<NodeRole>>;
 
 	/// Returns the state of the syncing of the node: starting block, current best block, highest
 	/// known block.
-	#[rpc(name = "system_syncState", returns = "SyncState<Number>")]
-	fn system_sync_state(&self) -> Receiver<SyncState<Number>>;
+	#[method(name = "system_syncState")]
+	async fn system_sync_state(&self) -> RpcResult<SyncState<Number>>;
 
 	/// Adds the supplied directives to the current log filter
 	///
 	/// The syntax is identical to the CLI `<target>=<level>`:
 	///
 	/// `sync=debug,state=trace`
-	#[rpc(name = "system_addLogFilter", returns = "()")]
-	fn system_add_log_filter(&self, directives: String) -> Result<(), jsonrpc_core::Error>;
+	#[method(name = "system_addLogFilter")]
+	fn system_add_log_filter(&self, directives: String) -> RpcResult<()>;
 
 	/// Resets the log filter to Substrate defaults
-	#[rpc(name = "system_resetLogFilter", returns = "()")]
-	fn system_reset_log_filter(&self) -> Result<(), jsonrpc_core::Error>;
+	#[method(name = "system_resetLogFilter")]
+	fn system_reset_log_filter(&self) -> RpcResult<()>;
 }