jsonrpsee integration (#8783)

* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

substrate/frame/contracts/rpc/src/lib.rs

@@ -17,11 +17,16 @@
 
 //! Node-specific RPC methods for interaction with contracts.
 
-use std::sync::Arc;
+#![warn(unused_crate_dependencies)]
+
+use std::{marker::PhantomData, sync::Arc};
 
 use codec::Codec;
-use jsonrpc_core::{Error, ErrorCode, Result};
-use jsonrpc_derive::rpc;
+use jsonrpsee::{
+	core::{async_trait, Error as JsonRpseeError, RpcResult},
+	proc_macros::rpc,
+	types::error::{CallError, ErrorCode, ErrorObject},
+};
 use pallet_contracts_primitives::{
 	Code, CodeUploadResult, ContractExecResult, ContractInstantiateResult,
 };
@@ -37,8 +42,8 @@ use sp_runtime::{
 
 pub use pallet_contracts_rpc_runtime_api::ContractsApi as ContractsRuntimeApi;
 
-const RUNTIME_ERROR: i64 = 1;
-const CONTRACT_DOESNT_EXIST: i64 = 2;
+const RUNTIME_ERROR: i32 = 1;
+const CONTRACT_DOESNT_EXIST: i32 = 2;
 
 pub type Weight = u64;
 
@@ -58,15 +63,17 @@ const GAS_LIMIT: Weight = 5 * GAS_PER_SECOND;
 
 /// A private newtype for converting `ContractAccessError` into an RPC error.
 struct ContractAccessError(pallet_contracts_primitives::ContractAccessError);
-impl From<ContractAccessError> for Error {
-	fn from(e: ContractAccessError) -> Error {
+
+impl From<ContractAccessError> for JsonRpseeError {
+	fn from(e: ContractAccessError) -> Self {
 		use pallet_contracts_primitives::ContractAccessError::*;
 		match e.0 {
-			DoesntExist => Error {
-				code: ErrorCode::ServerError(CONTRACT_DOESNT_EXIST),
-				message: "The specified contract doesn't exist.".into(),
-				data: None,
-			},
+			DoesntExist => CallError::Custom(ErrorObject::owned(
+				CONTRACT_DOESNT_EXIST,
+				"The specified contract doesn't exist.",
+				None::<()>,
+			))
+			.into(),
 		}
 	}
 }
@@ -109,7 +116,7 @@ pub struct CodeUploadRequest<AccountId> {
 }
 
 /// Contracts RPC methods.
-#[rpc]
+#[rpc(client, server)]
 pub trait ContractsApi<BlockHash, BlockNumber, AccountId, Balance, Hash>
 where
 	Balance: Copy + TryFrom<NumberOrHex> + Into<NumberOrHex>,
@@ -121,12 +128,12 @@ where
 	///
 	/// This method is useful for calling getter-like methods on contracts or to dry-run a
 	/// a contract call in order to determine the `gas_limit`.
-	#[rpc(name = "contracts_call")]
+	#[method(name = "contracts_call")]
 	fn call(
 		&self,
 		call_request: CallRequest<AccountId>,
 		at: Option<BlockHash>,
-	) -> Result<ContractExecResult<Balance>>;
+	) -> RpcResult<ContractExecResult<Balance>>;
 
 	/// Instantiate a new contract.
 	///
@@ -134,12 +141,12 @@ where
 	/// is not actually created.
 	///
 	/// This method is useful for UIs to dry-run contract instantiations.
-	#[rpc(name = "contracts_instantiate")]
+	#[method(name = "contracts_instantiate")]
 	fn instantiate(
 		&self,
 		instantiate_request: InstantiateRequest<AccountId, Hash>,
 		at: Option<BlockHash>,
-	) -> Result<ContractInstantiateResult<AccountId, Balance>>;
+	) -> RpcResult<ContractInstantiateResult<AccountId, Balance>>;
 
 	/// Upload new code without instantiating a contract from it.
 	///
@@ -147,48 +154,50 @@ where
 	/// won't change any state.
 	///
 	/// This method is useful for UIs to dry-run code upload.
-	#[rpc(name = "contracts_upload_code")]
+	#[method(name = "contracts_upload_code")]
 	fn upload_code(
 		&self,
 		upload_request: CodeUploadRequest<AccountId>,
 		at: Option<BlockHash>,
-	) -> Result<CodeUploadResult<Hash, Balance>>;
+	) -> RpcResult<CodeUploadResult<Hash, Balance>>;
 
 	/// Returns the value under a specified storage `key` in a contract given by `address` param,
 	/// or `None` if it is not set.
-	#[rpc(name = "contracts_getStorage")]
+	#[method(name = "contracts_getStorage")]
 	fn get_storage(
 		&self,
 		address: AccountId,
 		key: H256,
 		at: Option<BlockHash>,
-	) -> Result<Option<Bytes>>;
+	) -> RpcResult<Option<Bytes>>;
 }
 
-/// An implementation of contract specific RPC methods.
-pub struct Contracts<C, B> {
-	client: Arc<C>,
-	_marker: std::marker::PhantomData<B>,
+/// Contracts RPC methods.
+pub struct ContractsRpc<Client, Block> {
+	client: Arc<Client>,
+	_marker: PhantomData<Block>,
 }
 
-impl<C, B> Contracts<C, B> {
+impl<Client, Block> ContractsRpc<Client, Block> {
 	/// Create new `Contracts` with the given reference to the client.
-	pub fn new(client: Arc<C>) -> Self {
-		Contracts { client, _marker: Default::default() }
+	pub fn new(client: Arc<Client>) -> Self {
+		Self { client, _marker: Default::default() }
 	}
 }
-impl<C, Block, AccountId, Balance, Hash>
-	ContractsApi<
+
+#[async_trait]
+impl<Client, Block, AccountId, Balance, Hash>
+	ContractsApiServer<
 		<Block as BlockT>::Hash,
 		<<Block as BlockT>::Header as HeaderT>::Number,
 		AccountId,
 		Balance,
 		Hash,
-	> for Contracts<C, Block>
+	> for ContractsRpc<Client, Block>
 where
 	Block: BlockT,
-	C: Send + Sync + 'static + ProvideRuntimeApi<Block> + HeaderBackend<Block>,
-	C::Api: ContractsRuntimeApi<
+	Client: Send + Sync + 'static + ProvideRuntimeApi<Block> + HeaderBackend<Block>,
+	Client::Api: ContractsRuntimeApi<
 		Block,
 		AccountId,
 		Balance,
@@ -203,7 +212,7 @@ where
 		&self,
 		call_request: CallRequest<AccountId>,
 		at: Option<<Block as BlockT>::Hash>,
-	) -> Result<ContractExecResult<Balance>> {
+	) -> RpcResult<ContractExecResult<Balance>> {
 		let api = self.client.runtime_api();
 		let at = BlockId::hash(at.unwrap_or_else(||
 			// If the block hash is not supplied assume the best block.
@@ -226,7 +235,7 @@ where
 		&self,
 		instantiate_request: InstantiateRequest<AccountId, Hash>,
 		at: Option<<Block as BlockT>::Hash>,
-	) -> Result<ContractInstantiateResult<AccountId, Balance>> {
+	) -> RpcResult<ContractInstantiateResult<AccountId, Balance>> {
 		let api = self.client.runtime_api();
 		let at = BlockId::hash(at.unwrap_or_else(||
 			// If the block hash is not supplied assume the best block.
@@ -265,7 +274,7 @@ where
 		&self,
 		upload_request: CodeUploadRequest<AccountId>,
 		at: Option<<Block as BlockT>::Hash>,
-	) -> Result<CodeUploadResult<Hash, Balance>> {
+	) -> RpcResult<CodeUploadResult<Hash, Balance>> {
 		let api = self.client.runtime_api();
 		let at = BlockId::hash(at.unwrap_or_else(||
 			// If the block hash is not supplied assume the best block.
@@ -285,12 +294,9 @@ where
 		address: AccountId,
 		key: H256,
 		at: Option<<Block as BlockT>::Hash>,
-	) -> Result<Option<Bytes>> {
+	) -> RpcResult<Option<Bytes>> {
 		let api = self.client.runtime_api();
-		let at = BlockId::hash(at.unwrap_or_else(||
-			// If the block hash is not supplied assume the best block.
-			self.client.info().best_hash));
-
+		let at = BlockId::hash(at.unwrap_or_else(|| self.client.info().best_hash));
 		let result = api
 			.get_storage(&at, address, key.into())
 			.map_err(runtime_error_into_rpc_err)?
@@ -302,32 +308,35 @@ where
 }
 
 /// Converts a runtime trap into an RPC error.
-fn runtime_error_into_rpc_err(err: impl std::fmt::Display) -> Error {
-	Error {
-		code: ErrorCode::ServerError(RUNTIME_ERROR),
-		message: "Runtime error".into(),
-		data: Some(err.to_string().into()),
-	}
+fn runtime_error_into_rpc_err(err: impl std::fmt::Debug) -> JsonRpseeError {
+	CallError::Custom(ErrorObject::owned(
+		RUNTIME_ERROR,
+		"Runtime error",
+		Some(format!("{:?}", err)),
+	))
+	.into()
 }
 
-fn decode_hex<H: std::fmt::Debug + Copy, T: TryFrom<H>>(from: H, name: &str) -> Result<T> {
-	from.try_into().map_err(|_| Error {
-		code: ErrorCode::InvalidParams,
-		message: format!("{:?} does not fit into the {} type", from, name),
-		data: None,
+fn decode_hex<H: std::fmt::Debug + Copy, T: TryFrom<H>>(from: H, name: &str) -> RpcResult<T> {
+	from.try_into().map_err(|_| {
+		JsonRpseeError::Call(CallError::Custom(ErrorObject::owned(
+			ErrorCode::InvalidParams.code(),
+			format!("{:?} does not fit into the {} type", from, name),
+			None::<()>,
+		)))
 	})
 }
 
-fn limit_gas(gas_limit: Weight) -> Result<()> {
+fn limit_gas(gas_limit: Weight) -> RpcResult<()> {
 	if gas_limit > GAS_LIMIT {
-		Err(Error {
-			code: ErrorCode::InvalidParams,
-			message: format!(
+		Err(JsonRpseeError::Call(CallError::Custom(ErrorObject::owned(
+			ErrorCode::InvalidParams.code(),
+			format!(
 				"Requested gas limit is greater than maximum allowed: {} > {}",
 				gas_limit, GAS_LIMIT
 			),
-			data: None,
-		})
+			None::<()>,
+		))))
 	} else {
 		Ok(())
 	}
@@ -336,6 +345,7 @@ fn limit_gas(gas_limit: Weight) -> Result<()> {
 #[cfg(test)]
 mod tests {
 	use super::*;
+	use pallet_contracts_primitives::{ContractExecResult, ContractInstantiateResult};
 	use sp_core::U256;
 
 	fn trim(json: &str) -> String {