jsonrpsee integration (#8783)

* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

substrate/bin/node/rpc/src/lib.rs

@@ -29,18 +29,20 @@
 //! be placed here or imported from corresponding FRAME RPC definitions.
 
 #![warn(missing_docs)]
+#![warn(unused_crate_dependencies)]
 
 use std::sync::Arc;
 
+use jsonrpsee::RpcModule;
 use node_primitives::{AccountId, Balance, Block, BlockNumber, Hash, Index};
 use sc_client_api::AuxStore;
 use sc_consensus_babe::{Config, Epoch};
-use sc_consensus_babe_rpc::BabeRpcHandler;
+use sc_consensus_babe_rpc::BabeRpc;
 use sc_consensus_epochs::SharedEpochChanges;
 use sc_finality_grandpa::{
 	FinalityProofProvider, GrandpaJustificationStream, SharedAuthoritySet, SharedVoterState,
 };
-use sc_finality_grandpa_rpc::GrandpaRpcHandler;
+use sc_finality_grandpa_rpc::GrandpaRpc;
 use sc_rpc::SubscriptionTaskExecutor;
 pub use sc_rpc_api::DenyUnsafe;
 use sc_transaction_pool_api::TransactionPool;
@@ -93,14 +95,11 @@ pub struct FullDeps<C, P, SC, B> {
 	pub grandpa: GrandpaDeps<B>,
 }
 
-/// A IO handler that uses all Full RPC extensions.
-pub type IoHandler = jsonrpc_core::IoHandler<sc_rpc::Metadata>;
-
 /// Instantiate all Full RPC extensions.
 pub fn create_full<C, P, SC, B>(
 	deps: FullDeps<C, P, SC, B>,
 	backend: Arc<B>,
-) -> Result<jsonrpc_core::IoHandler<sc_rpc_api::Metadata>, Box<dyn std::error::Error + Send + Sync>>
+) -> Result<RpcModule<()>, Box<dyn std::error::Error + Send + Sync>>
 where
 	C: ProvideRuntimeApi<Block>
 		+ sc_client_api::BlockBackend<Block>
@@ -121,13 +120,17 @@ where
 	B: sc_client_api::Backend<Block> + Send + Sync + 'static,
 	B::State: sc_client_api::backend::StateBackend<sp_runtime::traits::HashFor<Block>>,
 {
-	use pallet_contracts_rpc::{Contracts, ContractsApi};
-	use pallet_mmr_rpc::{Mmr, MmrApi};
-	use pallet_transaction_payment_rpc::{TransactionPayment, TransactionPaymentApi};
-	use sc_rpc::dev::{Dev, DevApi};
-	use substrate_frame_rpc_system::{FullSystem, SystemApi};
+	use pallet_contracts_rpc::{ContractsApiServer, ContractsRpc};
+	use pallet_mmr_rpc::{MmrApiServer, MmrRpc};
+	use pallet_transaction_payment_rpc::{TransactionPaymentApiServer, TransactionPaymentRpc};
+	use sc_consensus_babe_rpc::BabeApiServer;
+	use sc_finality_grandpa_rpc::GrandpaApiServer;
+	use sc_rpc::dev::{Dev, DevApiServer};
+	use sc_sync_state_rpc::{SyncStateRpc, SyncStateRpcApiServer};
+	use substrate_frame_rpc_system::{SystemApiServer, SystemRpc};
+	use substrate_state_trie_migration_rpc::StateMigrationApiServer;
 
-	let mut io = jsonrpc_core::IoHandler::default();
+	let mut io = RpcModule::new(());
 	let FullDeps { client, pool, select_chain, chain_spec, deny_unsafe, babe, grandpa } = deps;
 
 	let BabeDeps { keystore, babe_config, shared_epoch_changes } = babe;
@@ -139,40 +142,45 @@ where
 		finality_provider,
 	} = grandpa;
 
-	io.extend_with(SystemApi::to_delegate(FullSystem::new(client.clone(), pool, deny_unsafe)));
+	io.merge(SystemRpc::new(client.clone(), pool, deny_unsafe).into_rpc())?;
 	// Making synchronous calls in light client freezes the browser currently,
 	// more context: https://github.com/paritytech/substrate/pull/3480
 	// These RPCs should use an asynchronous caller instead.
-	io.extend_with(ContractsApi::to_delegate(Contracts::new(client.clone())));
-	io.extend_with(MmrApi::to_delegate(Mmr::new(client.clone())));
-	io.extend_with(TransactionPaymentApi::to_delegate(TransactionPayment::new(client.clone())));
-	io.extend_with(sc_consensus_babe_rpc::BabeApi::to_delegate(BabeRpcHandler::new(
-		client.clone(),
-		shared_epoch_changes.clone(),
-		keystore,
-		babe_config,
-		select_chain,
-		deny_unsafe,
-	)));
-	io.extend_with(sc_finality_grandpa_rpc::GrandpaApi::to_delegate(GrandpaRpcHandler::new(
-		shared_authority_set.clone(),
-		shared_voter_state,
-		justification_stream,
-		subscription_executor,
-		finality_provider,
-	)));
-	io.extend_with(substrate_state_trie_migration_rpc::StateMigrationApi::to_delegate(
-		substrate_state_trie_migration_rpc::MigrationRpc::new(client.clone(), backend, deny_unsafe),
-	));
-	io.extend_with(sc_sync_state_rpc::SyncStateRpcApi::to_delegate(
-		sc_sync_state_rpc::SyncStateRpcHandler::new(
-			chain_spec,
+	io.merge(ContractsRpc::new(client.clone()).into_rpc())?;
+	io.merge(MmrRpc::new(client.clone()).into_rpc())?;
+	io.merge(TransactionPaymentRpc::new(client.clone()).into_rpc())?;
+	io.merge(
+		BabeRpc::new(
 			client.clone(),
-			shared_authority_set,
-			shared_epoch_changes,
-		)?,
-	));
-	io.extend_with(DevApi::to_delegate(Dev::new(client, deny_unsafe)));
+			shared_epoch_changes.clone(),
+			keystore,
+			babe_config,
+			select_chain,
+			deny_unsafe,
+		)
+		.into_rpc(),
+	)?;
+	io.merge(
+		GrandpaRpc::new(
+			subscription_executor,
+			shared_authority_set.clone(),
+			shared_voter_state,
+			justification_stream,
+			finality_provider,
+		)
+		.into_rpc(),
+	)?;
+
+	io.merge(
+		SyncStateRpc::new(chain_spec, client.clone(), shared_authority_set, shared_epoch_changes)?
+			.into_rpc(),
+	)?;
+
+	io.merge(
+		substrate_state_trie_migration_rpc::MigrationRpc::new(client.clone(), backend, deny_unsafe)
+			.into_rpc(),
+	)?;
+	io.merge(Dev::new(client, deny_unsafe).into_rpc())?;
 
 	Ok(io)
 }