jsonrpsee integration (#8783)

* Add tokio * No need to map CallError to CallError * jsonrpsee proc macros (#9673) * port error types to `JsonRpseeError` * migrate chain module to proc macro api * make it compile with proc macros * update branch * update branch * update to jsonrpsee master * port system rpc * port state rpc * port childstate & offchain * frame system rpc * frame transaction payment * bring back CORS hack to work with polkadot UI * port babe rpc * port manual seal rpc * port frame mmr rpc * port frame contracts rpc * port finality grandpa rpc * port sync state rpc * resolve a few TODO + no jsonrpc deps * Update bin/node/rpc-client/src/main.rs * Update bin/node/rpc-client/src/main.rs * Update bin/node/rpc-client/src/main.rs * Update bin/node/rpc-client/src/main.rs * Port over system_ rpc tests * Make it compile * Use prost 0.8 * Use prost 0.8 * Make it compile * Ignore more failing tests * Comment out WIP tests * fix nit in frame system api * Update lockfile * No more juggling tokio versions * No more wait_for_stop ? * Remove browser-testing * Arguments must be arrays * Use same argument names * Resolve todo: no wait_for_stop for WS server Add todo: is parse_rpc_result used? Cleanup imports * fmt * log * One test passes * update jsonrpsee * update jsonrpsee * cleanup rpc-servers crate * jsonrpsee: add host and origin filtering (#9787) * add access control in the jsonrpsee servers * use master * fix nits * rpc runtime_version safe * fix nits * fix grumbles * remove unused files * resolve some todos * jsonrpsee more cleanup (#9803) * more cleanup * resolve TODOs * fix some unwraps * remove type hints * update jsonrpsee * downgrade zeroize * pin jsonrpsee rev * remove unwrap nit * Comment out more tests that aren't ported * Comment out more tests * Fix tests after merge * Subscription test * Invalid nonce test * Pending exts * WIP removeExtrinsic test * Test remove_extrinsic * Make state test: should_return_storage work * Uncomment/fix the other non-subscription related state tests * test: author_insertKey * test: author_rotateKeys * Get rest of state tests passing * asyncify a little more * Add todo to note #msg change * Crashing test for has_session_keys * Fix error conversion to avoid stack overflows Port author_hasSessionKeys test fmt * test author_hasKey * Add two missing tests Add a check on the return type Add todos for James's concerns * RPC tests for state, author and system (#9859) * Fix test runner * Impl Default for SubscriptionTaskExecutor * Keep the minimul amount of code needed to compile tests * Re-instate `RpcSession` (for now) * cleanup * Port over RPC tests * Add tokio * No need to map CallError to CallError * Port over system_ rpc tests * Make it compile * Use prost 0.8 * Use prost 0.8 * Make it compile * Ignore more failing tests * Comment out WIP tests * Update lockfile * No more juggling tokio versions * No more wait_for_stop ? * Remove browser-testing * Arguments must be arrays * Use same argument names * Resolve todo: no wait_for_stop for WS server Add todo: is parse_rpc_result used? Cleanup imports * fmt * log * One test passes * Comment out more tests that aren't ported * Comment out more tests * Fix tests after merge * Subscription test * Invalid nonce test * Pending exts * WIP removeExtrinsic test * Test remove_extrinsic * Make state test: should_return_storage work * Uncomment/fix the other non-subscription related state tests * test: author_insertKey * test: author_rotateKeys * Get rest of state tests passing * asyncify a little more * Add todo to note #msg change * Crashing test for has_session_keys * Fix error conversion to avoid stack overflows Port author_hasSessionKeys test fmt * test author_hasKey * Add two missing tests Add a check on the return type Add todos for James's concerns * offchain rpc tests * Address todos * fmt Co-authored-by: James Wilson <james@jsdw.me> * fix drop in state test * update jsonrpsee * fix ignored system test * fix chain tests * remove some boiler plate * Port BEEFY RPC (#9883) * Merge master * Port beefy RPC (ty @niklas!) * trivial changes left over from merge * Remove unused code * Update jsonrpsee * fix build * make tests compile again * beefy update jsonrpsee * fix: respect rpc methods policy * update cargo.lock * update jsonrpsee * update jsonrpsee * downgrade error logs * update jsonrpsee * Fix typo * remove unused file * Better name * Port Babe RPC tests * Put docs back * Resolve todo * Port tests for System RPCs * Resolve todo * fix build * Updated jsonrpsee to current master * fix: port finality grandpa rpc tests * Move .into() outside of the match * more review grumbles * jsonrpsee: add `rpc handlers` back (#10245) * add back RpcHandlers * cargo fmt * fix docs * fix grumble: remove needless alloc * resolve TODO * fmt * Fix typo * grumble: Use constants based on BASE_ERROR * grumble: DRY whitelisted listening addresses grumble: s/JSONRPC/JSON-RPC/ * cleanup * grumbles: Making readers aware of the possibility of gaps * review grumbles * grumbles * remove notes from niklasad1 * Update `jsonrpsee` * fix: jsonrpsee features * jsonrpsee: fallback to random port in case the specified port failed (#10304) * jsonrpsee: fallback to random port * better comment * Update client/rpc-servers/src/lib.rs Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com> * Update client/rpc-servers/src/lib.rs Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com> * address grumbles * cargo fmt * addrs already slice Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com> * Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae * lockfile * update jsonrpsee * fix warning * Don't fetch jsonrpsee from crates * make tests compile again * fix rpc tests * remove unused deps * update tokio * fix rpc tests again * fix: test runner `HttpServerBuilder::builder` fails unless it's called within tokio runtime * cargo fmt * grumbles: fix subscription aliases * make clippy happy * update remaining subscriptions alias * cleanup * cleanup * fix chain subscription: less boiler plate (#10285) * fix chain subscription: less boiler plate * fix bad merge * cargo fmt * Switch to jsonrpsee 0.5 * fix build * add missing features * fix nit: remove needless Box::pin * Integrate jsonrpsee metrics (#10395) * draft metrics impl * Use latest api * Add missing file * Http server metrics * cleanup * bump jsonrpsee * Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics. * fix build * remove needless Arc::clone * Update to jsonrpsee 0.6 * lolz * fix metrics * Revert "lolz" This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1. * fix: in-memory rpc support subscriptions * commit Cargo.lock * Update tests to 0.7 * fix TODOs * ws server: generate subscriptionIDs as Strings Some libraries seems to expect the subscription IDs to be Strings, let's not break this in this PR. * Increase timeout * Port over tests * cleanup * Using error codes from the spec * fix clippy * cargo fmt * update jsonrpsee * fix nits * fix: rpc_query * enable custom subid gen through spawn_tasks * remove unsed deps * unify tokio deps * Revert "enable custom subid gen through spawn_tasks" This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470. * fix bad merge of `test-utils` * fix more nits * downgrade wasm-instrument to 0.1.0 * [jsonrpsee]: enable custom RPC subscription ID generatation (#10731) * enable custom subid gen through spawn_tasks * fix nits * Update client/service/src/builder.rs Co-authored-by: David <dvdplm@gmail.com> * add Poc; needs jsonrpsee pr * update jsonrpsee * add re-exports * add docs Co-authored-by: David <dvdplm@gmail.com> * cargo fmt * fmt * port RPC-API dev * Remove unused file * fix nit: remove async trait * fix doc links * fix merge nit: remove jsonrpc deps * kill namespace on rpc apis * companion for jsonrpsee v0.10 (#11158) * companion for jsonrpsee v0.10 * update versions v0.10.0 * add some fixes * spelling * fix spaces Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> * send error before subs are closed * fix unsubscribe method names: chain * fix tests * jsonrpc server: print binded local address * grumbles: kill SubscriptionTaskExecutor * Update client/sync-state-rpc/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/rpc/src/chain/chain_full.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update client/rpc/src/chain/chain_full.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * sync-state-rpc: kill anyhow * no more anyhow * remove todo * jsonrpsee: fix bad params in subscriptions. (#11251) * update jsonrpsee * fix error responses * revert error codes * dont do weird stuff in drop impl * rpc servers: remove needless clone * Remove silly constants * chore: update jsonrpsee v0.12 * commit Cargo.lock * deps: downgrade git2 * feat: CLI flag max subscriptions per connection * metrics: use old logging format * fix: read WS address from substrate output (#11379) Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> Co-authored-by: James Wilson <james@jsdw.me> Co-authored-by: Maciej Hirsz <hello@maciej.codes> Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2026-07-06 11:27:25 +00:00 · 2022-05-10 10:52:19 +02:00
parent e45d53552d
commit 29c0c6a4a8
93 changed files with 3813 additions and 5094 deletions
@@ -23,26 +23,27 @@ mod tests;

 use std::sync::Arc;

-use sp_blockchain::HeaderBackend;
+use crate::SubscriptionTaskExecutor;

 use codec::{Decode, Encode};
-use futures::{
-	future::{FutureExt, TryFutureExt},
-	SinkExt, StreamExt as _,
+use futures::{FutureExt, TryFutureExt};
+use jsonrpsee::{
+	core::{async_trait, Error as JsonRpseeError, RpcResult},
+	PendingSubscription,
 };
-use jsonrpc_pubsub::{manager::SubscriptionManager, typed::Subscriber, SubscriptionId};
 use sc_rpc_api::DenyUnsafe;
 use sc_transaction_pool_api::{
 	error::IntoPoolError, BlockHash, InPoolTransaction, TransactionFor, TransactionPool,
-	TransactionSource, TransactionStatus, TxHash,
+	TransactionSource, TxHash,
 };
 use sp_api::ProvideRuntimeApi;
+use sp_blockchain::HeaderBackend;
 use sp_core::Bytes;
 use sp_keystore::{SyncCryptoStore, SyncCryptoStorePtr};
 use sp_runtime::{generic, traits::Block as BlockT};
 use sp_session::SessionKeys;

-use self::error::{Error, FutureResult, Result};
+use self::error::{Error, Result};
 /// Re-export the API for backward compatibility.
 pub use sc_rpc_api::author::*;

@@ -52,12 +53,12 @@ pub struct Author<P, Client> {
 	client: Arc<Client>,
 	/// Transactions pool
 	pool: Arc<P>,
-	/// Subscriptions manager
-	subscriptions: SubscriptionManager,
 	/// The key store.
 	keystore: SyncCryptoStorePtr,
 	/// Whether to deny unsafe calls
 	deny_unsafe: DenyUnsafe,
+	/// Executor to spawn subscriptions.
+	executor: SubscriptionTaskExecutor,
 }

 impl<P, Client> Author<P, Client> {
@@ -65,11 +66,11 @@ impl<P, Client> Author<P, Client> {
 	pub fn new(
 		client: Arc<Client>,
 		pool: Arc<P>,
-		subscriptions: SubscriptionManager,
 		keystore: SyncCryptoStorePtr,
 		deny_unsafe: DenyUnsafe,
+		executor: SubscriptionTaskExecutor,
 	) -> Self {
-		Author { client, pool, subscriptions, keystore, deny_unsafe }
+		Author { client, pool, keystore, deny_unsafe, executor }
 	}
 }

@@ -80,7 +81,8 @@ impl<P, Client> Author<P, Client> {
 /// some unique transactions via RPC and have them included in the pool.
 const TX_SOURCE: TransactionSource = TransactionSource::External;

-impl<P, Client> AuthorApi<TxHash<P>, BlockHash<P>> for Author<P, Client>
+#[async_trait]
+impl<P, Client> AuthorApiServer<TxHash<P>, BlockHash<P>> for Author<P, Client>
 where
 	P: TransactionPool + Sync + Send + 'static,
 	Client: HeaderBackend<P::Block> + ProvideRuntimeApi<P::Block> + Send + Sync + 'static,
@@ -88,9 +90,24 @@ where
 	P::Hash: Unpin,
 	<P::Block as BlockT>::Hash: Unpin,
 {
-	type Metadata = crate::Metadata;
+	async fn submit_extrinsic(&self, ext: Bytes) -> RpcResult<TxHash<P>> {
+		let xt = match Decode::decode(&mut &ext[..]) {
+			Ok(xt) => xt,
+			Err(err) => return Err(Error::Client(Box::new(err)).into()),
+		};
+		let best_block_hash = self.client.info().best_hash;
+		self.pool
+			.submit_one(&generic::BlockId::hash(best_block_hash), TX_SOURCE, xt)
+			.await
+			.map_err(|e| {
+				e.into_pool_error()
+					.map(|e| Error::Pool(e))
+					.unwrap_or_else(|e| Error::Verification(Box::new(e)))
+					.into()
+			})
+	}

-	fn insert_key(&self, key_type: String, suri: String, public: Bytes) -> Result<()> {
+	fn insert_key(&self, key_type: String, suri: String, public: Bytes) -> RpcResult<()> {
 		self.deny_unsafe.check_if_safe()?;

 		let key_type = key_type.as_str().try_into().map_err(|_| Error::BadKeyType)?;
@@ -99,7 +116,7 @@ where
 		Ok(())
 	}

-	fn rotate_keys(&self) -> Result<Bytes> {
+	fn rotate_keys(&self) -> RpcResult<Bytes> {
 		self.deny_unsafe.check_if_safe()?;

 		let best_block_hash = self.client.info().best_hash;
@@ -107,10 +124,10 @@ where
 			.runtime_api()
 			.generate_session_keys(&generic::BlockId::Hash(best_block_hash), None)
 			.map(Into::into)
-			.map_err(|e| Error::Client(Box::new(e)))
+			.map_err(|api_err| Error::Client(Box::new(api_err)).into())
 	}

-	fn has_session_keys(&self, session_keys: Bytes) -> Result<bool> {
+	fn has_session_keys(&self, session_keys: Bytes) -> RpcResult<bool> {
 		self.deny_unsafe.check_if_safe()?;

 		let best_block_hash = self.client.info().best_hash;
@@ -124,40 +141,22 @@ where
 		Ok(SyncCryptoStore::has_keys(&*self.keystore, &keys))
 	}

-	fn has_key(&self, public_key: Bytes, key_type: String) -> Result<bool> {
+	fn has_key(&self, public_key: Bytes, key_type: String) -> RpcResult<bool> {
 		self.deny_unsafe.check_if_safe()?;

 		let key_type = key_type.as_str().try_into().map_err(|_| Error::BadKeyType)?;
 		Ok(SyncCryptoStore::has_keys(&*self.keystore, &[(public_key.to_vec(), key_type)]))
 	}

-	fn submit_extrinsic(&self, ext: Bytes) -> FutureResult<TxHash<P>> {
-		let xt = match Decode::decode(&mut &ext[..]) {
-			Ok(xt) => xt,
-			Err(err) => return async move { Err(err.into()) }.boxed(),
-		};
-		let best_block_hash = self.client.info().best_hash;
-
-		self.pool
-			.submit_one(&generic::BlockId::hash(best_block_hash), TX_SOURCE, xt)
-			.map_err(|e| {
-				e.into_pool_error()
-					.map(Into::into)
-					.unwrap_or_else(|e| error::Error::Verification(Box::new(e)))
-			})
-			.boxed()
-	}
-
-	fn pending_extrinsics(&self) -> Result<Vec<Bytes>> {
+	fn pending_extrinsics(&self) -> RpcResult<Vec<Bytes>> {
 		Ok(self.pool.ready().map(|tx| tx.data().encode().into()).collect())
 	}

 	fn remove_extrinsic(
 		&self,
 		bytes_or_hash: Vec<hash::ExtrinsicOrHash<TxHash<P>>>,
-	) -> Result<Vec<TxHash<P>>> {
+	) -> RpcResult<Vec<TxHash<P>>> {
 		self.deny_unsafe.check_if_safe()?;
-
 		let hashes = bytes_or_hash
 			.into_iter()
 			.map(|x| match x {
@@ -177,20 +176,12 @@ where
 			.collect())
 	}

-	fn watch_extrinsic(
-		&self,
-		_metadata: Self::Metadata,
-		subscriber: Subscriber<TransactionStatus<TxHash<P>, BlockHash<P>>>,
-		xt: Bytes,
-	) {
+	fn watch_extrinsic(&self, pending: PendingSubscription, xt: Bytes) {
 		let best_block_hash = self.client.info().best_hash;
-		let dxt = match TransactionFor::<P>::decode(&mut &xt[..]).map_err(error::Error::from) {
-			Ok(tx) => tx,
-			Err(err) => {
-				log::debug!("Failed to submit extrinsic: {}", err);
-				// reject the subscriber (ignore errors - we don't care if subscriber is no longer
-				// there).
-				let _ = subscriber.reject(err.into());
+		let dxt = match TransactionFor::<P>::decode(&mut &xt[..]).map_err(|e| Error::from(e)) {
+			Ok(dxt) => dxt,
+			Err(e) => {
+				pending.reject(JsonRpseeError::from(e));
 				return
 			},
 		};
@@ -204,41 +195,25 @@ where
 					.unwrap_or_else(|e| error::Error::Verification(Box::new(e)))
 			});

-		let subscriptions = self.subscriptions.clone();
-
-		let future = async move {
-			let tx_stream = match submit.await {
-				Ok(s) => s,
+		let fut = async move {
+			let stream = match submit.await {
+				Ok(stream) => stream,
 				Err(err) => {
-					log::debug!("Failed to submit extrinsic: {}", err);
-					// reject the subscriber (ignore errors - we don't care if subscriber is no
-					// longer there).
-					let _ = subscriber.reject(err.into());
+					pending.reject(JsonRpseeError::from(err));
 					return
 				},
 			};

-			subscriptions.add(subscriber, move |sink| {
-				tx_stream
-					.map(|v| Ok(Ok(v)))
-					.forward(
-						sink.sink_map_err(|e| log::debug!("Error sending notifications: {:?}", e)),
-					)
-					.map(drop)
-			});
-		};
+			let mut sink = match pending.accept() {
+				Some(sink) => sink,
+				_ => return,
+			};

-		let res = self.subscriptions.executor().spawn_obj(future.boxed().into());
-		if res.is_err() {
-			log::warn!("Error spawning subscription RPC task.");
+			sink.pipe_from_stream(stream).await;
 		}
-	}
+		.boxed();

-	fn unwatch_extrinsic(
-		&self,
-		_metadata: Option<Self::Metadata>,
-		id: SubscriptionId,
-	) -> Result<bool> {
-		Ok(self.subscriptions.cancel(id))
+		self.executor
+			.spawn("substrate-rpc-subscription", Some("rpc"), fut.map(drop).boxed());
 	}
 }