rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313)

This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
This commit is contained in:
Niklas Adolfsson
2024-01-23 09:55:13 +01:00
committed by GitHub
parent 76c37c930b
commit e16ef0861f
117 changed files with 1245 additions and 1090 deletions
@@ -21,9 +21,12 @@ use std::{convert::TryInto, sync::Arc};
use codec::{Codec, Decode};
use jsonrpsee::{
core::{Error as JsonRpseeError, RpcResult},
core::RpcResult,
proc_macros::rpc,
types::error::{CallError, ErrorCode, ErrorObject},
types::{
error::{ErrorCode, ErrorObject},
ErrorObjectOwned,
},
};
use pallet_transaction_payment_rpc_runtime_api::{FeeDetails, InclusionFee, RuntimeDispatchInfo};
use sp_api::ProvideRuntimeApi;
@@ -100,19 +103,15 @@ where
let encoded_len = encoded_xt.len() as u32;
let uxt: Block::Extrinsic = Decode::decode(&mut &*encoded_xt).map_err(|e| {
CallError::Custom(ErrorObject::owned(
ErrorObject::owned(
Error::DecodeError.into(),
"Unable to query dispatch info.",
Some(format!("{:?}", e)),
))
)
})?;
fn map_err(error: impl ToString, desc: &'static str) -> CallError {
CallError::Custom(ErrorObject::owned(
Error::RuntimeError.into(),
desc,
Some(error.to_string()),
))
fn map_err(error: impl ToString, desc: &'static str) -> ErrorObjectOwned {
ErrorObject::owned(Error::RuntimeError.into(), desc, Some(error.to_string()))
}
let res = api
@@ -137,27 +136,27 @@ where
let encoded_len = encoded_xt.len() as u32;
let uxt: Block::Extrinsic = Decode::decode(&mut &*encoded_xt).map_err(|e| {
CallError::Custom(ErrorObject::owned(
ErrorObject::owned(
Error::DecodeError.into(),
"Unable to query fee details.",
Some(format!("{:?}", e)),
))
)
})?;
let fee_details = api.query_fee_details(at_hash, uxt, encoded_len).map_err(|e| {
CallError::Custom(ErrorObject::owned(
ErrorObject::owned(
Error::RuntimeError.into(),
"Unable to query fee details.",
Some(e.to_string()),
))
)
})?;
let try_into_rpc_balance = |value: Balance| {
value.try_into().map_err(|_| {
JsonRpseeError::Call(CallError::Custom(ErrorObject::owned(
ErrorObject::owned(
ErrorCode::InvalidParams.code(),
format!("{} doesn't fit in NumberOrHex representation", value),
None::<()>,
)))
)
})
};