rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313)

This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
This commit is contained in:
Niklas Adolfsson
2024-01-23 09:55:13 +01:00
committed by GitHub
parent 76c37c930b
commit e16ef0861f
117 changed files with 1245 additions and 1090 deletions
@@ -20,10 +20,13 @@
//! API trait of the chain head.
use crate::{
chain_head::event::{FollowEvent, MethodResponse},
chain_head::{
error::Error,
event::{FollowEvent, MethodResponse},
},
common::events::StorageQuery,
};
use jsonrpsee::{core::RpcResult, proc_macros::rpc};
use jsonrpsee::proc_macros::rpc;
use sp_rpc::list::ListOrValue;
#[rpc(client, server)]
@@ -56,7 +59,7 @@ pub trait ChainHeadApi<Hash> {
&self,
follow_subscription: String,
hash: Hash,
) -> RpcResult<MethodResponse>;
) -> Result<MethodResponse, Error>;
/// Retrieves the header of a pinned block.
///
@@ -75,7 +78,7 @@ pub trait ChainHeadApi<Hash> {
&self,
follow_subscription: String,
hash: Hash,
) -> RpcResult<Option<String>>;
) -> Result<Option<String>, Error>;
/// Returns storage entries at a specific block's state.
///
@@ -89,7 +92,7 @@ pub trait ChainHeadApi<Hash> {
hash: Hash,
items: Vec<StorageQuery<String>>,
child_trie: Option<String>,
) -> RpcResult<MethodResponse>;
) -> Result<MethodResponse, Error>;
/// Call into the Runtime API at a specified block's state.
///
@@ -103,7 +106,7 @@ pub trait ChainHeadApi<Hash> {
hash: Hash,
function: String,
call_parameters: String,
) -> RpcResult<MethodResponse>;
) -> Result<MethodResponse, Error>;
/// Unpin a block or multiple blocks reported by the `follow` method.
///
@@ -120,7 +123,7 @@ pub trait ChainHeadApi<Hash> {
&self,
follow_subscription: String,
hash_or_hashes: ListOrValue<Hash>,
) -> RpcResult<()>;
) -> Result<(), Error>;
/// Resumes a storage fetch started with `chainHead_storage` after it has generated an
/// `operationWaitingForContinue` event.
@@ -133,7 +136,7 @@ pub trait ChainHeadApi<Hash> {
&self,
follow_subscription: String,
operation_id: String,
) -> RpcResult<()>;
) -> Result<(), Error>;
/// Stops an operation started with chainHead_unstable_body, chainHead_unstable_call, or
/// chainHead_unstable_storage. If the operation was still in progress, this interrupts it. If
@@ -147,5 +150,5 @@ pub trait ChainHeadApi<Hash> {
&self,
follow_subscription: String,
operation_id: String,
) -> RpcResult<()>;
) -> Result<(), Error>;
}