rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313)

This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
This commit is contained in:
Niklas Adolfsson
2024-01-23 09:55:13 +01:00
committed by GitHub
parent 76c37c930b
commit e16ef0861f
117 changed files with 1245 additions and 1090 deletions
@@ -17,8 +17,8 @@
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! Substrate child state API
use crate::state::ReadProof;
use jsonrpsee::{core::RpcResult, proc_macros::rpc};
use crate::state::{Error, ReadProof};
use jsonrpsee::proc_macros::rpc;
use sp_core::storage::{PrefixedStorageKey, StorageData, StorageKey};
/// Substrate child state API
@@ -35,7 +35,7 @@ pub trait ChildStateApi<Hash> {
child_storage_key: PrefixedStorageKey,
prefix: StorageKey,
hash: Option<Hash>,
) -> RpcResult<Vec<StorageKey>>;
) -> Result<Vec<StorageKey>, Error>;
/// Returns the keys with prefix from a child storage with pagination support.
/// Up to `count` keys will be returned.
@@ -48,7 +48,7 @@ pub trait ChildStateApi<Hash> {
count: u32,
start_key: Option<StorageKey>,
hash: Option<Hash>,
) -> RpcResult<Vec<StorageKey>>;
) -> Result<Vec<StorageKey>, Error>;
/// Returns a child storage entry at a specific block's state.
#[method(name = "childstate_getStorage", blocking)]
@@ -57,7 +57,7 @@ pub trait ChildStateApi<Hash> {
child_storage_key: PrefixedStorageKey,
key: StorageKey,
hash: Option<Hash>,
) -> RpcResult<Option<StorageData>>;
) -> Result<Option<StorageData>, Error>;
/// Returns child storage entries for multiple keys at a specific block's state.
#[method(name = "childstate_getStorageEntries", blocking)]
@@ -66,7 +66,7 @@ pub trait ChildStateApi<Hash> {
child_storage_key: PrefixedStorageKey,
keys: Vec<StorageKey>,
hash: Option<Hash>,
) -> RpcResult<Vec<Option<StorageData>>>;
) -> Result<Vec<Option<StorageData>>, Error>;
/// Returns the hash of a child storage entry at a block's state.
#[method(name = "childstate_getStorageHash", blocking)]
@@ -75,7 +75,7 @@ pub trait ChildStateApi<Hash> {
child_storage_key: PrefixedStorageKey,
key: StorageKey,
hash: Option<Hash>,
) -> RpcResult<Option<Hash>>;
) -> Result<Option<Hash>, Error>;
/// Returns the size of a child storage entry at a block's state.
#[method(name = "childstate_getStorageSize", blocking)]
@@ -84,7 +84,7 @@ pub trait ChildStateApi<Hash> {
child_storage_key: PrefixedStorageKey,
key: StorageKey,
hash: Option<Hash>,
) -> RpcResult<Option<u64>>;
) -> Result<Option<u64>, Error>;
/// Returns proof of storage for child key entries at a specific block's state.
#[method(name = "state_getChildReadProof", blocking)]
@@ -93,5 +93,5 @@ pub trait ChildStateApi<Hash> {
child_storage_key: PrefixedStorageKey,
keys: Vec<StorageKey>,
hash: Option<Hash>,
) -> RpcResult<ReadProof<Hash>>;
) -> Result<ReadProof<Hash>, Error>;
}