rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313)

This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
This commit is contained in:
Niklas Adolfsson
2024-01-23 09:55:13 +01:00
committed by GitHub
parent 76c37c930b
commit e16ef0861f
117 changed files with 1245 additions and 1090 deletions
+5 -5
View File
@@ -284,7 +284,7 @@ pub struct RpcTransactionOutput {
/// The output string of the transaction if any.
pub result: String,
/// An async receiver if data will be returned via a callback.
pub receiver: futures::channel::mpsc::UnboundedReceiver<String>,
pub receiver: tokio::sync::mpsc::Receiver<String>,
}
impl std::fmt::Debug for RpcTransactionOutput {
@@ -344,7 +344,7 @@ impl RpcHandlersExt for RpcHandlers {
pub(crate) fn parse_rpc_result(
result: String,
receiver: futures::channel::mpsc::UnboundedReceiver<String>,
receiver: tokio::sync::mpsc::Receiver<String>,
) -> Result<RpcTransactionOutput, RpcTransactionError> {
let json: serde_json::Value =
serde_json::from_str(&result).expect("the result can only be a JSONRPC string; qed");
@@ -398,7 +398,7 @@ where
mod tests {
#[test]
fn parses_error_properly() {
let (_, rx) = futures::channel::mpsc::unbounded();
let (_, rx) = tokio::sync::mpsc::channel(1);
assert!(super::parse_rpc_result(
r#"{
"jsonrpc": "2.0",
@@ -410,7 +410,7 @@ mod tests {
)
.is_ok());
let (_, rx) = futures::channel::mpsc::unbounded();
let (_, rx) = tokio::sync::mpsc::channel(1);
let error = super::parse_rpc_result(
r#"{
"jsonrpc": "2.0",
@@ -428,7 +428,7 @@ mod tests {
assert_eq!(error.message, "Method not found");
assert!(error.data.is_none());
let (_, rx) = futures::channel::mpsc::unbounded();
let (_, rx) = tokio::sync::mpsc::channel(1);
let error = super::parse_rpc_result(
r#"{
"jsonrpc": "2.0",