Files
pezkuwi-subxt/substrate/client/service/src/config.rs
T
David 29c0c6a4a8 jsonrpsee integration (#8783)
* Add tokio

* No need to map CallError to CallError

* jsonrpsee proc macros (#9673)

* port error types to `JsonRpseeError`

* migrate chain module to proc macro api

* make it compile with proc macros

* update branch

* update branch

* update to jsonrpsee master

* port system rpc

* port state rpc

* port childstate & offchain

* frame system rpc

* frame transaction payment

* bring back CORS hack to work with polkadot UI

* port babe rpc

* port manual seal rpc

* port frame mmr rpc

* port frame contracts rpc

* port finality grandpa rpc

* port sync state rpc

* resolve a few TODO + no jsonrpc deps

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Update bin/node/rpc-client/src/main.rs

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* fix nit in frame system api

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* update jsonrpsee

* update jsonrpsee

* cleanup rpc-servers crate

* jsonrpsee: add host and origin filtering (#9787)

* add access control in the jsonrpsee servers

* use master

* fix nits

* rpc runtime_version safe

* fix nits

* fix grumbles

* remove unused files

* resolve some todos

* jsonrpsee more cleanup (#9803)

* more cleanup

* resolve TODOs

* fix some unwraps

* remove type hints

* update jsonrpsee

* downgrade zeroize

* pin jsonrpsee rev

* remove unwrap nit

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* RPC tests for state, author and system (#9859)

* Fix test runner

* Impl Default for SubscriptionTaskExecutor

* Keep the minimul amount of code needed to compile tests

* Re-instate `RpcSession` (for now)

* cleanup

* Port over RPC tests

* Add tokio

* No need to map CallError to CallError

* Port over system_ rpc tests

* Make it compile

* Use prost 0.8

* Use prost 0.8

* Make it compile

* Ignore more failing tests

* Comment out WIP tests

* Update lockfile

* No more juggling tokio versions

* No more wait_for_stop ?

* Remove browser-testing

* Arguments must be arrays

* Use same argument names

* Resolve todo: no wait_for_stop for WS server
Add todo: is parse_rpc_result used?
Cleanup imports

* fmt

* log

* One test passes

* Comment out more tests that aren't ported

* Comment out more tests

* Fix tests after merge

* Subscription test

* Invalid nonce test

* Pending exts

* WIP removeExtrinsic test

* Test remove_extrinsic

* Make state test: should_return_storage work

* Uncomment/fix the other non-subscription related state tests

* test: author_insertKey

* test: author_rotateKeys

* Get rest of state tests passing

* asyncify a little more

* Add todo to note #msg change

* Crashing test for has_session_keys

* Fix error conversion to avoid stack overflows
Port author_hasSessionKeys test
fmt

* test author_hasKey

* Add two missing tests
Add a check on the return type
Add todos for James's concerns

* offchain rpc tests

* Address todos

* fmt

Co-authored-by: James Wilson <james@jsdw.me>

* fix drop in state test

* update jsonrpsee

* fix ignored system test

* fix chain tests

* remove some boiler plate

* Port BEEFY RPC (#9883)

* Merge master

* Port beefy RPC (ty @niklas!)

* trivial changes left over from merge

* Remove unused code

* Update jsonrpsee

* fix build

* make tests compile again

* beefy update jsonrpsee

* fix: respect rpc methods policy

* update cargo.lock

* update jsonrpsee

* update jsonrpsee

* downgrade error logs

* update jsonrpsee

* Fix typo

* remove unused file

* Better name

* Port Babe RPC tests

* Put docs back

* Resolve todo

* Port tests for System RPCs

* Resolve todo

* fix build

* Updated jsonrpsee to current master

* fix: port finality grandpa rpc tests

* Move .into() outside of the match

* more review grumbles

* jsonrpsee: add `rpc handlers` back (#10245)

* add back RpcHandlers

* cargo fmt

* fix docs

* fix grumble: remove needless alloc

* resolve TODO

* fmt

* Fix typo

* grumble: Use constants based on BASE_ERROR

* grumble: DRY whitelisted listening addresses
grumble: s/JSONRPC/JSON-RPC/

* cleanup

* grumbles: Making readers aware of the possibility of gaps

* review grumbles

* grumbles

* remove notes from niklasad1

* Update `jsonrpsee`

* fix: jsonrpsee features

* jsonrpsee: fallback to random port in case the specified port failed (#10304)

* jsonrpsee: fallback to random port

* better comment

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update client/rpc-servers/src/lib.rs

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* address grumbles

* cargo fmt

* addrs already slice

Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>

* Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae

* lockfile

* update jsonrpsee

* fix warning

* Don't fetch jsonrpsee from crates

* make tests compile again

* fix rpc tests

* remove unused deps

* update tokio

* fix rpc tests again

* fix: test runner

`HttpServerBuilder::builder` fails unless it's called within tokio runtime

* cargo fmt

* grumbles: fix subscription aliases

* make clippy happy

* update remaining subscriptions alias

* cleanup

* cleanup

* fix chain subscription: less boiler plate (#10285)

* fix chain subscription: less boiler plate

* fix bad merge

* cargo fmt

* Switch to jsonrpsee 0.5

* fix build

* add missing features

* fix nit: remove needless Box::pin

* Integrate jsonrpsee metrics (#10395)

* draft metrics impl

* Use latest api

* Add missing file

* Http server metrics

* cleanup

* bump jsonrpsee

* Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.

* fix build

* remove needless Arc::clone

* Update to jsonrpsee 0.6

* lolz

* fix metrics

* Revert "lolz"

This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.

* fix: in-memory rpc support subscriptions

* commit Cargo.lock

* Update tests to 0.7

* fix TODOs

* ws server: generate subscriptionIDs as Strings

Some libraries seems to expect the subscription IDs to be Strings, let's not break
this in this PR.

* Increase timeout

* Port over tests

* cleanup

* Using error codes from the spec

* fix clippy

* cargo fmt

* update jsonrpsee

* fix nits

* fix: rpc_query

* enable custom subid gen through spawn_tasks

* remove unsed deps

* unify tokio deps

* Revert "enable custom subid gen through spawn_tasks"

This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.

* fix bad merge of `test-utils`

* fix more nits

* downgrade wasm-instrument to 0.1.0

* [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)

* enable custom subid gen through spawn_tasks

* fix nits

* Update client/service/src/builder.rs

Co-authored-by: David <dvdplm@gmail.com>

* add Poc; needs jsonrpsee pr

* update jsonrpsee

* add re-exports

* add docs

Co-authored-by: David <dvdplm@gmail.com>

* cargo fmt

* fmt

* port RPC-API dev

* Remove unused file

* fix nit: remove async trait

* fix doc links

* fix merge nit: remove jsonrpc deps

* kill namespace on rpc apis

* companion for jsonrpsee v0.10 (#11158)

* companion for jsonrpsee v0.10

* update versions v0.10.0

* add some fixes

* spelling

* fix spaces

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* send error before subs are closed

* fix unsubscribe method names: chain

* fix tests

* jsonrpc server: print binded local address

* grumbles: kill SubscriptionTaskExecutor

* Update client/sync-state-rpc/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/rpc/src/chain/chain_full.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* sync-state-rpc: kill anyhow

* no more anyhow

* remove todo

* jsonrpsee:  fix bad params in subscriptions. (#11251)

* update jsonrpsee

* fix error responses

* revert error codes

* dont do weird stuff in drop impl

* rpc servers: remove needless clone

* Remove silly constants

* chore: update jsonrpsee v0.12

* commit Cargo.lock

* deps: downgrade git2

* feat: CLI flag max subscriptions per connection

* metrics: use old logging format

* fix: read WS address from substrate output (#11379)

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Maciej Hirsz <hello@maciej.codes>
Co-authored-by: Maciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-05-10 08:52:19 +00:00

319 lines
10 KiB
Rust

// This file is part of Substrate.
// Copyright (C) 2017-2022 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//! Service configuration.
pub use sc_client_api::execution_extensions::{ExecutionStrategies, ExecutionStrategy};
pub use sc_client_db::{Database, DatabaseSource, KeepBlocks, PruningMode};
pub use sc_executor::WasmExecutionMethod;
pub use sc_network::{
config::{
MultiaddrWithPeerId, NetworkConfiguration, NodeKeyConfig, NonDefaultSetConfig, Role,
SetConfig, TransportConfig,
},
Multiaddr,
};
pub use sc_network_common::{
config::ProtocolId,
request_responses::{
IncomingRequest, OutgoingResponse, ProtocolConfig as RequestResponseConfig,
},
};
use prometheus_endpoint::Registry;
use sc_chain_spec::ChainSpec;
pub use sc_telemetry::TelemetryEndpoints;
pub use sc_transaction_pool::Options as TransactionPoolOptions;
use sp_core::crypto::SecretString;
use std::{
io, iter,
net::SocketAddr,
path::{Path, PathBuf},
};
use tempfile::TempDir;
/// Service configuration.
#[derive(Debug)]
pub struct Configuration {
/// Implementation name
pub impl_name: String,
/// Implementation version (see sc-cli to see an example of format)
pub impl_version: String,
/// Node role.
pub role: Role,
/// Handle to the tokio runtime. Will be used to spawn futures by the task manager.
pub tokio_handle: tokio::runtime::Handle,
/// Extrinsic pool configuration.
pub transaction_pool: TransactionPoolOptions,
/// Network configuration.
pub network: NetworkConfiguration,
/// Configuration for the keystore.
pub keystore: KeystoreConfig,
/// Remote URI to connect to for async keystore support
pub keystore_remote: Option<String>,
/// Configuration for the database.
pub database: DatabaseSource,
/// Size of internal state cache in Bytes
pub state_cache_size: usize,
/// Size in percent of cache size dedicated to child tries
pub state_cache_child_ratio: Option<usize>,
/// State pruning settings.
pub state_pruning: Option<PruningMode>,
/// Number of blocks to keep in the db.
pub keep_blocks: KeepBlocks,
/// Chain configuration.
pub chain_spec: Box<dyn ChainSpec>,
/// Wasm execution method.
pub wasm_method: WasmExecutionMethod,
/// Directory where local WASM runtimes live. These runtimes take precedence
/// over on-chain runtimes when the spec version matches. Set to `None` to
/// disable overrides (default).
pub wasm_runtime_overrides: Option<PathBuf>,
/// Execution strategies.
pub execution_strategies: ExecutionStrategies,
/// RPC over HTTP binding address. `None` if disabled.
pub rpc_http: Option<SocketAddr>,
/// RPC over Websockets binding address. `None` if disabled.
pub rpc_ws: Option<SocketAddr>,
/// RPC over IPC binding path. `None` if disabled.
pub rpc_ipc: Option<String>,
/// Maximum number of connections for WebSockets RPC server. `None` if default.
pub rpc_ws_max_connections: Option<usize>,
/// CORS settings for HTTP & WS servers. `None` if all origins are allowed.
pub rpc_cors: Option<Vec<String>>,
/// RPC methods to expose (by default only a safe subset or all of them).
pub rpc_methods: RpcMethods,
/// Maximum payload of rpc request/responses.
pub rpc_max_payload: Option<usize>,
/// Maximum payload of a rpc request
pub rpc_max_request_size: Option<usize>,
/// Maximum payload of a rpc request
pub rpc_max_response_size: Option<usize>,
/// Custom JSON-RPC subscription ID provider.
///
/// Default: [`crate::RandomStringSubscriptionId`].
pub rpc_id_provider: Option<Box<dyn crate::RpcSubscriptionIdProvider>>,
/// Maximum allowed subscriptions per rpc connection
///
/// Default: 1024.
pub rpc_max_subs_per_conn: Option<usize>,
/// Maximum size of the output buffer capacity for websocket connections.
pub ws_max_out_buffer_capacity: Option<usize>,
/// Prometheus endpoint configuration. `None` if disabled.
pub prometheus_config: Option<PrometheusConfig>,
/// Telemetry service URL. `None` if disabled.
pub telemetry_endpoints: Option<TelemetryEndpoints>,
/// The default number of 64KB pages to allocate for Wasm execution
pub default_heap_pages: Option<u64>,
/// Should offchain workers be executed.
pub offchain_worker: OffchainWorkerConfig,
/// Enable authoring even when offline.
pub force_authoring: bool,
/// Disable GRANDPA when running in validator mode
pub disable_grandpa: bool,
/// Development key seed.
///
/// When running in development mode, the seed will be used to generate authority keys by the
/// keystore.
///
/// Should only be set when `node` is running development mode.
pub dev_key_seed: Option<String>,
/// Tracing targets
pub tracing_targets: Option<String>,
/// Tracing receiver
pub tracing_receiver: sc_tracing::TracingReceiver,
/// The size of the instances cache.
///
/// The default value is 8.
pub max_runtime_instances: usize,
/// Announce block automatically after they have been imported
pub announce_block: bool,
/// Base path of the configuration
pub base_path: Option<BasePath>,
/// Configuration of the output format that the informant uses.
pub informant_output_format: sc_informant::OutputFormat,
/// Maximum number of different runtime versions that can be cached.
pub runtime_cache_size: u8,
}
/// Type for tasks spawned by the executor.
#[derive(PartialEq)]
pub enum TaskType {
/// Regular non-blocking futures. Polling the task is expected to be a lightweight operation.
Async,
/// The task might perform a lot of expensive CPU operations and/or call `thread::sleep`.
Blocking,
}
/// Configuration of the client keystore.
#[derive(Debug, Clone)]
pub enum KeystoreConfig {
/// Keystore at a path on-disk. Recommended for native nodes.
Path {
/// The path of the keystore.
path: PathBuf,
/// Node keystore's password.
password: Option<SecretString>,
},
/// In-memory keystore. Recommended for in-browser nodes.
InMemory,
}
impl KeystoreConfig {
/// Returns the path for the keystore.
pub fn path(&self) -> Option<&Path> {
match self {
Self::Path { path, .. } => Some(path),
Self::InMemory => None,
}
}
}
/// Configuration of the database of the client.
#[derive(Debug, Clone, Default)]
pub struct OffchainWorkerConfig {
/// If this is allowed.
pub enabled: bool,
/// allow writes from the runtime to the offchain worker database.
pub indexing_enabled: bool,
}
/// Configuration of the Prometheus endpoint.
#[derive(Debug, Clone)]
pub struct PrometheusConfig {
/// Port to use.
pub port: SocketAddr,
/// A metrics registry to use. Useful for setting the metric prefix.
pub registry: Registry,
}
impl PrometheusConfig {
/// Create a new config using the default registry.
pub fn new_with_default_registry(port: SocketAddr, chain_id: String) -> Self {
let param = iter::once((String::from("chain"), chain_id)).collect();
Self {
port,
registry: Registry::new_custom(None, Some(param))
.expect("this can only fail if the prefix is empty"),
}
}
}
impl Configuration {
/// Returns a string displaying the node role.
pub fn display_role(&self) -> String {
self.role.to_string()
}
/// Returns the prometheus metrics registry, if available.
pub fn prometheus_registry(&self) -> Option<&Registry> {
self.prometheus_config.as_ref().map(|config| &config.registry)
}
/// Returns the network protocol id from the chain spec, or the default.
pub fn protocol_id(&self) -> ProtocolId {
let protocol_id_full = match self.chain_spec.protocol_id() {
Some(pid) => pid,
None => {
log::warn!(
"Using default protocol ID {:?} because none is configured in the \
chain specs",
crate::DEFAULT_PROTOCOL_ID
);
crate::DEFAULT_PROTOCOL_ID
},
};
ProtocolId::from(protocol_id_full)
}
}
/// Available RPC methods.
#[derive(Debug, Copy, Clone)]
pub enum RpcMethods {
/// Expose every RPC method only when RPC is listening on `localhost`,
/// otherwise serve only safe RPC methods.
Auto,
/// Allow only a safe subset of RPC methods.
Safe,
/// Expose every RPC method (even potentially unsafe ones).
Unsafe,
}
impl Default for RpcMethods {
fn default() -> RpcMethods {
RpcMethods::Auto
}
}
/// The base path that is used for everything that needs to be write on disk to run a node.
#[derive(Debug)]
pub enum BasePath {
/// A temporary directory is used as base path and will be deleted when dropped.
Temporary(TempDir),
/// A path on the disk.
Permanenent(PathBuf),
}
impl BasePath {
/// Create a `BasePath` instance using a temporary directory prefixed with "substrate" and use
/// it as base path.
///
/// Note: the temporary directory will be created automatically and deleted when the `BasePath`
/// instance is dropped.
pub fn new_temp_dir() -> io::Result<BasePath> {
Ok(BasePath::Temporary(tempfile::Builder::new().prefix("substrate").tempdir()?))
}
/// Create a `BasePath` instance based on an existing path on disk.
///
/// Note: this function will not ensure that the directory exist nor create the directory. It
/// will also not delete the directory when the instance is dropped.
pub fn new<P: AsRef<Path>>(path: P) -> BasePath {
BasePath::Permanenent(path.as_ref().to_path_buf())
}
/// Create a base path from values describing the project.
pub fn from_project(qualifier: &str, organization: &str, application: &str) -> BasePath {
BasePath::new(
directories::ProjectDirs::from(qualifier, organization, application)
.expect("app directories exist on all supported platforms; qed")
.data_local_dir(),
)
}
/// Retrieve the base path.
pub fn path(&self) -> &Path {
match self {
BasePath::Temporary(temp_dir) => temp_dir.path(),
BasePath::Permanenent(path) => path.as_path(),
}
}
/// Returns the configuration directory inside this base path.
///
/// The path looks like `$base_path/chains/$chain_id`
pub fn config_dir(&self, chain_id: &str) -> PathBuf {
self.path().join("chains").join(chain_id)
}
}
impl From<PathBuf> for BasePath {
fn from(path: PathBuf) -> Self {
BasePath::new(path)
}
}