CLI: refactoring: remove Options from sc_service::Configuration's fields (#5271)

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Rename IntoConfiguration to CliConfiguration

* Renamed into_configuration to create_configuration

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Move keystore params to its own module

* Use in-memory keystore even for build-spec

* Enforce proper value for node name

* dev_key_seed

* Telemetry endpoints

* rustfmt

* Converted all RunCmd

* rustfmt

* Added export-blocks

* Missed something

* Removed config_path in NetworkConfiguration (not used)

* Fixed warnings

* public_addresses is used but never set, keeping it

* Merge Configuration.node and NetworkConfiguration.node_name

...because they are the same thing

* Added: import-blocks

* Adding a proc_macro to help impl SubstrateCli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Re-export spec_factory from sc_cli

* Re-added all the commands

* Refactored node_key_params

* Fixed previous refucktoring

* Clean-up and removed full_version()

* Renamed get_is_dev to not confuse with Configuration field

* Fixed sc-cli-derive example

* Fixing tests

* Fixing tests and removing some (will re-add later)

* Fixing more tests

* Removes the need of type parameter

* Converting bin/node and simplifying API

* Converting more

* Converting last command

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fixing tests and added default for WasmExecutionMethod

* Fixing stuff

* Fixed something I broke oops

* Update Cargo.lock

* Moving things around

* Convert everything to Result

* Added new macros to simplify the impl of CliConfiguration

* Added a macro to generate CliConfiguration automatically for subcommands

* Revert... too many macros (this one is not really useful)

This reverts commit 9c516dd38b40fbc420b02c1f8e61d5b2b1a4e434.

* Renamed is_dev to get_is_dev

Good enough for now

* Fixed name roles (this is plural, not singular)

* Clean-up

* Re-export NodeKeyConfig and TelemetryEndpoints from sc_service

* Improve styling/formatting

* Added copyrights

* Added doc and fixed warnings

* Added myself to code owners

* Yes it is needed according to the history

* Revert formatting

* Fixing conflict

* Updated build.rs

* Cargo.lock

* Clean-up

* Update client/cli-derive/Cargo.toml

Co-Authored-By: Seun Lanlege <seunlanlege@gmail.com>

* Fail if using proc_macro and build.rs is not set properly

* Dropped all get_ in front of methods

* Clean-up

* Fixing proc macro missing env var

* Get the configuration inside the Runtime (needed for polkadot)

* Clean-up

* Get is_dev from argument like the others

* Get chain ID instead of chain spec from shared params

* &self is passed to spec_factory/load_spec

* Wrong text

* Fix example

* Officialize macro and made a cool doc

* Renamed spec_factory to load_spec (substrate_cli_configuration)

* Removed not so useful ChainSpec

* Renamed SubstrateCLI to SubstrateCli

* Added changelog for impl_version being full now

* Renamed Runtime to Runner

* Update changelog to show example

* Removed option on database cache size

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fix on removal of option

* typo

* Clean-up imports

* Added info in Cargo.toml

* typo

* remarks

* Moved function for build.rs to substrate-build-script-utils

* Fixed example & test of cli-derive

* Moved function for build.rs to substrate-build-script-utils

* Renamed substrate_cli_configuration to substrate_cli oops

It implements SubstrateCli not CliConfiguration!

* Added documentation and wrapper macro

* Removed option on database cache size

* Removed option on database cache size

* Clean-up

* Reduce risk of errors due to typos

* Removed option on database cache size

* Added NOTE as suggested

* Added doc as suggested

* Fixed test

* typo

* renamed runtime to runner

* Fixed weird argument

* More commas

* Moved client/cli-derive to client/cli/derive

* Added 7 tests for the macros

* Improve error message

* Upgrade assert_cmd

* Fixing missing stuff

* Fixed unused import

* Improve SubstrateCli doc

* Applied suggestions

* Fix and clean-up imports

* Started replacing macros WIP

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Started removing substrate_cli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* fixed bug introduced while refactoring

* Renamed NetworkConfigurationParams to NetworkParams for consistency sake

* Fixed test

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/check_block_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 5906776953392c02beac6bc0bf50f8cbe1a12a01.

* Revert "Update client/cli/src/commands/check_block_cmd.rs"

This reverts commit f705f42b7f3d732be001141afee210fe46a1ef47.

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 8d57c0550164449e6eb2d3bacb04c750c714fcea.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 93e74cf5d2e1c0dc49cdff8608d59fc40fc59338.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 11d527ba345c0d79f0d3b5b071933d95474d0614.

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/import_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Changed ::sc_cli to $crate in the macro

* fixed tests

* fixed conflicts

* Fixing test

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove comment as suggested

* Apply suggestion

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/node_key_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/network_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added doc

* Fixed error introduced after applying suggestion

* Revert "Update client/cli/src/params/pruning_params.rs"

This reverts commit 0574d06a4f1efd86e94c1214420a12e7a4be0099.

* Print error

* Apply suggestions from code review

* Remove useless Results

* Fixed CI failing on polkadot approval

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This commit is contained in:
Cecile Tonglet
2020-04-07 11:38:07 +02:00
committed by GitHub
parent fb9bbf306d
commit 3da069e359
66 changed files with 2916 additions and 2156 deletions
@@ -14,16 +14,16 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use structopt::StructOpt;
use sc_service::{Configuration, config::DatabaseConfig};
use crate::error;
use crate::arg_enums::{
WasmExecutionMethod, TracingReceiver, ExecutionStrategy, DEFAULT_EXECUTION_BLOCK_CONSTRUCTION,
ExecutionStrategy, TracingReceiver, WasmExecutionMethod, DEFAULT_EXECUTION_BLOCK_CONSTRUCTION,
DEFAULT_EXECUTION_IMPORT_BLOCK, DEFAULT_EXECUTION_OFFCHAIN_WORKER, DEFAULT_EXECUTION_OTHER,
DEFAULT_EXECUTION_SYNCING
DEFAULT_EXECUTION_SYNCING,
};
use crate::params::PruningParams;
use crate::Result;
use sc_client_api::execution_extensions::ExecutionStrategies;
use sc_service::{PruningMode, Role};
use structopt::StructOpt;
/// Parameters for block import.
#[derive(Debug, StructOpt, Clone)]
@@ -52,11 +52,11 @@ pub struct ImportParams {
#[allow(missing_docs)]
#[structopt(flatten)]
pub execution_strategies: ExecutionStrategies,
pub execution_strategies: ExecutionStrategiesParams,
/// Limit the memory the database cache can use.
#[structopt(long = "db-cache", value_name = "MiB", default_value = "128")]
pub database_cache_size: u32,
#[structopt(long = "db-cache", value_name = "MiB")]
pub database_cache_size: Option<usize>,
/// Specify the state cache size.
#[structopt(long = "state-cache-size", value_name = "Bytes", default_value = "67108864")]
@@ -78,25 +78,31 @@ pub struct ImportParams {
}
impl ImportParams {
/// Put block import CLI params into `config` object.
pub fn update_config(
/// Receiver to process tracing messages.
pub fn tracing_receiver(&self) -> sc_service::TracingReceiver {
self.tracing_receiver.clone().into()
}
/// Comma separated list of targets for tracing.
pub fn tracing_targets(&self) -> Option<String> {
self.tracing_targets.clone()
}
/// Specify the state cache size.
pub fn state_cache_size(&self) -> usize {
self.state_cache_size
}
/// Get the WASM execution method from the parameters
pub fn wasm_method(&self) -> sc_service::config::WasmExecutionMethod {
self.wasm_method.into()
}
/// Get execution strategies for the parameters
pub fn execution_strategies(
&self,
mut config: &mut Configuration,
role: &sc_service::Role,
is_dev: bool,
) -> error::Result<()> {
use sc_client_api::execution_extensions::ExecutionStrategies;
if let Some(DatabaseConfig::Path { ref mut cache_size, .. }) = config.database {
*cache_size = Some(self.database_cache_size);
}
config.state_cache_size = self.state_cache_size;
self.pruning_params.update_config(&mut config, role, self.unsafe_pruning)?;
config.wasm_method = self.wasm_method.into();
) -> ExecutionStrategies {
let exec = &self.execution_strategies;
let exec_all_or = |strat: ExecutionStrategy, default: ExecutionStrategy| {
exec.execution.unwrap_or(if strat == default && is_dev {
@@ -106,7 +112,7 @@ impl ImportParams {
}).into()
};
config.execution_strategies = ExecutionStrategies {
ExecutionStrategies {
syncing: exec_all_or(exec.execution_syncing, DEFAULT_EXECUTION_SYNCING),
importing: exec_all_or(exec.execution_import_block, DEFAULT_EXECUTION_IMPORT_BLOCK),
block_construction:
@@ -114,15 +120,23 @@ impl ImportParams {
offchain_worker:
exec_all_or(exec.execution_offchain_worker, DEFAULT_EXECUTION_OFFCHAIN_WORKER),
other: exec_all_or(exec.execution_other, DEFAULT_EXECUTION_OTHER),
};
}
}
Ok(())
/// Get the pruning mode from the parameters
pub fn pruning(&self, unsafe_pruning: bool, role: &Role) -> Result<PruningMode> {
self.pruning_params.pruning(unsafe_pruning, role)
}
/// Limit the memory the database cache can use.
pub fn database_cache_size(&self) -> Option<usize> {
self.database_cache_size
}
}
/// Execution strategies parameters.
#[derive(Debug, StructOpt, Clone)]
pub struct ExecutionStrategies {
pub struct ExecutionStrategiesParams {
/// The means of execution used when calling into the runtime while syncing blocks.
#[structopt(
long = "execution-syncing",
@@ -0,0 +1,92 @@
// Copyright 2018-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::error::Result;
use sc_service::config::KeystoreConfig;
use std::fs;
use std::path::PathBuf;
use structopt::StructOpt;
/// default sub directory for the key store
const DEFAULT_KEYSTORE_CONFIG_PATH: &'static str = "keystore";
/// Parameters of the keystore
#[derive(Debug, StructOpt, Clone)]
pub struct KeystoreParams {
/// Specify custom keystore path.
#[structopt(long = "keystore-path", value_name = "PATH", parse(from_os_str))]
pub keystore_path: Option<PathBuf>,
/// Use interactive shell for entering the password used by the keystore.
#[structopt(
long = "password-interactive",
conflicts_with_all = &[ "password", "password-filename" ]
)]
pub password_interactive: bool,
/// Password used by the keystore.
#[structopt(
long = "password",
conflicts_with_all = &[ "password-interactive", "password-filename" ]
)]
pub password: Option<String>,
/// File that contains the password used by the keystore.
#[structopt(
long = "password-filename",
value_name = "PATH",
parse(from_os_str),
conflicts_with_all = &[ "password-interactive", "password" ]
)]
pub password_filename: Option<PathBuf>,
}
impl KeystoreParams {
/// Get the keystore configuration for the parameters
pub fn keystore_config(&self, base_path: &PathBuf) -> Result<KeystoreConfig> {
let password = if self.password_interactive {
#[cfg(not(target_os = "unknown"))]
{
Some(input_keystore_password()?.into())
}
#[cfg(target_os = "unknown")]
None
} else if let Some(ref file) = self.password_filename {
Some(
fs::read_to_string(file)
.map_err(|e| format!("{}", e))?
.into(),
)
} else if let Some(ref password) = self.password {
Some(password.clone().into())
} else {
None
};
let path = self
.keystore_path
.clone()
.unwrap_or(base_path.join(DEFAULT_KEYSTORE_CONFIG_PATH));
Ok(KeystoreConfig::Path { path, password })
}
}
#[cfg(not(target_os = "unknown"))]
fn input_keystore_password() -> Result<String> {
rpassword::read_password_from_tty(Some("Keystore password: "))
.map_err(|e| format!("{:?}", e).into())
}
+9 -7
View File
@@ -15,21 +15,23 @@
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
mod import_params;
mod transaction_pool_params;
mod shared_params;
mod keystore_params;
mod network_params;
mod node_key_params;
mod network_configuration_params;
mod pruning_params;
mod shared_params;
mod transaction_pool_params;
use std::str::FromStr;
use std::fmt::Debug;
use std::str::FromStr;
pub use crate::params::import_params::*;
pub use crate::params::transaction_pool_params::*;
pub use crate::params::shared_params::*;
pub use crate::params::keystore_params::*;
pub use crate::params::network_params::*;
pub use crate::params::node_key_params::*;
pub use crate::params::network_configuration_params::*;
pub use crate::params::pruning_params::*;
pub use crate::params::shared_params::*;
pub use crate::params::transaction_pool_params::*;
/// Wrapper type of `String` that holds an unsigned integer of arbitrary size, formatted as a decimal.
#[derive(Debug, Clone)]
@@ -14,21 +14,20 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::path::PathBuf;
use crate::params::node_key_params::NodeKeyParams;
use sc_network::{
config::{NetworkConfiguration, NodeKeyConfig, NonReservedPeerMode, TransportConfig},
multiaddr::Protocol,
};
use sc_service::{ChainSpec, config::{Multiaddr, MultiaddrWithPeerId}};
use std::iter;
use std::net::Ipv4Addr;
use std::path::PathBuf;
use structopt::StructOpt;
use sc_network::{
config::{MultiaddrWithPeerId, NonReservedPeerMode, TransportConfig}, multiaddr::Protocol, Multiaddr,
};
use sc_service::Configuration;
use crate::error;
use crate::params::node_key_params::NodeKeyParams;
/// Parameters used to create the network configuration.
#[derive(Debug, StructOpt, Clone)]
pub struct NetworkConfigurationParams {
pub struct NetworkParams {
/// Specify a list of bootnodes.
#[structopt(long = "bootnodes", value_name = "ADDR")]
pub bootnodes: Vec<MultiaddrWithPeerId>,
@@ -44,14 +43,6 @@ pub struct NetworkConfigurationParams {
#[structopt(long = "reserved-only")]
pub reserved_only: bool,
/// Specify a list of sentry node public addresses.
#[structopt(
long = "sentry-nodes",
value_name = "ADDR",
conflicts_with_all = &[ "sentry" ]
)]
pub sentry_nodes: Vec<MultiaddrWithPeerId>,
/// Listen on this multiaddress.
#[structopt(long = "listen-addr", value_name = "LISTEN_ADDR")]
pub listen_addr: Vec<Multiaddr>,
@@ -87,7 +78,11 @@ pub struct NetworkConfigurationParams {
///
/// This allows downloading announced blocks from multiple peers. Decrease to save
/// traffic and risk increased latency.
#[structopt(long = "max-parallel-downloads", value_name = "COUNT", default_value = "5")]
#[structopt(
long = "max-parallel-downloads",
value_name = "COUNT",
default_value = "5"
)]
pub max_parallel_downloads: u32,
#[allow(missing_docs)]
@@ -99,53 +94,50 @@ pub struct NetworkConfigurationParams {
pub use_yamux_flow_control: bool,
}
impl NetworkConfigurationParams {
impl NetworkParams {
/// Fill the given `NetworkConfiguration` by looking at the cli parameters.
pub fn update_config(
pub fn network_config(
&self,
mut config: &mut Configuration,
config_path: PathBuf,
client_id: String,
chain_spec: &Box<dyn ChainSpec>,
is_dev: bool,
) -> error::Result<()> {
config.network.boot_nodes.extend(self.bootnodes.clone());
config.network.config_path = Some(config_path.clone());
config.network.net_config_path = Some(config_path.clone());
net_config_path: &PathBuf,
client_id: &str,
node_name: &str,
node_key: NodeKeyConfig,
) -> NetworkConfiguration {
let port = self.port.unwrap_or(30333);
let mut listen_addresses = vec![iter::once(Protocol::Ip4(Ipv4Addr::new(0, 0, 0, 0)))
.chain(iter::once(Protocol::Tcp(port)))
.collect()];
config.network.reserved_nodes.extend(self.reserved_nodes.clone());
if self.reserved_only {
config.network.non_reserved_mode = NonReservedPeerMode::Deny;
listen_addresses.extend(self.listen_addr.iter().cloned());
let mut boot_nodes = chain_spec.boot_nodes().to_vec();
boot_nodes.extend(self.bootnodes.clone());
NetworkConfiguration {
boot_nodes,
net_config_path: net_config_path.clone(),
reserved_nodes: self.reserved_nodes.clone(),
non_reserved_mode: if self.reserved_only {
NonReservedPeerMode::Deny
} else {
NonReservedPeerMode::Accept
},
listen_addresses,
public_addresses: Vec::new(),
node_key,
node_name: node_name.to_string(),
client_version: client_id.to_string(),
in_peers: self.in_peers,
out_peers: self.out_peers,
transport: TransportConfig::Normal {
enable_mdns: !is_dev && !self.no_mdns,
allow_private_ipv4: !self.no_private_ipv4,
wasm_external_transport: None,
use_yamux_flow_control: self.use_yamux_flow_control,
},
max_parallel_downloads: self.max_parallel_downloads,
}
config.network.listen_addresses.extend(self.listen_addr.iter().cloned());
if config.network.listen_addresses.is_empty() {
let port = match self.port {
Some(port) => port,
None => 30333,
};
config.network.listen_addresses = vec![
iter::once(Protocol::Ip4(Ipv4Addr::new(0, 0, 0, 0)))
.chain(iter::once(Protocol::Tcp(port)))
.collect()
];
}
config.network.client_version = client_id;
self.node_key_params.update_config(&mut config, Some(&config_path))?;
config.network.in_peers = self.in_peers;
config.network.out_peers = self.out_peers;
config.network.transport = TransportConfig::Normal {
enable_mdns: !is_dev && !self.no_mdns,
allow_private_ipv4: !self.no_private_ipv4,
wasm_external_transport: None,
use_yamux_flow_control: self.use_yamux_flow_control,
};
config.network.max_parallel_downloads = self.max_parallel_downloads;
Ok(())
}
}
@@ -14,14 +14,13 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::{path::PathBuf, str::FromStr};
use structopt::StructOpt;
use sc_service::Configuration;
use sc_network::config::NodeKeyConfig;
use sp_core::H256;
use std::{path::PathBuf, str::FromStr};
use structopt::StructOpt;
use crate::error;
use crate::arg_enums::NodeKeyType;
use crate::error;
/// The file name of the node's Ed25519 secret key inside the chain-specific
/// network config directory, if neither `--node-key` nor `--node-key-file`
@@ -93,31 +92,23 @@ pub struct NodeKeyParams {
impl NodeKeyParams {
/// Create a `NodeKeyConfig` from the given `NodeKeyParams` in the context
/// of an optional network config storage directory.
pub fn update_config<'a>(
&self,
mut config: &'a mut Configuration,
net_config_path: Option<&PathBuf>,
) -> error::Result<&'a NodeKeyConfig> {
config.network.node_key = match self.node_key_type {
pub fn node_key(&self, net_config_dir: &PathBuf) -> error::Result<NodeKeyConfig> {
Ok(match self.node_key_type {
NodeKeyType::Ed25519 => {
let secret = if let Some(node_key) = self.node_key.as_ref() {
parse_ed25519_secret(node_key)?
} else {
let path = self.node_key_file.clone()
.or_else(|| net_config_path.map(|d| d.join(NODE_KEY_ED25519_FILE)));
let path = self
.node_key_file
.clone()
.unwrap_or_else(|| net_config_dir.join(NODE_KEY_ED25519_FILE));
if let Some(path) = path {
sc_network::config::Secret::File(path)
} else {
sc_network::config::Secret::New
}
sc_network::config::Secret::File(path)
};
NodeKeyConfig::Ed25519(secret)
}
};
Ok(&config.network.node_key)
})
}
}
@@ -128,114 +119,107 @@ fn invalid_node_key(e: impl std::fmt::Display) -> error::Error {
/// Parse a Ed25519 secret key from a hex string into a `sc_network::Secret`.
fn parse_ed25519_secret(hex: &str) -> error::Result<sc_network::config::Ed25519Secret> {
H256::from_str(&hex).map_err(invalid_node_key).and_then(|bytes|
sc_network::config::identity::ed25519::SecretKey::from_bytes(bytes)
.map(sc_network::config::Secret::Input)
.map_err(invalid_node_key))
H256::from_str(&hex)
.map_err(invalid_node_key)
.and_then(|bytes| {
sc_network::config::identity::ed25519::SecretKey::from_bytes(bytes)
.map(sc_network::config::Secret::Input)
.map_err(invalid_node_key)
})
}
#[cfg(test)]
mod tests {
use sc_network::config::identity::ed25519;
use super::*;
use sc_network::config::identity::ed25519;
#[test]
fn test_node_key_config_input() {
fn secret_input(net_config_dir: Option<&PathBuf>) -> error::Result<()> {
fn secret_input(net_config_dir: &PathBuf) -> error::Result<()> {
NodeKeyType::variants().iter().try_for_each(|t| {
let mut config = Configuration::default();
let node_key_type = NodeKeyType::from_str(t).unwrap();
let sk = match node_key_type {
NodeKeyType::Ed25519 => ed25519::SecretKey::generate().as_ref().to_vec()
NodeKeyType::Ed25519 => ed25519::SecretKey::generate().as_ref().to_vec(),
};
let params = NodeKeyParams {
node_key_type,
node_key: Some(format!("{:x}", H256::from_slice(sk.as_ref()))),
node_key_file: None
node_key_file: None,
};
params.update_config(&mut config, net_config_dir).and_then(|c| match c {
params.node_key(net_config_dir).and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::Input(ref ski))
if node_key_type == NodeKeyType::Ed25519 &&
&sk[..] == ski.as_ref() => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
if node_key_type == NodeKeyType::Ed25519 && &sk[..] == ski.as_ref() =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(secret_input(None).is_ok());
assert!(secret_input(Some(&PathBuf::from_str("x").unwrap())).is_ok());
assert!(secret_input(&PathBuf::from_str("x").unwrap()).is_ok());
}
#[test]
fn test_node_key_config_file() {
fn secret_file(net_config_dir: Option<&PathBuf>) -> error::Result<()> {
fn secret_file(net_config_dir: &PathBuf) -> error::Result<()> {
NodeKeyType::variants().iter().try_for_each(|t| {
let mut config = Configuration::default();
let node_key_type = NodeKeyType::from_str(t).unwrap();
let tmp = tempfile::Builder::new().prefix("alice").tempdir()?;
let file = tmp.path().join(format!("{}_mysecret", t)).to_path_buf();
let params = NodeKeyParams {
node_key_type,
node_key: None,
node_key_file: Some(file.clone())
node_key_file: Some(file.clone()),
};
params.update_config(&mut config, net_config_dir).and_then(|c| match c {
params.node_key(net_config_dir).and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::File(ref f))
if node_key_type == NodeKeyType::Ed25519 && f == &file => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
if node_key_type == NodeKeyType::Ed25519 && f == &file =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(secret_file(None).is_ok());
assert!(secret_file(Some(&PathBuf::from_str("x").unwrap())).is_ok());
assert!(secret_file(&PathBuf::from_str("x").unwrap()).is_ok());
}
#[test]
fn test_node_key_config_default() {
fn with_def_params<F>(f: F) -> error::Result<()>
where
F: Fn(NodeKeyParams) -> error::Result<()>
F: Fn(NodeKeyParams) -> error::Result<()>,
{
NodeKeyType::variants().iter().try_for_each(|t| {
let node_key_type = NodeKeyType::from_str(t).unwrap();
f(NodeKeyParams {
node_key_type,
node_key: None,
node_key_file: None
node_key_file: None,
})
})
}
fn no_config_dir() -> error::Result<()> {
with_def_params(|params| {
let mut config = Configuration::default();
let typ = params.node_key_type;
params.update_config(&mut config, None)
.and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::New)
if typ == NodeKeyType::Ed25519 => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
})
})
}
fn some_config_dir(net_config_dir: &PathBuf) -> error::Result<()> {
with_def_params(|params| {
let mut config = Configuration::default();
let dir = PathBuf::from(net_config_dir.clone());
let typ = params.node_key_type;
params.update_config(&mut config, Some(net_config_dir))
params
.node_key(net_config_dir)
.and_then(move |c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::File(ref f))
if typ == NodeKeyType::Ed25519 &&
f == &dir.join(NODE_KEY_ED25519_FILE) => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
})
if typ == NodeKeyType::Ed25519
&& f == &dir.join(NODE_KEY_ED25519_FILE) =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(no_config_dir().is_ok());
assert!(some_config_dir(&PathBuf::from_str("x").unwrap()).is_ok());
}
}
@@ -14,10 +14,9 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use structopt::StructOpt;
use sc_service::{Configuration, PruningMode};
use crate::error;
use sc_service::{PruningMode, Role};
use structopt::StructOpt;
/// Parameters to define the pruning mode
#[derive(Debug, StructOpt, Clone)]
@@ -32,18 +31,13 @@ pub struct PruningParams {
}
impl PruningParams {
/// Put block pruning CLI params into `config` object.
pub fn update_config(
&self,
mut config: &mut Configuration,
role: &sc_service::Role,
unsafe_pruning: bool,
) -> error::Result<()> {
/// Get the pruning value from the parameters
pub fn pruning(&self, unsafe_pruning: bool, role: &Role) -> error::Result<PruningMode> {
// by default we disable pruning if the node is an authority (i.e.
// `ArchiveAll`), otherwise we keep state for the last 256 blocks. if the
// node is an authority and pruning is enabled explicitly, then we error
// unless `unsafe_pruning` is set.
config.pruning = match &self.pruning {
Ok(match &self.pruning {
Some(ref s) if s == "archive" => PruningMode::ArchiveAll,
None if role.is_network_authority() => PruningMode::ArchiveAll,
None => PruningMode::default(),
@@ -51,16 +45,15 @@ impl PruningParams {
if role.is_network_authority() && !unsafe_pruning {
return Err(error::Error::Input(
"Validators should run with state pruning disabled (i.e. archive). \
You can ignore this check with `--unsafe-pruning`.".to_string()
You can ignore this check with `--unsafe-pruning`."
.to_string(),
));
}
PruningMode::keep_blocks(s.parse()
.map_err(|_| error::Error::Input("Invalid pruning mode specified".to_string()))?
)
},
};
Ok(())
PruningMode::keep_blocks(s.parse().map_err(|_| {
error::Error::Input("Invalid pruning mode specified".to_string())
})?)
}
})
}
}
@@ -14,18 +14,12 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sc_service::config::DatabaseConfig;
use std::path::PathBuf;
use structopt::StructOpt;
use app_dirs::{AppInfo, AppDataType};
use sc_service::{
Configuration, config::DatabaseConfig, ChainSpec,
};
use crate::VersionInfo;
use crate::error;
/// default sub directory to store database
const DEFAULT_DB_CONFIG_PATH : &'static str = "db";
const DEFAULT_DB_CONFIG_PATH: &'static str = "db";
/// Shared parameters used by all `CoreParams`.
#[derive(Debug, StructOpt, Clone)]
@@ -39,7 +33,12 @@ pub struct SharedParams {
pub dev: bool,
/// Specify custom base path.
#[structopt(long = "base-path", short = "d", value_name = "PATH", parse(from_os_str))]
#[structopt(
long = "base-path",
short = "d",
value_name = "PATH",
parse(from_os_str)
)]
pub base_path: Option<PathBuf>,
/// Sets a custom logging filter. Syntax is <target>=<level>, e.g. -lsync=debug.
@@ -51,62 +50,44 @@ pub struct SharedParams {
}
impl SharedParams {
/// Load spec to `Configuration` from `SharedParams` and spec factory.
pub fn update_config<'a, F>(
&self,
mut config: &'a mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<&'a dyn ChainSpec> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
let chain_key = match self.chain {
/// Specify custom base path.
pub fn base_path(&self) -> Option<PathBuf> {
self.base_path.clone()
}
/// Specify the development chain.
pub fn is_dev(&self) -> bool {
self.dev
}
/// Get the chain spec for the parameters provided
pub fn chain_id(&self, is_dev: bool) -> String {
match self.chain {
Some(ref chain) => chain.clone(),
None => if self.dev { "dev".into() } else { "".into() }
};
let spec = spec_factory(&chain_key)?;
config.network.boot_nodes = spec.boot_nodes().to_vec();
config.telemetry_endpoints = spec.telemetry_endpoints().clone();
config.chain_spec = Some(spec);
if config.config_dir.is_none() {
config.config_dir = Some(base_path(self, version));
}
if config.database.is_none() {
config.database = Some(DatabaseConfig::Path {
path: config
.in_chain_config_dir(DEFAULT_DB_CONFIG_PATH)
.expect("We provided a base_path/config_dir."),
cache_size: None,
});
}
Ok(config.expect_chain_spec())
}
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
pub fn init(&self, version: &VersionInfo) -> error::Result<()> {
crate::init(self.log.as_ref().map(|v| v.as_ref()).unwrap_or(""), version)
}
}
fn base_path(cli: &SharedParams, version: &VersionInfo) -> PathBuf {
cli.base_path.clone()
.unwrap_or_else(||
app_dirs::get_app_root(
AppDataType::UserData,
&AppInfo {
name: version.executable_name,
author: version.author
None => {
if is_dev {
"dev".into()
} else {
"".into()
}
).expect("app directories exist on all supported platforms; qed")
)
}
}
}
/// Get the database configuration object for the parameters provided
pub fn database_config(
&self,
base_path: &PathBuf,
cache_size: usize,
) -> DatabaseConfig {
DatabaseConfig::Path {
path: base_path.join(DEFAULT_DB_CONFIG_PATH),
cache_size,
}
}
/// Get the filters for the logging
pub fn log_filters(&self) -> Option<String> {
self.log.clone()
}
}
@@ -14,9 +14,8 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sc_service::config::TransactionPoolOptions;
use structopt::StructOpt;
use sc_service::Configuration;
use crate::error;
/// Parameters used to create the pool configuration.
#[derive(Debug, StructOpt, Clone)]
@@ -24,6 +23,7 @@ pub struct TransactionPoolParams {
/// Maximum number of transactions in the transaction pool.
#[structopt(long = "pool-limit", value_name = "COUNT", default_value = "8192")]
pub pool_limit: usize,
/// Maximum number of kilobytes of all transactions stored in the pool.
#[structopt(long = "pool-kbytes", value_name = "COUNT", default_value = "20480")]
pub pool_kbytes: usize,
@@ -31,19 +31,18 @@ pub struct TransactionPoolParams {
impl TransactionPoolParams {
/// Fill the given `PoolConfiguration` by looking at the cli parameters.
pub fn update_config(
&self,
config: &mut Configuration,
) -> error::Result<()> {
pub fn transaction_pool(&self) -> TransactionPoolOptions {
let mut opts = TransactionPoolOptions::default();
// ready queue
config.transaction_pool.ready.count = self.pool_limit;
config.transaction_pool.ready.total_bytes = self.pool_kbytes * 1024;
opts.ready.count = self.pool_limit;
opts.ready.total_bytes = self.pool_kbytes * 1024;
// future queue
let factor = 10;
config.transaction_pool.future.count = self.pool_limit / factor;
config.transaction_pool.future.total_bytes = self.pool_kbytes * 1024 / factor;
opts.future.count = self.pool_limit / factor;
opts.future.total_bytes = self.pool_kbytes * 1024 / factor;
Ok(())
opts
}
}