CLI: refactoring: remove Options from sc_service::Configuration's fields (#5271)

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Rename IntoConfiguration to CliConfiguration

* Renamed into_configuration to create_configuration

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Move keystore params to its own module

* Use in-memory keystore even for build-spec

* Enforce proper value for node name

* dev_key_seed

* Telemetry endpoints

* rustfmt

* Converted all RunCmd

* rustfmt

* Added export-blocks

* Missed something

* Removed config_path in NetworkConfiguration (not used)

* Fixed warnings

* public_addresses is used but never set, keeping it

* Merge Configuration.node and NetworkConfiguration.node_name

...because they are the same thing

* Added: import-blocks

* Adding a proc_macro to help impl SubstrateCli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Re-export spec_factory from sc_cli

* Re-added all the commands

* Refactored node_key_params

* Fixed previous refucktoring

* Clean-up and removed full_version()

* Renamed get_is_dev to not confuse with Configuration field

* Fixed sc-cli-derive example

* Fixing tests

* Fixing tests and removing some (will re-add later)

* Fixing more tests

* Removes the need of type parameter

* Converting bin/node and simplifying API

* Converting more

* Converting last command

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fixing tests and added default for WasmExecutionMethod

* Fixing stuff

* Fixed something I broke oops

* Update Cargo.lock

* Moving things around

* Convert everything to Result

* Added new macros to simplify the impl of CliConfiguration

* Added a macro to generate CliConfiguration automatically for subcommands

* Revert... too many macros (this one is not really useful)

This reverts commit 9c516dd38b40fbc420b02c1f8e61d5b2b1a4e434.

* Renamed is_dev to get_is_dev

Good enough for now

* Fixed name roles (this is plural, not singular)

* Clean-up

* Re-export NodeKeyConfig and TelemetryEndpoints from sc_service

* Improve styling/formatting

* Added copyrights

* Added doc and fixed warnings

* Added myself to code owners

* Yes it is needed according to the history

* Revert formatting

* Fixing conflict

* Updated build.rs

* Cargo.lock

* Clean-up

* Update client/cli-derive/Cargo.toml

Co-Authored-By: Seun Lanlege <seunlanlege@gmail.com>

* Fail if using proc_macro and build.rs is not set properly

* Dropped all get_ in front of methods

* Clean-up

* Fixing proc macro missing env var

* Get the configuration inside the Runtime (needed for polkadot)

* Clean-up

* Get is_dev from argument like the others

* Get chain ID instead of chain spec from shared params

* &self is passed to spec_factory/load_spec

* Wrong text

* Fix example

* Officialize macro and made a cool doc

* Renamed spec_factory to load_spec (substrate_cli_configuration)

* Removed not so useful ChainSpec

* Renamed SubstrateCLI to SubstrateCli

* Added changelog for impl_version being full now

* Renamed Runtime to Runner

* Update changelog to show example

* Removed option on database cache size

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fix on removal of option

* typo

* Clean-up imports

* Added info in Cargo.toml

* typo

* remarks

* Moved function for build.rs to substrate-build-script-utils

* Fixed example & test of cli-derive

* Moved function for build.rs to substrate-build-script-utils

* Renamed substrate_cli_configuration to substrate_cli oops

It implements SubstrateCli not CliConfiguration!

* Added documentation and wrapper macro

* Removed option on database cache size

* Removed option on database cache size

* Clean-up

* Reduce risk of errors due to typos

* Removed option on database cache size

* Added NOTE as suggested

* Added doc as suggested

* Fixed test

* typo

* renamed runtime to runner

* Fixed weird argument

* More commas

* Moved client/cli-derive to client/cli/derive

* Added 7 tests for the macros

* Improve error message

* Upgrade assert_cmd

* Fixing missing stuff

* Fixed unused import

* Improve SubstrateCli doc

* Applied suggestions

* Fix and clean-up imports

* Started replacing macros WIP

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Started removing substrate_cli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* fixed bug introduced while refactoring

* Renamed NetworkConfigurationParams to NetworkParams for consistency sake

* Fixed test

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/check_block_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 5906776953392c02beac6bc0bf50f8cbe1a12a01.

* Revert "Update client/cli/src/commands/check_block_cmd.rs"

This reverts commit f705f42b7f3d732be001141afee210fe46a1ef47.

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 8d57c0550164449e6eb2d3bacb04c750c714fcea.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 93e74cf5d2e1c0dc49cdff8608d59fc40fc59338.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 11d527ba345c0d79f0d3b5b071933d95474d0614.

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/import_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Changed ::sc_cli to $crate in the macro

* fixed tests

* fixed conflicts

* Fixing test

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove comment as suggested

* Apply suggestion

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/node_key_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/network_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added doc

* Fixed error introduced after applying suggestion

* Revert "Update client/cli/src/params/pruning_params.rs"

This reverts commit 0574d06a4f1efd86e94c1214420a12e7a4be0099.

* Print error

* Apply suggestions from code review

* Remove useless Results

* Fixed CI failing on polkadot approval

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This commit is contained in:
Cecile Tonglet
2020-04-07 11:38:07 +02:00
committed by GitHub
parent fb9bbf306d
commit 3da069e359
66 changed files with 2916 additions and 2156 deletions
@@ -14,14 +14,13 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::{path::PathBuf, str::FromStr};
use structopt::StructOpt;
use sc_service::Configuration;
use sc_network::config::NodeKeyConfig;
use sp_core::H256;
use std::{path::PathBuf, str::FromStr};
use structopt::StructOpt;
use crate::error;
use crate::arg_enums::NodeKeyType;
use crate::error;
/// The file name of the node's Ed25519 secret key inside the chain-specific
/// network config directory, if neither `--node-key` nor `--node-key-file`
@@ -93,31 +92,23 @@ pub struct NodeKeyParams {
impl NodeKeyParams {
/// Create a `NodeKeyConfig` from the given `NodeKeyParams` in the context
/// of an optional network config storage directory.
pub fn update_config<'a>(
&self,
mut config: &'a mut Configuration,
net_config_path: Option<&PathBuf>,
) -> error::Result<&'a NodeKeyConfig> {
config.network.node_key = match self.node_key_type {
pub fn node_key(&self, net_config_dir: &PathBuf) -> error::Result<NodeKeyConfig> {
Ok(match self.node_key_type {
NodeKeyType::Ed25519 => {
let secret = if let Some(node_key) = self.node_key.as_ref() {
parse_ed25519_secret(node_key)?
} else {
let path = self.node_key_file.clone()
.or_else(|| net_config_path.map(|d| d.join(NODE_KEY_ED25519_FILE)));
let path = self
.node_key_file
.clone()
.unwrap_or_else(|| net_config_dir.join(NODE_KEY_ED25519_FILE));
if let Some(path) = path {
sc_network::config::Secret::File(path)
} else {
sc_network::config::Secret::New
}
sc_network::config::Secret::File(path)
};
NodeKeyConfig::Ed25519(secret)
}
};
Ok(&config.network.node_key)
})
}
}
@@ -128,114 +119,107 @@ fn invalid_node_key(e: impl std::fmt::Display) -> error::Error {
/// Parse a Ed25519 secret key from a hex string into a `sc_network::Secret`.
fn parse_ed25519_secret(hex: &str) -> error::Result<sc_network::config::Ed25519Secret> {
H256::from_str(&hex).map_err(invalid_node_key).and_then(|bytes|
sc_network::config::identity::ed25519::SecretKey::from_bytes(bytes)
.map(sc_network::config::Secret::Input)
.map_err(invalid_node_key))
H256::from_str(&hex)
.map_err(invalid_node_key)
.and_then(|bytes| {
sc_network::config::identity::ed25519::SecretKey::from_bytes(bytes)
.map(sc_network::config::Secret::Input)
.map_err(invalid_node_key)
})
}
#[cfg(test)]
mod tests {
use sc_network::config::identity::ed25519;
use super::*;
use sc_network::config::identity::ed25519;
#[test]
fn test_node_key_config_input() {
fn secret_input(net_config_dir: Option<&PathBuf>) -> error::Result<()> {
fn secret_input(net_config_dir: &PathBuf) -> error::Result<()> {
NodeKeyType::variants().iter().try_for_each(|t| {
let mut config = Configuration::default();
let node_key_type = NodeKeyType::from_str(t).unwrap();
let sk = match node_key_type {
NodeKeyType::Ed25519 => ed25519::SecretKey::generate().as_ref().to_vec()
NodeKeyType::Ed25519 => ed25519::SecretKey::generate().as_ref().to_vec(),
};
let params = NodeKeyParams {
node_key_type,
node_key: Some(format!("{:x}", H256::from_slice(sk.as_ref()))),
node_key_file: None
node_key_file: None,
};
params.update_config(&mut config, net_config_dir).and_then(|c| match c {
params.node_key(net_config_dir).and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::Input(ref ski))
if node_key_type == NodeKeyType::Ed25519 &&
&sk[..] == ski.as_ref() => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
if node_key_type == NodeKeyType::Ed25519 && &sk[..] == ski.as_ref() =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(secret_input(None).is_ok());
assert!(secret_input(Some(&PathBuf::from_str("x").unwrap())).is_ok());
assert!(secret_input(&PathBuf::from_str("x").unwrap()).is_ok());
}
#[test]
fn test_node_key_config_file() {
fn secret_file(net_config_dir: Option<&PathBuf>) -> error::Result<()> {
fn secret_file(net_config_dir: &PathBuf) -> error::Result<()> {
NodeKeyType::variants().iter().try_for_each(|t| {
let mut config = Configuration::default();
let node_key_type = NodeKeyType::from_str(t).unwrap();
let tmp = tempfile::Builder::new().prefix("alice").tempdir()?;
let file = tmp.path().join(format!("{}_mysecret", t)).to_path_buf();
let params = NodeKeyParams {
node_key_type,
node_key: None,
node_key_file: Some(file.clone())
node_key_file: Some(file.clone()),
};
params.update_config(&mut config, net_config_dir).and_then(|c| match c {
params.node_key(net_config_dir).and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::File(ref f))
if node_key_type == NodeKeyType::Ed25519 && f == &file => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
if node_key_type == NodeKeyType::Ed25519 && f == &file =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(secret_file(None).is_ok());
assert!(secret_file(Some(&PathBuf::from_str("x").unwrap())).is_ok());
assert!(secret_file(&PathBuf::from_str("x").unwrap()).is_ok());
}
#[test]
fn test_node_key_config_default() {
fn with_def_params<F>(f: F) -> error::Result<()>
where
F: Fn(NodeKeyParams) -> error::Result<()>
F: Fn(NodeKeyParams) -> error::Result<()>,
{
NodeKeyType::variants().iter().try_for_each(|t| {
let node_key_type = NodeKeyType::from_str(t).unwrap();
f(NodeKeyParams {
node_key_type,
node_key: None,
node_key_file: None
node_key_file: None,
})
})
}
fn no_config_dir() -> error::Result<()> {
with_def_params(|params| {
let mut config = Configuration::default();
let typ = params.node_key_type;
params.update_config(&mut config, None)
.and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::New)
if typ == NodeKeyType::Ed25519 => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
})
})
}
fn some_config_dir(net_config_dir: &PathBuf) -> error::Result<()> {
with_def_params(|params| {
let mut config = Configuration::default();
let dir = PathBuf::from(net_config_dir.clone());
let typ = params.node_key_type;
params.update_config(&mut config, Some(net_config_dir))
params
.node_key(net_config_dir)
.and_then(move |c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::File(ref f))
if typ == NodeKeyType::Ed25519 &&
f == &dir.join(NODE_KEY_ED25519_FILE) => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
})
if typ == NodeKeyType::Ed25519
&& f == &dir.join(NODE_KEY_ED25519_FILE) =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(no_config_dir().is_ok());
assert!(some_config_dir(&PathBuf::from_str("x").unwrap()).is_ok());
}
}