CLI: refactoring: remove Options from sc_service::Configuration's fields (#5271)

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Rename IntoConfiguration to CliConfiguration

* Renamed into_configuration to create_configuration

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Move keystore params to its own module

* Use in-memory keystore even for build-spec

* Enforce proper value for node name

* dev_key_seed

* Telemetry endpoints

* rustfmt

* Converted all RunCmd

* rustfmt

* Added export-blocks

* Missed something

* Removed config_path in NetworkConfiguration (not used)

* Fixed warnings

* public_addresses is used but never set, keeping it

* Merge Configuration.node and NetworkConfiguration.node_name

...because they are the same thing

* Added: import-blocks

* Adding a proc_macro to help impl SubstrateCli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Re-export spec_factory from sc_cli

* Re-added all the commands

* Refactored node_key_params

* Fixed previous refucktoring

* Clean-up and removed full_version()

* Renamed get_is_dev to not confuse with Configuration field

* Fixed sc-cli-derive example

* Fixing tests

* Fixing tests and removing some (will re-add later)

* Fixing more tests

* Removes the need of type parameter

* Converting bin/node and simplifying API

* Converting more

* Converting last command

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fixing tests and added default for WasmExecutionMethod

* Fixing stuff

* Fixed something I broke oops

* Update Cargo.lock

* Moving things around

* Convert everything to Result

* Added new macros to simplify the impl of CliConfiguration

* Added a macro to generate CliConfiguration automatically for subcommands

* Revert... too many macros (this one is not really useful)

This reverts commit 9c516dd38b40fbc420b02c1f8e61d5b2b1a4e434.

* Renamed is_dev to get_is_dev

Good enough for now

* Fixed name roles (this is plural, not singular)

* Clean-up

* Re-export NodeKeyConfig and TelemetryEndpoints from sc_service

* Improve styling/formatting

* Added copyrights

* Added doc and fixed warnings

* Added myself to code owners

* Yes it is needed according to the history

* Revert formatting

* Fixing conflict

* Updated build.rs

* Cargo.lock

* Clean-up

* Update client/cli-derive/Cargo.toml

Co-Authored-By: Seun Lanlege <seunlanlege@gmail.com>

* Fail if using proc_macro and build.rs is not set properly

* Dropped all get_ in front of methods

* Clean-up

* Fixing proc macro missing env var

* Get the configuration inside the Runtime (needed for polkadot)

* Clean-up

* Get is_dev from argument like the others

* Get chain ID instead of chain spec from shared params

* &self is passed to spec_factory/load_spec

* Wrong text

* Fix example

* Officialize macro and made a cool doc

* Renamed spec_factory to load_spec (substrate_cli_configuration)

* Removed not so useful ChainSpec

* Renamed SubstrateCLI to SubstrateCli

* Added changelog for impl_version being full now

* Renamed Runtime to Runner

* Update changelog to show example

* Removed option on database cache size

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fix on removal of option

* typo

* Clean-up imports

* Added info in Cargo.toml

* typo

* remarks

* Moved function for build.rs to substrate-build-script-utils

* Fixed example & test of cli-derive

* Moved function for build.rs to substrate-build-script-utils

* Renamed substrate_cli_configuration to substrate_cli oops

It implements SubstrateCli not CliConfiguration!

* Added documentation and wrapper macro

* Removed option on database cache size

* Removed option on database cache size

* Clean-up

* Reduce risk of errors due to typos

* Removed option on database cache size

* Added NOTE as suggested

* Added doc as suggested

* Fixed test

* typo

* renamed runtime to runner

* Fixed weird argument

* More commas

* Moved client/cli-derive to client/cli/derive

* Added 7 tests for the macros

* Improve error message

* Upgrade assert_cmd

* Fixing missing stuff

* Fixed unused import

* Improve SubstrateCli doc

* Applied suggestions

* Fix and clean-up imports

* Started replacing macros WIP

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Started removing substrate_cli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* fixed bug introduced while refactoring

* Renamed NetworkConfigurationParams to NetworkParams for consistency sake

* Fixed test

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/check_block_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 5906776953392c02beac6bc0bf50f8cbe1a12a01.

* Revert "Update client/cli/src/commands/check_block_cmd.rs"

This reverts commit f705f42b7f3d732be001141afee210fe46a1ef47.

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 8d57c0550164449e6eb2d3bacb04c750c714fcea.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 93e74cf5d2e1c0dc49cdff8608d59fc40fc59338.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 11d527ba345c0d79f0d3b5b071933d95474d0614.

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/import_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Changed ::sc_cli to $crate in the macro

* fixed tests

* fixed conflicts

* Fixing test

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove comment as suggested

* Apply suggestion

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/node_key_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/network_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added doc

* Fixed error introduced after applying suggestion

* Revert "Update client/cli/src/params/pruning_params.rs"

This reverts commit 0574d06a4f1efd86e94c1214420a12e7a4be0099.

* Print error

* Apply suggestions from code review

* Remove useless Results

* Fixed CI failing on polkadot approval

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This commit is contained in:
Cecile Tonglet
2020-04-07 11:38:07 +02:00
committed by GitHub
parent fb9bbf306d
commit 3da069e359
66 changed files with 2916 additions and 2156 deletions
+330 -92
View File
@@ -14,35 +14,24 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
mod runcmd;
mod export_blocks_cmd;
mod build_spec_cmd;
mod import_blocks_cmd;
mod check_block_cmd;
mod revert_cmd;
mod export_blocks_cmd;
mod import_blocks_cmd;
mod purge_chain_cmd;
mod revert_cmd;
mod runcmd;
pub use crate::commands::build_spec_cmd::BuildSpecCmd;
pub use crate::commands::check_block_cmd::CheckBlockCmd;
pub use crate::commands::export_blocks_cmd::ExportBlocksCmd;
pub use crate::commands::import_blocks_cmd::ImportBlocksCmd;
pub use crate::commands::purge_chain_cmd::PurgeChainCmd;
pub use crate::commands::revert_cmd::RevertCmd;
pub use crate::commands::runcmd::RunCmd;
use std::fmt::Debug;
use structopt::StructOpt;
use sc_service::{ Configuration, ServiceBuilderCommand, ChainSpec };
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use crate::error;
use crate::VersionInfo;
use crate::params::SharedParams;
pub use crate::commands::runcmd::RunCmd;
pub use crate::commands::export_blocks_cmd::ExportBlocksCmd;
pub use crate::commands::build_spec_cmd::BuildSpecCmd;
pub use crate::commands::import_blocks_cmd::ImportBlocksCmd;
pub use crate::commands::check_block_cmd::CheckBlockCmd;
pub use crate::commands::revert_cmd::RevertCmd;
pub use crate::commands::purge_chain_cmd::PurgeChainCmd;
/// default sub directory to store network config
const DEFAULT_NETWORK_CONFIG_PATH : &'static str = "network";
/// All core commands that are provided by default.
///
/// The core commands are split into multiple subcommands and `Run` is the default subcommand. From
@@ -51,89 +40,338 @@ const DEFAULT_NETWORK_CONFIG_PATH : &'static str = "network";
#[derive(Debug, Clone, StructOpt)]
pub enum Subcommand {
/// Build a spec.json file, outputs to stdout.
BuildSpec(build_spec_cmd::BuildSpecCmd),
BuildSpec(BuildSpecCmd),
/// Export blocks to a file.
ExportBlocks(export_blocks_cmd::ExportBlocksCmd),
ExportBlocks(ExportBlocksCmd),
/// Import blocks from file.
ImportBlocks(import_blocks_cmd::ImportBlocksCmd),
ImportBlocks(ImportBlocksCmd),
/// Validate a single block.
CheckBlock(check_block_cmd::CheckBlockCmd),
CheckBlock(CheckBlockCmd),
/// Revert chain to the previous state.
Revert(revert_cmd::RevertCmd),
Revert(RevertCmd),
/// Remove the whole chain data.
PurgeChain(purge_chain_cmd::PurgeChainCmd),
PurgeChain(PurgeChainCmd),
}
impl Subcommand {
/// Get the shared parameters of a `CoreParams` command.
pub fn get_shared_params(&self) -> &SharedParams {
use Subcommand::*;
// TODO: move to config.rs?
/// Macro that helps implement CliConfiguration on an enum of subcommand automatically
///
/// # Example
///
/// ```
/// # #[macro_use] extern crate sc_cli;
///
/// # struct EmptyVariant {}
///
/// # impl sc_cli::CliConfiguration for EmptyVariant {
/// # fn shared_params(&self) -> &sc_cli::SharedParams { unimplemented!() }
/// # fn chain_id(&self, _: bool) -> sc_cli::Result<String> { Ok("test-chain-id".to_string()) }
/// # }
///
/// # fn main() {
/// enum Subcommand {
/// Variant1(EmptyVariant),
/// Variant2(EmptyVariant),
/// }
///
/// substrate_cli_subcommands!(
/// Subcommand => Variant1, Variant2
/// );
///
/// # use sc_cli::CliConfiguration;
/// # assert_eq!(Subcommand::Variant1(EmptyVariant {}).chain_id(false).unwrap(), "test-chain-id");
///
/// # }
/// ```
///
/// Which will expand to:
///
/// ```ignore
/// impl CliConfiguration for Subcommand {
/// fn base_path(&self) -> Result<Option<PathBuf>> {
/// match self {
/// Subcommand::Variant1(cmd) => cmd.base_path(),
/// Subcommand::Variant2(cmd) => cmd.base_path(),
/// }
/// }
///
/// fn is_dev(&self) -> Result<bool> {
/// match self {
/// Subcommand::Variant1(cmd) => cmd.is_dev(),
/// Subcommand::Variant2(cmd) => cmd.is_dev(),
/// }
/// }
///
/// // ...
/// }
/// ```
#[macro_export]
macro_rules! substrate_cli_subcommands {
($enum:ident => $($variant:ident),*) => {
impl $crate::CliConfiguration for $enum {
fn shared_params(&self) -> &$crate::SharedParams {
match self {
$($enum::$variant(cmd) => cmd.shared_params()),*
}
}
match self {
BuildSpec(params) => &params.shared_params,
ExportBlocks(params) => &params.shared_params,
ImportBlocks(params) => &params.shared_params,
CheckBlock(params) => &params.shared_params,
Revert(params) => &params.shared_params,
PurgeChain(params) => &params.shared_params,
fn import_params(&self) -> Option<&$crate::ImportParams> {
match self {
$($enum::$variant(cmd) => cmd.import_params()),*
}
}
fn pruning_params(&self) -> Option<&$crate::PruningParams> {
match self {
$($enum::$variant(cmd) => cmd.pruning_params()),*
}
}
fn keystore_params(&self) -> Option<&$crate::KeystoreParams> {
match self {
$($enum::$variant(cmd) => cmd.keystore_params()),*
}
}
fn network_params(&self) -> Option<&$crate::NetworkParams> {
match self {
$($enum::$variant(cmd) => cmd.network_params()),*
}
}
fn base_path(&self) -> $crate::Result<::std::option::Option<::std::path::PathBuf>> {
match self {
$($enum::$variant(cmd) => cmd.base_path()),*
}
}
fn is_dev(&self) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.is_dev()),*
}
}
fn role(&self, is_dev: bool) -> $crate::Result<::sc_service::Role> {
match self {
$($enum::$variant(cmd) => cmd.role(is_dev)),*
}
}
fn transaction_pool(&self)
-> $crate::Result<::sc_service::config::TransactionPoolOptions> {
match self {
$($enum::$variant(cmd) => cmd.transaction_pool()),*
}
}
fn network_config(
&self,
chain_spec: &::std::boxed::Box<dyn ::sc_service::ChainSpec>,
is_dev: bool,
net_config_dir: &::std::path::PathBuf,
client_id: &str,
node_name: &str,
node_key: ::sc_service::config::NodeKeyConfig,
) -> $crate::Result<::sc_service::config::NetworkConfiguration> {
match self {
$(
$enum::$variant(cmd) => cmd.network_config(
chain_spec, is_dev, net_config_dir, client_id, node_name, node_key
)
),*
}
}
fn keystore_config(&self, base_path: &::std::path::PathBuf)
-> $crate::Result<::sc_service::config::KeystoreConfig> {
match self {
$($enum::$variant(cmd) => cmd.keystore_config(base_path)),*
}
}
fn database_cache_size(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.database_cache_size()),*
}
}
fn database_config(
&self,
base_path: &::std::path::PathBuf,
cache_size: usize,
) -> $crate::Result<::sc_service::config::DatabaseConfig> {
match self {
$($enum::$variant(cmd) => cmd.database_config(base_path, cache_size)),*
}
}
fn state_cache_size(&self) -> $crate::Result<usize> {
match self {
$($enum::$variant(cmd) => cmd.state_cache_size()),*
}
}
fn state_cache_child_ratio(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.state_cache_child_ratio()),*
}
}
fn pruning(&self, is_dev: bool, role: &::sc_service::Role)
-> $crate::Result<::sc_service::config::PruningMode> {
match self {
$($enum::$variant(cmd) => cmd.pruning(is_dev, role)),*
}
}
fn chain_id(&self, is_dev: bool) -> $crate::Result<String> {
match self {
$($enum::$variant(cmd) => cmd.chain_id(is_dev)),*
}
}
fn init<C: $crate::SubstrateCli>(&self) -> $crate::Result<()> {
match self {
$($enum::$variant(cmd) => cmd.init::<C>()),*
}
}
fn node_name(&self) -> $crate::Result<String> {
match self {
$($enum::$variant(cmd) => cmd.node_name()),*
}
}
fn wasm_method(&self) -> $crate::Result<::sc_service::config::WasmExecutionMethod> {
match self {
$($enum::$variant(cmd) => cmd.wasm_method()),*
}
}
fn execution_strategies(&self, is_dev: bool)
-> $crate::Result<::sc_service::config::ExecutionStrategies> {
match self {
$($enum::$variant(cmd) => cmd.execution_strategies(is_dev)),*
}
}
fn rpc_http(&self) -> $crate::Result<::std::option::Option<::std::net::SocketAddr>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_http()),*
}
}
fn rpc_ws(&self) -> $crate::Result<::std::option::Option<::std::net::SocketAddr>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_ws()),*
}
}
fn rpc_ws_max_connections(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_ws_max_connections()),*
}
}
fn rpc_cors(&self, is_dev: bool)
-> $crate::Result<::std::option::Option<::std::vec::Vec<String>>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_cors(is_dev)),*
}
}
fn prometheus_config(&self)
-> $crate::Result<::std::option::Option<::sc_service::config::PrometheusConfig>> {
match self {
$($enum::$variant(cmd) => cmd.prometheus_config()),*
}
}
fn telemetry_endpoints(
&self,
chain_spec: &Box<dyn ::sc_service::ChainSpec>,
) -> $crate::Result<::std::option::Option<::sc_service::config::TelemetryEndpoints>> {
match self {
$($enum::$variant(cmd) => cmd.telemetry_endpoints(chain_spec)),*
}
}
fn telemetry_external_transport(&self)
-> $crate::Result<::std::option::Option<::sc_service::config::ExtTransport>> {
match self {
$($enum::$variant(cmd) => cmd.telemetry_external_transport()),*
}
}
fn default_heap_pages(&self) -> $crate::Result<::std::option::Option<u64>> {
match self {
$($enum::$variant(cmd) => cmd.default_heap_pages()),*
}
}
fn offchain_worker(&self, role: &::sc_service::Role) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.offchain_worker(role)),*
}
}
fn force_authoring(&self) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.force_authoring()),*
}
}
fn disable_grandpa(&self) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.disable_grandpa()),*
}
}
fn dev_key_seed(&self, is_dev: bool) -> $crate::Result<::std::option::Option<String>> {
match self {
$($enum::$variant(cmd) => cmd.dev_key_seed(is_dev)),*
}
}
fn tracing_targets(&self) -> $crate::Result<::std::option::Option<String>> {
match self {
$($enum::$variant(cmd) => cmd.tracing_targets()),*
}
}
fn tracing_receiver(&self) -> $crate::Result<::sc_service::TracingReceiver> {
match self {
$($enum::$variant(cmd) => cmd.tracing_receiver()),*
}
}
fn node_key(&self, net_config_dir: &::std::path::PathBuf)
-> $crate::Result<::sc_service::config::NodeKeyConfig> {
match self {
$($enum::$variant(cmd) => cmd.node_key(net_config_dir)),*
}
}
fn max_runtime_instances(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.max_runtime_instances()),*
}
}
fn log_filters(&self) -> $crate::Result<::std::option::Option<String>> {
match self {
$($enum::$variant(cmd) => cmd.log_filters()),*
}
}
}
}
/// Run any `CoreParams` command.
pub fn run<B, BC, BB>(
self,
config: Configuration,
builder: B,
) -> error::Result<()>
where
B: FnOnce(Configuration) -> Result<BC, sc_service::error::Error>,
BC: ServiceBuilderCommand<Block = BB> + Unpin,
BB: sp_runtime::traits::Block + Debug,
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: std::fmt::Debug,
<BB as BlockT>::Hash: std::str::FromStr,
{
match self {
Subcommand::BuildSpec(cmd) => cmd.run(config),
Subcommand::ExportBlocks(cmd) => cmd.run(config, builder),
Subcommand::ImportBlocks(cmd) => cmd.run(config, builder),
Subcommand::CheckBlock(cmd) => cmd.run(config, builder),
Subcommand::PurgeChain(cmd) => cmd.run(config),
Subcommand::Revert(cmd) => cmd.run(config, builder),
}
}
/// Update and prepare a `Configuration` with command line parameters.
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
match self {
Subcommand::BuildSpec(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::ExportBlocks(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::ImportBlocks(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::CheckBlock(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::PurgeChain(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::Revert(cmd) => cmd.update_config(&mut config, spec_factory, version),
}
}
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
pub fn init(&self, version: &VersionInfo) -> error::Result<()> {
self.get_shared_params().init(version)
}
}
substrate_cli_subcommands!(
Subcommand => BuildSpec, ExportBlocks, ImportBlocks, CheckBlock, Revert, PurgeChain
);