pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-09 20:11:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

CLI: refactoring: remove Options from sc_service::Configuration's fields (#5271)

Browse Source 
* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Rename IntoConfiguration to CliConfiguration

* Renamed into_configuration to create_configuration

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Move keystore params to its own module

* Use in-memory keystore even for build-spec

* Enforce proper value for node name

* dev_key_seed

* Telemetry endpoints

* rustfmt

* Converted all RunCmd

* rustfmt

* Added export-blocks

* Missed something

* Removed config_path in NetworkConfiguration (not used)

* Fixed warnings

* public_addresses is used but never set, keeping it

* Merge Configuration.node and NetworkConfiguration.node_name

...because they are the same thing

* Added: import-blocks

* Adding a proc_macro to help impl SubstrateCli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Re-export spec_factory from sc_cli

* Re-added all the commands

* Refactored node_key_params

* Fixed previous refucktoring

* Clean-up and removed full_version()

* Renamed get_is_dev to not confuse with Configuration field

* Fixed sc-cli-derive example

* Fixing tests

* Fixing tests and removing some (will re-add later)

* Fixing more tests

* Removes the need of type parameter

* Converting bin/node and simplifying API

* Converting more

* Converting last command

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fixing tests and added default for WasmExecutionMethod

* Fixing stuff

* Fixed something I broke oops

* Update Cargo.lock

* Moving things around

* Convert everything to Result

* Added new macros to simplify the impl of CliConfiguration

* Added a macro to generate CliConfiguration automatically for subcommands

* Revert... too many macros (this one is not really useful)

This reverts commit 9c516dd38b40fbc420b02c1f8e61d5b2b1a4e434.

* Renamed is_dev to get_is_dev

Good enough for now

* Fixed name roles (this is plural, not singular)

* Clean-up

* Re-export NodeKeyConfig and TelemetryEndpoints from sc_service

* Improve styling/formatting

* Added copyrights

* Added doc and fixed warnings

* Added myself to code owners

* Yes it is needed according to the history

* Revert formatting

* Fixing conflict

* Updated build.rs

* Cargo.lock

* Clean-up

* Update client/cli-derive/Cargo.toml

Co-Authored-By: Seun Lanlege <seunlanlege@gmail.com>

* Fail if using proc_macro and build.rs is not set properly

* Dropped all get_ in front of methods

* Clean-up

* Fixing proc macro missing env var

* Get the configuration inside the Runtime (needed for polkadot)

* Clean-up

* Get is_dev from argument like the others

* Get chain ID instead of chain spec from shared params

* &self is passed to spec_factory/load_spec

* Wrong text

* Fix example

* Officialize macro and made a cool doc

* Renamed spec_factory to load_spec (substrate_cli_configuration)

* Removed not so useful ChainSpec

* Renamed SubstrateCLI to SubstrateCli

* Added changelog for impl_version being full now

* Renamed Runtime to Runner

* Update changelog to show example

* Removed option on database cache size

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fix on removal of option

* typo

* Clean-up imports

* Added info in Cargo.toml

* typo

* remarks

* Moved function for build.rs to substrate-build-script-utils

* Fixed example & test of cli-derive

* Moved function for build.rs to substrate-build-script-utils

* Renamed substrate_cli_configuration to substrate_cli oops

It implements SubstrateCli not CliConfiguration!

* Added documentation and wrapper macro

* Removed option on database cache size

* Removed option on database cache size

* Clean-up

* Reduce risk of errors due to typos

* Removed option on database cache size

* Added NOTE as suggested

* Added doc as suggested

* Fixed test

* typo

* renamed runtime to runner

* Fixed weird argument

* More commas

* Moved client/cli-derive to client/cli/derive

* Added 7 tests for the macros

* Improve error message

* Upgrade assert_cmd

* Fixing missing stuff

* Fixed unused import

* Improve SubstrateCli doc

* Applied suggestions

* Fix and clean-up imports

* Started replacing macros WIP

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Started removing substrate_cli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* fixed bug introduced while refactoring

* Renamed NetworkConfigurationParams to NetworkParams for consistency sake

* Fixed test

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/check_block_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 5906776953392c02beac6bc0bf50f8cbe1a12a01.

* Revert "Update client/cli/src/commands/check_block_cmd.rs"

This reverts commit f705f42b7f3d732be001141afee210fe46a1ef47.

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 8d57c0550164449e6eb2d3bacb04c750c714fcea.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 93e74cf5d2e1c0dc49cdff8608d59fc40fc59338.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 11d527ba345c0d79f0d3b5b071933d95474d0614.

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/import_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Changed ::sc_cli to $crate in the macro

* fixed tests

* fixed conflicts

* Fixing test

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove comment as suggested

* Apply suggestion

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/node_key_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/network_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added doc

* Fixed error introduced after applying suggestion

* Revert "Update client/cli/src/params/pruning_params.rs"

This reverts commit 0574d06a4f1efd86e94c1214420a12e7a4be0099.

* Print error

* Apply suggestions from code review

* Remove useless Results

* Fixed CI failing on polkadot approval

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This commit is contained in:
Cecile Tonglet
2020-04-07 11:38:07 +02:00
committed by GitHub
parent fb9bbf306d
commit 3da069e359
66 changed files with 2916 additions and 2156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
substrate/.maintain/gitlab/check_polkadot_companion_status.sh
+2 -1
View File
@@ -87,7 +87,8 @@ fi
curl -H "${github_header}" -sS -o companion_pr_reviews.json \
${github_api_polkadot_pull_url}/${pr_companion}/reviews
if [ "$(jq -r -e '.[].state' < companion_pr_reviews.json | uniq)" != "APPROVED" ]
if [ -n "$(jq -r -e '.[].state | select(. == "CHANGES_REQUESTED")' < companion_pr_reviews.json)" ] && \
[ -z "$(jq -r -e '.[].state | select(. == "APPROVED")' < companion_pr_reviews.json)" ]
then
boldprint "polkadot pr #${pr_companion} not APPROVED"
exit 1
substrate/Cargo.lock
Generated
+11 -19
View File
@@ -3395,7 +3395,9 @@ dependencies = [
"pallet-timestamp",
"pallet-transaction-payment",
"parity-scale-codec",
"platforms",
"rand 0.7.3",
"regex",
"sc-authority-discovery",
"sc-basic-authorship",
"sc-chain-spec",
@@ -3434,7 +3436,6 @@ dependencies = [
"substrate-build-script-utils",
"tempfile",
"tracing",
"vergen",
"wasm-bindgen",
"wasm-bindgen-futures",
]
@@ -3625,7 +3626,6 @@ dependencies = [
"sp-transaction-pool",
"structopt",
"substrate-build-script-utils",
"vergen",
]
[[package]]
@@ -4935,6 +4935,12 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
[[package]]
name = "platforms"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "feb3b2b1033b8a60b4da6ee470325f887758c95d5320f52f9ce0df055a55940e"
[[package]]
name = "plotters"
version = "0.2.12"
@@ -6597,7 +6603,6 @@ dependencies = [
"substrate-prometheus-endpoint",
"substrate-test-runtime-client",
"sysinfo",
"target_info",
"tracing",
"wasm-timer",
]
@@ -7879,6 +7884,9 @@ dependencies = [
[[package]]
name = "substrate-build-script-utils"
version = "2.0.0-alpha.5"
dependencies = [
"platforms",
]
[[package]]
name = "substrate-frame-rpc-support"
@@ -8210,12 +8218,6 @@ version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab0e7238dcc7b40a7be719a25365910f6807bd864f4cce6b2e6b873658e2b19d"
[[package]]
name = "target_info"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c63f48baada5c52e65a29eef93ab4f8982681b67f9e8d29c7b05abcfec2b9ffe"
[[package]]
name = "tempfile"
version = "3.1.0"
@@ -8895,16 +8897,6 @@ version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05c78687fb1a80548ae3250346c3db86a80a7cdd77bda190189f2d0a0987c81a"
[[package]]
name = "vergen"
version = "3.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ce50d8996df1f85af15f2cd8d33daae6e479575123ef4314a51a70a230739cb"
dependencies = [
"bitflags",
"chrono",
]
[[package]]
name = "version_check"
version = "0.9.1"
substrate/bin/node-template/node/Cargo.toml
+2 -2
View File
@@ -2,6 +2,7 @@
name = "node-template"
version = "2.0.0-alpha.5"
authors = ["Anonymous"]
description = "Substrate Node template"
edition = "2018"
license = "Unlicense"
build = "build.rs"
@@ -37,8 +38,7 @@ sc-basic-authorship = { path = "../../../client/basic-authorship", version = "0.
node-template-runtime = { version = "2.0.0-alpha.5", path = "../runtime" }
[build-dependencies]
vergen = "3.0.4"
build-script-utils = { version = "2.0.0-alpha.5", package = "substrate-build-script-utils", path = "../../../utils/build-script-utils" }
substrate-build-script-utils = { version = "2.0.0-alpha.5", path = "../../../utils/build-script-utils" }
[package.metadata.docs.rs]
targets = ["x86_64-unknown-linux-gnu"]
substrate/bin/node-template/node/build.rs
+3 -5
View File
@@ -1,9 +1,7 @@
use vergen::{ConstantsFlags, generate_cargo_keys};
const ERROR_MSG: &str = "Failed to generate metadata files";
use substrate_build_script_utils::{generate_cargo_keys, rerun_if_git_head_changed};
fn main() {
generate_cargo_keys(ConstantsFlags::SHA_SHORT).expect(ERROR_MSG);
generate_cargo_keys();
build_script_utils::rerun_if_git_head_changed();
rerun_if_git_head_changed();
}
substrate/bin/node-template/node/src/chain_spec.rs
+57 -85
View File
@@ -14,17 +14,6 @@ use sp_runtime::traits::{Verify, IdentifyAccount};
/// Specialized `ChainSpec`. This is a specialization of the general Substrate ChainSpec type.
pub type ChainSpec = sc_service::GenericChainSpec<GenesisConfig>;
/// The chain specification option. This is expected to come in from the CLI and
/// is little more than one of a number of alternatives which can easily be converted
/// from a string (`--chain=...`) into a `ChainSpec`.
#[derive(Clone, Debug)]
pub enum Alternative {
/// Whatever the current runtime is, with just Alice as an auth.
Development,
/// Whatever the current runtime is, with simple Alice/Bob auths.
LocalTestnet,
}
/// Helper function to generate a crypto pair from seed
pub fn get_from_seed<TPublic: Public>(seed: &str) -> <TPublic::Pair as Pair>::Public {
TPublic::Pair::from_string(&format!("//{}", seed), None)
@@ -42,80 +31,70 @@ pub fn get_account_id_from_seed<TPublic: Public>(seed: &str) -> AccountId where
}
/// Helper function to generate an authority key for Aura
pub fn get_authority_keys_from_seed(s: &str) -> (AuraId, GrandpaId) {
pub fn authority_keys_from_seed(s: &str) -> (AuraId, GrandpaId) {
(
get_from_seed::<AuraId>(s),
get_from_seed::<GrandpaId>(s),
)
}
impl Alternative {
/// Get an actual chain config from one of the alternatives.
pub(crate) fn load(self) -> Result<ChainSpec, String> {
Ok(match self {
Alternative::Development => ChainSpec::from_genesis(
"Development",
"dev",
|| testnet_genesis(
vec![
get_authority_keys_from_seed("Alice"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
vec![
get_account_id_from_seed::<sr25519::Public>("Alice"),
get_account_id_from_seed::<sr25519::Public>("Bob"),
get_account_id_from_seed::<sr25519::Public>("Alice//stash"),
get_account_id_from_seed::<sr25519::Public>("Bob//stash"),
],
true,
),
vec![],
None,
None,
None,
None
),
Alternative::LocalTestnet => ChainSpec::from_genesis(
"Local Testnet",
"local_testnet",
|| testnet_genesis(
vec![
get_authority_keys_from_seed("Alice"),
get_authority_keys_from_seed("Bob"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
vec![
get_account_id_from_seed::<sr25519::Public>("Alice"),
get_account_id_from_seed::<sr25519::Public>("Bob"),
get_account_id_from_seed::<sr25519::Public>("Charlie"),
get_account_id_from_seed::<sr25519::Public>("Dave"),
get_account_id_from_seed::<sr25519::Public>("Eve"),
get_account_id_from_seed::<sr25519::Public>("Ferdie"),
get_account_id_from_seed::<sr25519::Public>("Alice//stash"),
get_account_id_from_seed::<sr25519::Public>("Bob//stash"),
get_account_id_from_seed::<sr25519::Public>("Charlie//stash"),
get_account_id_from_seed::<sr25519::Public>("Dave//stash"),
get_account_id_from_seed::<sr25519::Public>("Eve//stash"),
get_account_id_from_seed::<sr25519::Public>("Ferdie//stash"),
],
true,
),
vec![],
None,
None,
None,
None
),
})
}
pub fn development_config() -> ChainSpec {
ChainSpec::from_genesis(
"Development",
"dev",
|| testnet_genesis(
vec![
authority_keys_from_seed("Alice"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
vec![
get_account_id_from_seed::<sr25519::Public>("Alice"),
get_account_id_from_seed::<sr25519::Public>("Bob"),
get_account_id_from_seed::<sr25519::Public>("Alice//stash"),
get_account_id_from_seed::<sr25519::Public>("Bob//stash"),
],
true,
),
vec![],
None,
None,
None,
None,
)
}
pub(crate) fn from(s: &str) -> Option<Self> {
match s {
"dev" => Some(Alternative::Development),
"" | "local" => Some(Alternative::LocalTestnet),
_ => None,
}
}
pub fn local_testnet_config() -> ChainSpec {
ChainSpec::from_genesis(
"Local Testnet",
"local_testnet",
|| testnet_genesis(
vec![
authority_keys_from_seed("Alice"),
authority_keys_from_seed("Bob"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
vec![
get_account_id_from_seed::<sr25519::Public>("Alice"),
get_account_id_from_seed::<sr25519::Public>("Bob"),
get_account_id_from_seed::<sr25519::Public>("Charlie"),
get_account_id_from_seed::<sr25519::Public>("Dave"),
get_account_id_from_seed::<sr25519::Public>("Eve"),
get_account_id_from_seed::<sr25519::Public>("Ferdie"),
get_account_id_from_seed::<sr25519::Public>("Alice//stash"),
get_account_id_from_seed::<sr25519::Public>("Bob//stash"),
get_account_id_from_seed::<sr25519::Public>("Charlie//stash"),
get_account_id_from_seed::<sr25519::Public>("Dave//stash"),
get_account_id_from_seed::<sr25519::Public>("Eve//stash"),
get_account_id_from_seed::<sr25519::Public>("Ferdie//stash"),
],
true,
),
vec![],
None,
None,
None,
None,
)
}
fn testnet_genesis(initial_authorities: Vec<(AuraId, GrandpaId)>,
@@ -141,10 +120,3 @@ fn testnet_genesis(initial_authorities: Vec<(AuraId, GrandpaId)>,
}),
}
}
pub fn load_spec(id: &str) -> Result<Box<dyn sc_service::ChainSpec>, String> {
Ok(match Alternative::from(id) {
Some(spec) => Box::new(spec.load()?),
None => Box::new(ChainSpec::from_json_file(std::path::PathBuf::from(id))?),
})
}
substrate/bin/node-template/node/src/command.rs
+54 -26
View File
@@ -14,36 +14,64 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sp_consensus_aura::sr25519::{AuthorityPair as AuraPair};
use sc_cli::VersionInfo;
use crate::service;
use crate::chain_spec;
use crate::cli::Cli;
use crate::service;
use sc_cli::SubstrateCli;
use sp_consensus_aura::sr25519::AuthorityPair as AuraPair;
/// Parse and run command line arguments
pub fn run(version: VersionInfo) -> sc_cli::Result<()> {
let opt = sc_cli::from_args::<Cli>(&version);
impl SubstrateCli for Cli {
fn impl_name() -> &'static str {
"Substrate Node"
}
let mut config = sc_service::Configuration::from_version(&version);
fn impl_version() -> &'static str {
env!("SUBSTRATE_CLI_IMPL_VERSION")
}
match opt.subcommand {
Some(subcommand) => {
subcommand.init(&version)?;
subcommand.update_config(&mut config, chain_spec::load_spec, &version)?;
subcommand.run(
config,
|config: _| Ok(new_full_start!(config).0),
)
},
None => {
opt.run.init(&version)?;
opt.run.update_config(&mut config, chain_spec::load_spec, &version)?;
opt.run.run(
config,
service::new_light,
service::new_full,
&version,
)
},
fn description() -> &'static str {
env!("CARGO_PKG_DESCRIPTION")
}
fn author() -> &'static str {
env!("CARGO_PKG_AUTHORS")
}
fn support_url() -> &'static str {
"support.anonymous.an"
}
fn copyright_start_year() -> i32 {
2017
}
fn executable_name() -> &'static str {
env!("CARGO_PKG_NAME")
}
fn load_spec(&self, id: &str) -> Result<Box<dyn sc_service::ChainSpec>, String> {
Ok(match id {
"dev" => Box::new(chain_spec::development_config()),
"" | "local" => Box::new(chain_spec::local_testnet_config()),
path => Box::new(chain_spec::ChainSpec::from_json_file(
std::path::PathBuf::from(path),
)?),
})
}
}
/// Parse and run command line arguments
pub fn run() -> sc_cli::Result<()> {
let cli = Cli::from_args();
match &cli.subcommand {
Some(subcommand) => {
let runner = cli.create_runner(subcommand)?;
runner.run_subcommand(subcommand, |config| Ok(new_full_start!(config).0))
}
None => {
let runner = cli.create_runner(&cli.run)?;
runner.run_node(service::new_light, service::new_full)
}
}
}
substrate/bin/node-template/node/src/main.rs
+1 -12
View File
@@ -8,16 +8,5 @@ mod cli;
mod command;
fn main() -> sc_cli::Result<()> {
let version = sc_cli::VersionInfo {
name: "Substrate Node",
commit: env!("VERGEN_SHA_SHORT"),
version: env!("CARGO_PKG_VERSION"),
executable_name: "node-template",
author: "Anonymous",
description: "Template Node",
support_url: "support.anonymous.an",
copyright_start_year: 2017,
};
command::run(version)
command::run()
}
substrate/bin/node-template/node/src/service.rs
+1 -1
View File
@@ -74,7 +74,7 @@ pub fn new_full(config: Configuration)
{
let role = config.role.clone();
let force_authoring = config.force_authoring;
let name = config.name.clone();
let name = config.network.node_name.clone();
let disable_grandpa = config.disable_grandpa;
let (builder, mut import_setup, inherent_data_providers) = new_full_start!(config);
substrate/bin/node/cli/Cargo.toml
+5 -7
View File
@@ -2,7 +2,7 @@
name = "node-cli"
version = "2.0.0-alpha.5"
authors = ["Parity Technologies <admin@parity.io>"]
description = "Substrate node implementation in Rust."
description = "Generic Substrate node implementation in Rust."
build = "build.rs"
edition = "2018"
license = "GPL-3.0"
@@ -116,13 +116,15 @@ tempfile = "3.1.0"
assert_cmd = "1.0"
nix = "0.17"
serde_json = "1.0"
regex = "1"
platforms = "0.2.1"
[build-dependencies]
build-script-utils = { version = "2.0.0-alpha.5", package = "substrate-build-script-utils", path = "../../../utils/build-script-utils" }
structopt = { version = "0.3.8", optional = true }
node-transaction-factory = { version = "0.8.0-alpha.5", optional = true, path = "../transaction-factory" }
node-inspect = { version = "0.8.0-alpha.5", optional = true, path = "../inspect" }
frame-benchmarking-cli = { version = "2.0.0-alpha.5", optional = true, path = "../../../utils/frame/benchmarking-cli" }
substrate-build-script-utils = { version = "2.0.0-alpha.5", optional = true, path = "../../../utils/build-script-utils" }
[build-dependencies.sc-cli]
version = "0.8.0-alpha.5"
@@ -130,10 +132,6 @@ package = "sc-cli"
path = "../../../client/cli"
optional = true
[build-dependencies.vergen]
version = "3.0.4"
optional = true
[features]
default = [ "cli" ]
browser = [
@@ -149,7 +147,7 @@ cli = [
"frame-benchmarking-cli",
"sc-service/rocksdb",
"structopt",
"vergen",
"substrate-build-script-utils",
]
runtime-benchmarks = [ "node-runtime/runtime-benchmarks" ]
substrate/bin/node/cli/bin/main.rs
+1 -12
View File
@@ -19,16 +19,5 @@
#![warn(missing_docs)]
fn main() -> sc_cli::Result<()> {
let version = sc_cli::VersionInfo {
name: "Substrate Node",
commit: env!("VERGEN_SHA_SHORT"),
version: env!("CARGO_PKG_VERSION"),
executable_name: "substrate",
author: "Parity Technologies <admin@parity.io>",
description: "Generic substrate node",
support_url: "https://github.com/paritytech/substrate/issues/new",
copyright_start_year: 2017,
};
node_cli::run(std::env::args(), version)
node_cli::run()
}
substrate/bin/node/cli/build.rs
+4 -4
View File
@@ -24,14 +24,14 @@ mod cli {
include!("src/cli.rs");
use std::{fs, env, path::Path};
use sc_cli::{structopt::clap::Shell};
use vergen::{ConstantsFlags, generate_cargo_keys};
use sc_cli::structopt::clap::Shell;
use substrate_build_script_utils::{generate_cargo_keys, rerun_if_git_head_changed};
pub fn main() {
build_shell_completion();
generate_cargo_keys(ConstantsFlags::all()).expect("Failed to generate metadata files");
generate_cargo_keys();
build_script_utils::rerun_if_git_head_changed();
rerun_if_git_head_changed();
}
/// Build shell completion scripts for all known shells
substrate/bin/node/cli/src/browser.rs
+3 -3
View File
@@ -41,10 +41,10 @@ async fn start_inner(chain_spec: String, log_level: String) -> Result<Client, Bo
let config = browser_configuration(chain_spec).await?;
info!("Substrate browser node");
info!("✌️ version {}", config.full_version());
info!("✌️ version {}", config.impl_version);
info!("❤️ by Parity Technologies, 2017-2020");
info!("📋 Chain specification: {}", config.expect_chain_spec().name());
info!("🏷 Node name: {}", config.name);
info!("📋 Chain specification: {}", config.chain_spec.name());
info!("🏷 Node name: {}", config.network.node_name);
info!("👤 Role: {:?}", config.role);
// Create the service. This is the most heavy initialization step.
substrate/bin/node/cli/src/chain_spec.rs
+5 -5
View File
@@ -183,7 +183,7 @@ pub fn get_account_id_from_seed<TPublic: Public>(seed: &str) -> AccountId where
}
/// Helper function to generate stash, controller and session key from seed
pub fn get_authority_keys_from_seed(seed: &str) -> (
pub fn authority_keys_from_seed(seed: &str) -> (
AccountId,
AccountId,
GrandpaId,
@@ -325,7 +325,7 @@ pub fn testnet_genesis(
fn development_config_genesis() -> GenesisConfig {
testnet_genesis(
vec![
get_authority_keys_from_seed("Alice"),
authority_keys_from_seed("Alice"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
None,
@@ -350,8 +350,8 @@ pub fn development_config() -> ChainSpec {
fn local_testnet_genesis() -> GenesisConfig {
testnet_genesis(
vec![
get_authority_keys_from_seed("Alice"),
get_authority_keys_from_seed("Bob"),
authority_keys_from_seed("Alice"),
authority_keys_from_seed("Bob"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
None,
@@ -383,7 +383,7 @@ pub(crate) mod tests {
fn local_testnet_genesis_instant_single() -> GenesisConfig {
testnet_genesis(
vec![
get_authority_keys_from_seed("Alice"),
authority_keys_from_seed("Alice"),
],
get_account_id_from_seed::<sr25519::Public>("Alice"),
None,
substrate/bin/node/cli/src/cli.rs
+4 -7
View File
@@ -14,7 +14,7 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sc_cli::{SharedParams, ImportParams, RunCmd};
use sc_cli::{ImportParams, RunCmd, SharedParams};
use structopt::StructOpt;
/// An overarching CLI command definition.
@@ -50,10 +50,7 @@ pub enum Subcommand {
Inspect(node_inspect::cli::InspectCmd),
/// The custom benchmark subcommmand benchmarking runtime pallets.
#[structopt(
name = "benchmark",
about = "Benchmark runtime pallets."
)]
#[structopt(name = "benchmark", about = "Benchmark runtime pallets.")]
Benchmark(frame_benchmarking_cli::BenchmarkCmd),
}
@@ -62,11 +59,11 @@ pub enum Subcommand {
#[derive(Debug, StructOpt, Clone)]
pub struct FactoryCmd {
/// Number of blocks to generate.
#[structopt(long="blocks", default_value = "1")]
#[structopt(long = "blocks", default_value = "1")]
pub blocks: u32,
/// Number of transactions to push per block.
#[structopt(long="transactions", default_value = "8")]
#[structopt(long = "transactions", default_value = "8")]
pub transactions: u32,
#[allow(missing_docs)]
substrate/bin/node/cli/src/command.rs
+112 -90
View File
@@ -14,104 +14,126 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sc_cli::VersionInfo;
use sc_service::{Role as ServiceRole};
use crate::{chain_spec, factory_impl::FactoryState, service, Cli, FactoryCmd, Subcommand};
use node_executor::Executor;
use node_runtime::{Block, RuntimeApi};
use node_transaction_factory::RuntimeAdapter;
use crate::{Cli, service, ChainSpec, load_spec, Subcommand, factory_impl::FactoryState};
use sc_cli::{CliConfiguration, ImportParams, Result, SharedParams, SubstrateCli};
use sc_service::Configuration;
/// Parse command line arguments into service configuration.
pub fn run<I, T>(args: I, version: VersionInfo) -> sc_cli::Result<()>
where
I: Iterator<Item = T>,
T: Into<std::ffi::OsString> + Clone,
{
sc_cli::reset_signal_pipe_handler()?;
impl SubstrateCli for Cli {
fn impl_name() -> &'static str {
"Substrate Node"
}
let args: Vec<_> = args.collect();
let opt = sc_cli::from_iter::<Cli, _>(args.clone(), &version);
fn impl_version() -> &'static str {
env!("SUBSTRATE_CLI_IMPL_VERSION")
}
let mut config = sc_service::Configuration::from_version(&version);
fn description() -> &'static str {
env!("CARGO_PKG_DESCRIPTION")
}
match opt.subcommand {
None => {
opt.run.init(&version)?;
opt.run.update_config(&mut config, load_spec, &version)?;
opt.run.run(
config,
service::new_light,
service::new_full,
&version,
)
},
Some(Subcommand::Inspect(cmd)) => {
cmd.init(&version)?;
cmd.update_config(&mut config, load_spec, &version)?;
fn author() -> &'static str {
env!("CARGO_PKG_AUTHORS")
}
let client = sc_service::new_full_client::<
node_runtime::Block, node_runtime::RuntimeApi, node_executor::Executor,
>(&config)?;
let inspect = node_inspect::Inspector::<node_runtime::Block>::new(client);
fn support_url() -> &'static str {
"https://github.com/paritytech/substrate/issues/new"
}
cmd.run(inspect)
},
Some(Subcommand::Benchmark(cmd)) => {
cmd.init(&version)?;
cmd.update_config(&mut config, load_spec, &version)?;
fn copyright_start_year() -> i32 {
2017
}
cmd.run::<node_runtime::Block, node_executor::Executor>(config)
},
Some(Subcommand::Factory(cli_args)) => {
cli_args.shared_params.init(&version)?;
cli_args.shared_params.update_config(&mut config, load_spec, &version)?;
cli_args.import_params.update_config(
&mut config,
&ServiceRole::Full,
cli_args.shared_params.dev,
)?;
fn executable_name() -> &'static str {
"substrate"
}
config.use_in_memory_keystore()?;
match ChainSpec::from(config.expect_chain_spec().id()) {
Some(ref c) if c == &ChainSpec::Development || c == &ChainSpec::LocalTestnet => {},
_ => return Err(
"Factory is only supported for development and local testnet.".into()
),
}
// Setup tracing.
if let Some(tracing_targets) = cli_args.import_params.tracing_targets.as_ref() {
let subscriber = sc_tracing::ProfilingSubscriber::new(
cli_args.import_params.tracing_receiver.into(), tracing_targets
);
if let Err(e) = tracing::subscriber::set_global_default(subscriber) {
return Err(
format!("Unable to set global default subscriber {}", e).into()
);
}
}
let factory_state = FactoryState::new(
cli_args.blocks,
cli_args.transactions,
);
let service_builder = new_full_start!(config).0;
node_transaction_factory::factory(
factory_state,
service_builder.client(),
service_builder.select_chain()
.expect("The select_chain is always initialized by new_full_start!; QED")
).map_err(|e| format!("Error in transaction factory: {}", e))?;
Ok(())
},
Some(Subcommand::Base(subcommand)) => {
subcommand.init(&version)?;
subcommand.update_config(&mut config, load_spec, &version)?;
subcommand.run(
config,
|config: sc_service::Configuration| Ok(new_full_start!(config).0),
)
},
fn load_spec(&self, id: &str) -> std::result::Result<Box<dyn sc_service::ChainSpec>, String> {
Ok(match id {
"dev" => Box::new(chain_spec::development_config()),
"local" => Box::new(chain_spec::local_testnet_config()),
"" | "fir" | "flaming-fir" => Box::new(chain_spec::flaming_fir_config()?),
"staging" => Box::new(chain_spec::staging_testnet_config()),
path => Box::new(chain_spec::ChainSpec::from_json_file(
std::path::PathBuf::from(path),
)?),
})
}
}
/// Parse command line arguments into service configuration.
pub fn run() -> Result<()> {
sc_cli::reset_signal_pipe_handler()?;
let cli = Cli::from_args();
match &cli.subcommand {
None => {
let runner = cli.create_runner(&cli.run)?;
runner.run_node(service::new_light, service::new_full)
}
Some(Subcommand::Inspect(cmd)) => {
let runner = cli.create_runner(cmd)?;
runner.sync_run(|config| cmd.run::<Block, RuntimeApi, Executor>(config))
}
Some(Subcommand::Benchmark(cmd)) => {
let runner = cli.create_runner(cmd)?;
runner.sync_run(|config| cmd.run::<Block, Executor>(config))
}
Some(Subcommand::Factory(cmd)) => {
let runner = cli.create_runner(cmd)?;
runner.sync_run(|config| cmd.run(config))
}
Some(Subcommand::Base(subcommand)) => {
let runner = cli.create_runner(subcommand)?;
runner.run_subcommand(subcommand, |config| Ok(new_full_start!(config).0))
}
}
}
impl CliConfiguration for FactoryCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
fn import_params(&self) -> Option<&ImportParams> {
Some(&self.import_params)
}
}
impl FactoryCmd {
fn run(&self, config: Configuration) -> Result<()> {
match config.chain_spec.id() {
"dev" | "local" => {}
_ => return Err("Factory is only supported for development and local testnet.".into()),
}
// Setup tracing.
if let Some(tracing_targets) = self.import_params.tracing_targets.as_ref() {
let subscriber = sc_tracing::ProfilingSubscriber::new(
self.import_params.tracing_receiver.into(),
tracing_targets,
);
if let Err(e) = tracing::subscriber::set_global_default(subscriber) {
return Err(format!("Unable to set global default subscriber {}", e).into());
}
}
let factory_state = FactoryState::new(self.blocks, self.transactions);
let service_builder = new_full_start!(config).0;
node_transaction_factory::factory(
factory_state,
service_builder.client(),
service_builder
.select_chain()
.expect("The select_chain is always initialized by new_full_start!; qed"),
)
}
}
substrate/bin/node/cli/src/lib.rs
-42
View File
@@ -47,45 +47,3 @@ pub use browser::*;
pub use cli::*;
#[cfg(feature = "cli")]
pub use command::*;
/// The chain specification option.
#[derive(Clone, Debug, PartialEq)]
pub enum ChainSpec {
/// Whatever the current runtime is, with just Alice as an auth.
Development,
/// Whatever the current runtime is, with simple Alice/Bob auths.
LocalTestnet,
/// The Flaming Fir testnet.
FlamingFir,
/// Whatever the current runtime is with the "global testnet" defaults.
StagingTestnet,
}
/// Get a chain config from a spec setting.
impl ChainSpec {
pub(crate) fn load(self) -> Result<chain_spec::ChainSpec, String> {
Ok(match self {
ChainSpec::FlamingFir => chain_spec::flaming_fir_config()?,
ChainSpec::Development => chain_spec::development_config(),
ChainSpec::LocalTestnet => chain_spec::local_testnet_config(),
ChainSpec::StagingTestnet => chain_spec::staging_testnet_config(),
})
}
pub(crate) fn from(s: &str) -> Option<Self> {
match s {
"dev" => Some(ChainSpec::Development),
"local" => Some(ChainSpec::LocalTestnet),
"" | "fir" | "flaming-fir" => Some(ChainSpec::FlamingFir),
"staging" => Some(ChainSpec::StagingTestnet),
_ => None,
}
}
}
fn load_spec(id: &str) -> Result<Box<dyn sc_service::ChainSpec>, String> {
Ok(match ChainSpec::from(id) {
Some(spec) => Box::new(spec.load()?),
None => Box::new(chain_spec::ChainSpec::from_json_file(std::path::PathBuf::from(id))?),
})
}
substrate/bin/node/cli/src/service.rs
+2 -2
View File
@@ -88,7 +88,7 @@ macro_rules! new_full_start {
import_setup = Some((block_import, grandpa_link, babe_link));
Ok(import_queue)
})?
.with_rpc_extensions(|builder| -> Result<RpcExtension, _> {
.with_rpc_extensions(|builder| -> std::result::Result<RpcExtension, _> {
let babe_link = import_setup.as_ref().map(|s| &s.2)
.expect("BabeLink is present for full services or set up failed; qed.");
let deps = node_rpc::FullDeps {
@@ -127,7 +127,7 @@ macro_rules! new_full {
) = (
$config.role.clone(),
$config.force_authoring,
$config.name.clone(),
$config.network.node_name.clone(),
$config.disable_grandpa,
);
substrate/bin/node/cli/tests/version.rs
+83
View File
@@ -0,0 +1,83 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use assert_cmd::cargo::cargo_bin;
use platforms::*;
use regex::Regex;
use std::process::Command;
fn expected_regex() -> Regex {
Regex::new(r"^substrate (\d+\.\d+\.\d+(?:-.+?)?)-([a-f\d]+)-(.+?)-(.+?)(?:-(.+))?$").unwrap()
}
#[test]
fn version_is_full() {
let expected = expected_regex();
let output = Command::new(cargo_bin("substrate"))
.args(&["--version"])
.output()
.unwrap();
assert!(
output.status.success(),
"command returned with non-success exit code"
);
let output = String::from_utf8_lossy(&output.stdout).trim().to_owned();
let captures = expected
.captures(output.as_str())
.expect("could not parse version in output");
assert_eq!(&captures[1], env!("CARGO_PKG_VERSION"));
assert_eq!(&captures[3], TARGET_ARCH.as_str());
assert_eq!(&captures[4], TARGET_OS.as_str());
assert_eq!(
captures.get(5).map(|x| x.as_str()),
TARGET_ENV.map(|x| x.as_str())
);
}
#[test]
fn test_regex_matches_properly() {
let expected = expected_regex();
let captures = expected
.captures("substrate 2.0.0-da487d19d-x86_64-linux-gnu")
.unwrap();
assert_eq!(&captures[1], "2.0.0");
assert_eq!(&captures[2], "da487d19d");
assert_eq!(&captures[3], "x86_64");
assert_eq!(&captures[4], "linux");
assert_eq!(captures.get(5).map(|x| x.as_str()), Some("gnu"));
let captures = expected
.captures("substrate 2.0.0-alpha.5-da487d19d-x86_64-linux-gnu")
.unwrap();
assert_eq!(&captures[1], "2.0.0-alpha.5");
assert_eq!(&captures[2], "da487d19d");
assert_eq!(&captures[3], "x86_64");
assert_eq!(&captures[4], "linux");
assert_eq!(captures.get(5).map(|x| x.as_str()), Some("gnu"));
let captures = expected
.captures("substrate 2.0.0-alpha.5-da487d19d-x86_64-linux")
.unwrap();
assert_eq!(&captures[1], "2.0.0-alpha.5");
assert_eq!(&captures[2], "da487d19d");
assert_eq!(&captures[3], "x86_64");
assert_eq!(&captures[4], "linux");
assert_eq!(captures.get(5).map(|x| x.as_str()), None);
}
substrate/bin/node/inspect/src/command.rs
+23 -161
View File
@@ -16,186 +16,48 @@
//! Command ran by the CLI
use std::{
fmt::Debug,
str::FromStr,
};
use crate::cli::{InspectCmd, InspectSubCmd};
use crate::{Inspector, PrettyPrinter};
use crate::Inspector;
use sc_cli::{CliConfiguration, ImportParams, Result, SharedParams};
use sc_service::{new_full_client, Configuration, NativeExecutionDispatch};
use sp_runtime::traits::Block;
use std::str::FromStr;
impl InspectCmd {
/// Initialize
pub fn init(&self, version: &sc_cli::VersionInfo) -> sc_cli::Result<()> {
self.shared_params.init(version)
}
/// Parse CLI arguments and initialize given config.
pub fn update_config(
&self,
mut config: &mut sc_service::config::Configuration,
spec_factory: impl FnOnce(&str) -> Result<Box<dyn sc_service::ChainSpec>, String>,
version: &sc_cli::VersionInfo,
) -> sc_cli::Result<()> {
self.shared_params.update_config(config, spec_factory, version)?;
// make sure to configure keystore
config.use_in_memory_keystore()?;
// and all import params (especially pruning that has to match db meta)
self.import_params.update_config(
&mut config,
&sc_service::Role::Full,
self.shared_params.dev,
)?;
Ok(())
}
/// Run the inspect command, passing the inspector.
pub fn run<B, P>(
self,
inspect: Inspector<B, P>,
) -> sc_cli::Result<()> where
B: sp_runtime::traits::Block,
pub fn run<B, RA, EX>(&self, config: Configuration) -> Result<()>
where
B: Block,
B::Hash: FromStr,
P: PrettyPrinter<B>,
RA: Send + Sync + 'static,
EX: NativeExecutionDispatch + 'static,
{
match self.command {
let client = new_full_client::<B, RA, EX>(&config)?;
let inspect = Inspector::<B>::new(client);
match &self.command {
InspectSubCmd::Block { input } => {
let input = input.parse()?;
let res = inspect.block(input)
.map_err(|e| format!("{}", e))?;
let res = inspect.block(input).map_err(|e| format!("{}", e))?;
println!("{}", res);
Ok(())
},
}
InspectSubCmd::Extrinsic { input } => {
let input = input.parse()?;
let res = inspect.extrinsic(input)
.map_err(|e| format!("{}", e))?;
let res = inspect.extrinsic(input).map_err(|e| format!("{}", e))?;
println!("{}", res);
Ok(())
},
}
}
}
}
/// A block to retrieve.
#[derive(Debug, Clone, PartialEq)]
pub enum BlockAddress<Hash, Number> {
/// Get block by hash.
Hash(Hash),
/// Get block by number.
Number(Number),
/// Raw SCALE-encoded bytes.
Bytes(Vec<u8>),
}
impl<Hash: FromStr, Number: FromStr> FromStr for BlockAddress<Hash, Number> {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// try to parse hash first
if let Ok(hash) = s.parse() {
return Ok(Self::Hash(hash))
}
// then number
if let Ok(number) = s.parse() {
return Ok(Self::Number(number))
}
// then assume it's bytes (hex-encoded)
sp_core::bytes::from_hex(s)
.map(Self::Bytes)
.map_err(|e| format!(
"Given string does not look like hash or number. It could not be parsed as bytes either: {}",
e
))
}
}
/// An extrinsic address to decode and print out.
#[derive(Debug, Clone, PartialEq)]
pub enum ExtrinsicAddress<Hash, Number> {
/// Extrinsic as part of existing block.
Block(BlockAddress<Hash, Number>, usize),
/// Raw SCALE-encoded extrinsic bytes.
Bytes(Vec<u8>),
}
impl<Hash: FromStr + Debug, Number: FromStr + Debug> FromStr for ExtrinsicAddress<Hash, Number> {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// first try raw bytes
if let Ok(bytes) = sp_core::bytes::from_hex(s).map(Self::Bytes) {
return Ok(bytes)
}
// split by a bunch of different characters
let mut it = s.split(|c| c == '.' || c == ':' || c == ' ');
let block = it.next()
.expect("First element of split iterator is never empty; qed")
.parse()?;
let index = it.next()
.ok_or_else(|| format!("Extrinsic index missing: example \"5:0\""))?
.parse()
.map_err(|e| format!("Invalid index format: {}", e))?;
Ok(Self::Block(block, index))
}
}
#[cfg(test)]
mod tests {
use super::*;
use sp_core::hash::H160 as Hash;
#[test]
fn should_parse_block_strings() {
type BlockAddress = super::BlockAddress<Hash, u64>;
let b0 = BlockAddress::from_str("3BfC20f0B9aFcAcE800D73D2191166FF16540258");
let b1 = BlockAddress::from_str("1234");
let b2 = BlockAddress::from_str("0");
let b3 = BlockAddress::from_str("0x0012345f");
assert_eq!(b0, Ok(BlockAddress::Hash(
"3BfC20f0B9aFcAcE800D73D2191166FF16540258".parse().unwrap()
)));
assert_eq!(b1, Ok(BlockAddress::Number(1234)));
assert_eq!(b2, Ok(BlockAddress::Number(0)));
assert_eq!(b3, Ok(BlockAddress::Bytes(vec![0, 0x12, 0x34, 0x5f])));
impl CliConfiguration for InspectCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
#[test]
fn should_parse_extrinsic_address() {
type BlockAddress = super::BlockAddress<Hash, u64>;
type ExtrinsicAddress = super::ExtrinsicAddress<Hash, u64>;
let e0 = ExtrinsicAddress::from_str("1234");
let b0 = ExtrinsicAddress::from_str("3BfC20f0B9aFcAcE800D73D2191166FF16540258:5");
let b1 = ExtrinsicAddress::from_str("1234:0");
let b2 = ExtrinsicAddress::from_str("0 0");
let b3 = ExtrinsicAddress::from_str("0x0012345f");
assert_eq!(e0, Err("Extrinsic index missing: example \"5:0\"".into()));
assert_eq!(b0, Ok(ExtrinsicAddress::Block(
BlockAddress::Hash("3BfC20f0B9aFcAcE800D73D2191166FF16540258".parse().unwrap()),
5
)));
assert_eq!(b1, Ok(ExtrinsicAddress::Block(
BlockAddress::Number(1234),
0
)));
assert_eq!(b2, Ok(ExtrinsicAddress::Block(
BlockAddress::Number(0),
0
)));
assert_eq!(b3, Ok(ExtrinsicAddress::Bytes(vec![0, 0x12, 0x34, 0x5f])));
fn import_params(&self) -> Option<&ImportParams> {
Some(&self.import_params)
}
}
substrate/bin/node/inspect/src/lib.rs
+123 -3
View File
@@ -27,7 +27,9 @@ pub mod command;
use std::{
fmt,
marker::PhantomData
fmt::Debug,
marker::PhantomData,
str::FromStr,
};
use codec::{Encode, Decode};
use sc_client_api::BlockBackend;
@@ -38,8 +40,6 @@ use sp_runtime::{
traits::{Block, HashFor, NumberFor, Hash}
};
use command::{BlockAddress, ExtrinsicAddress};
/// A helper type for a generic block input.
pub type BlockAddressFor<TBlock> = BlockAddress<
<HashFor<TBlock> as Hash>::Output,
@@ -205,3 +205,123 @@ impl<TBlock: Block, TPrinter: PrettyPrinter<TBlock>> Inspector<TBlock, TPrinter>
Ok(format!("{}", ExtrinsicPrinter(ext, &self.printer)))
}
}
/// A block to retrieve.
#[derive(Debug, Clone, PartialEq)]
pub enum BlockAddress<Hash, Number> {
/// Get block by hash.
Hash(Hash),
/// Get block by number.
Number(Number),
/// Raw SCALE-encoded bytes.
Bytes(Vec<u8>),
}
impl<Hash: FromStr, Number: FromStr> FromStr for BlockAddress<Hash, Number> {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// try to parse hash first
if let Ok(hash) = s.parse() {
return Ok(Self::Hash(hash))
}
// then number
if let Ok(number) = s.parse() {
return Ok(Self::Number(number))
}
// then assume it's bytes (hex-encoded)
sp_core::bytes::from_hex(s)
.map(Self::Bytes)
.map_err(|e| format!(
"Given string does not look like hash or number. It could not be parsed as bytes either: {}",
e
))
}
}
/// An extrinsic address to decode and print out.
#[derive(Debug, Clone, PartialEq)]
pub enum ExtrinsicAddress<Hash, Number> {
/// Extrinsic as part of existing block.
Block(BlockAddress<Hash, Number>, usize),
/// Raw SCALE-encoded extrinsic bytes.
Bytes(Vec<u8>),
}
impl<Hash: FromStr + Debug, Number: FromStr + Debug> FromStr for ExtrinsicAddress<Hash, Number> {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
// first try raw bytes
if let Ok(bytes) = sp_core::bytes::from_hex(s).map(Self::Bytes) {
return Ok(bytes)
}
// split by a bunch of different characters
let mut it = s.split(|c| c == '.' || c == ':' || c == ' ');
let block = it.next()
.expect("First element of split iterator is never empty; qed")
.parse()?;
let index = it.next()
.ok_or_else(|| format!("Extrinsic index missing: example \"5:0\""))?
.parse()
.map_err(|e| format!("Invalid index format: {}", e))?;
Ok(Self::Block(block, index))
}
}
#[cfg(test)]
mod tests {
use super::*;
use sp_core::hash::H160 as Hash;
#[test]
fn should_parse_block_strings() {
type BlockAddress = super::BlockAddress<Hash, u64>;
let b0 = BlockAddress::from_str("3BfC20f0B9aFcAcE800D73D2191166FF16540258");
let b1 = BlockAddress::from_str("1234");
let b2 = BlockAddress::from_str("0");
let b3 = BlockAddress::from_str("0x0012345f");
assert_eq!(b0, Ok(BlockAddress::Hash(
"3BfC20f0B9aFcAcE800D73D2191166FF16540258".parse().unwrap()
)));
assert_eq!(b1, Ok(BlockAddress::Number(1234)));
assert_eq!(b2, Ok(BlockAddress::Number(0)));
assert_eq!(b3, Ok(BlockAddress::Bytes(vec![0, 0x12, 0x34, 0x5f])));
}
#[test]
fn should_parse_extrinsic_address() {
type BlockAddress = super::BlockAddress<Hash, u64>;
type ExtrinsicAddress = super::ExtrinsicAddress<Hash, u64>;
let e0 = ExtrinsicAddress::from_str("1234");
let b0 = ExtrinsicAddress::from_str("3BfC20f0B9aFcAcE800D73D2191166FF16540258:5");
let b1 = ExtrinsicAddress::from_str("1234:0");
let b2 = ExtrinsicAddress::from_str("0 0");
let b3 = ExtrinsicAddress::from_str("0x0012345f");
assert_eq!(e0, Err("Extrinsic index missing: example \"5:0\"".into()));
assert_eq!(b0, Ok(ExtrinsicAddress::Block(
BlockAddress::Hash("3BfC20f0B9aFcAcE800D73D2191166FF16540258".parse().unwrap()),
5
)));
assert_eq!(b1, Ok(ExtrinsicAddress::Block(
BlockAddress::Number(1234),
0
)));
assert_eq!(b2, Ok(ExtrinsicAddress::Block(
BlockAddress::Number(0),
0
)));
assert_eq!(b3, Ok(ExtrinsicAddress::Bytes(vec![0, 0x12, 0x34, 0x5f])));
}
}
substrate/bin/node/testing/src/bench.rs
+1 -1
View File
@@ -186,7 +186,7 @@ impl BenchDb {
pruning: PruningMode::ArchiveAll,
source: sc_client_db::DatabaseSettingsSrc::Path {
path: dir.into(),
cache_size: None,
cache_size: 128,
},
};
substrate/bin/utils/chain-spec-builder/src/main.rs
+2 -2
View File
@@ -87,7 +87,7 @@ fn genesis_constructor(
let authorities = authority_seeds
.iter()
.map(AsRef::as_ref)
.map(chain_spec::get_authority_keys_from_seed)
.map(chain_spec::authority_keys_from_seed)
.collect::<Vec<_>>();
let enable_println = true;
@@ -142,7 +142,7 @@ fn generate_authority_keys_and_store(
).map_err(|err| err.to_string())?;
let (_, _, grandpa, babe, im_online, authority_discovery) =
chain_spec::get_authority_keys_from_seed(seed);
chain_spec::authority_keys_from_seed(seed);
let insert_key = |key_type, public| {
keystore.write().insert_unknown(
substrate/client/cli/src/arg_enums.rs
+3 -1
View File
@@ -45,7 +45,9 @@ impl WasmExecutionMethod {
impl Into<sc_service::config::WasmExecutionMethod> for WasmExecutionMethod {
fn into(self) -> sc_service::config::WasmExecutionMethod {
match self {
WasmExecutionMethod::Interpreted => sc_service::config::WasmExecutionMethod::Interpreted,
WasmExecutionMethod::Interpreted => {
sc_service::config::WasmExecutionMethod::Interpreted
}
#[cfg(feature = "wasmtime")]
WasmExecutionMethod::Compiled => sc_service::config::WasmExecutionMethod::Compiled,
#[cfg(not(feature = "wasmtime"))]
substrate/client/cli/src/commands/build_spec_cmd.rs
+17 -32
View File
@@ -14,15 +14,14 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use structopt::StructOpt;
use log::info;
use sc_network::config::{build_multiaddr, MultiaddrWithPeerId};
use sc_service::{Configuration, ChainSpec};
use crate::error;
use crate::VersionInfo;
use crate::params::SharedParams;
use crate::params::NodeKeyParams;
use crate::params::SharedParams;
use crate::CliConfiguration;
use log::info;
use sc_network::config::build_multiaddr;
use sc_service::{config::MultiaddrWithPeerId, Configuration};
use structopt::StructOpt;
/// The `build-spec` command used to build a specification.
#[derive(Debug, StructOpt, Clone)]
@@ -49,12 +48,9 @@ pub struct BuildSpecCmd {
impl BuildSpecCmd {
/// Run the build-spec command
pub fn run(
self,
config: Configuration,
) -> error::Result<()> {
pub fn run(&self, config: Configuration) -> error::Result<()> {
info!("Building chain spec");
let mut spec = config.chain_spec.expect("`chain_spec` is set to `Some` in `update_config`");
let mut spec = config.chain_spec;
let raw_output = self.raw;
if spec.boot_nodes().is_empty() && !self.disable_default_bootnode {
@@ -73,25 +69,14 @@ impl BuildSpecCmd {
Ok(())
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
let net_config_path = config
.in_chain_config_dir(crate::commands::DEFAULT_NETWORK_CONFIG_PATH)
.expect("We provided a base_path");
self.node_key_params.update_config(&mut config, Some(&net_config_path))?;
Ok(())
}
}
impl CliConfiguration for BuildSpecCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
fn node_key_params(&self) -> Option<&NodeKeyParams> {
Some(&self.node_key_params)
}
}
substrate/client/cli/src/commands/check_block_cmd.rs
+28 -32
View File
@@ -14,20 +14,16 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::error;
use crate::params::ImportParams;
use crate::params::SharedParams;
use crate::CliConfiguration;
use sc_service::{Configuration, ServiceBuilderCommand};
use sp_runtime::generic::BlockId;
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use std::fmt::Debug;
use std::str::FromStr;
use structopt::StructOpt;
use sc_service::{
Configuration, ServiceBuilderCommand, Role, ChainSpec,
};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use sp_runtime::generic::BlockId;
use crate::error;
use crate::VersionInfo;
use crate::runtime::run_until_exit;
use crate::params::SharedParams;
use crate::params::ImportParams;
/// The `check-block` command used to validate blocks.
#[derive(Debug, StructOpt, Clone)]
@@ -53,8 +49,8 @@ pub struct CheckBlockCmd {
impl CheckBlockCmd {
/// Run the check-block command
pub fn run<B, BC, BB>(
self,
pub async fn run<B, BC, BB>(
&self,
config: Configuration,
builder: B,
) -> error::Result<()>
@@ -65,37 +61,37 @@ impl CheckBlockCmd {
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: std::fmt::Debug,
<BB as BlockT>::Hash: std::str::FromStr,
{
let input = if self.input.starts_with("0x") { &self.input[2..] } else { &self.input[..] };
let input = if self.input.starts_with("0x") {
&self.input[2..]
} else {
&self.input[..]
};
let block_id = match FromStr::from_str(input) {
Ok(hash) => BlockId::hash(hash),
Err(_) => match self.input.parse::<u32>() {
Ok(n) => BlockId::number((n as u32).into()),
Err(_) => return Err(error::Error::Input("Invalid hash or number specified".into())),
}
Err(_) => {
return Err(error::Error::Input(
"Invalid hash or number specified".into(),
))
}
},
};
let start = std::time::Instant::now();
run_until_exit(config, |config| {
Ok(builder(config)?.check_block(block_id))
})?;
builder(config)?.check_block(block_id).await?;
println!("Completed in {} ms.", start.elapsed().as_millis());
Ok(())
}
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
self.import_params.update_config(&mut config, &Role::Full, self.shared_params.dev)?;
config.use_in_memory_keystore()?;
impl CliConfiguration for CheckBlockCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
Ok(())
fn import_params(&self) -> Option<&ImportParams> {
Some(&self.import_params)
}
}
substrate/client/cli/src/commands/export_blocks_cmd.rs
+27 -34
View File
@@ -14,22 +14,19 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::io;
use std::fs;
use std::path::PathBuf;
use std::fmt::Debug;
use crate::error;
use crate::params::{BlockNumber, PruningParams, SharedParams};
use crate::CliConfiguration;
use log::info;
use structopt::StructOpt;
use sc_service::{
Configuration, ServiceBuilderCommand, ChainSpec,
config::DatabaseConfig, Role,
config::DatabaseConfig, Configuration, ServiceBuilderCommand,
};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use crate::error;
use crate::VersionInfo;
use crate::runtime::run_until_exit;
use crate::params::{SharedParams, BlockNumber, PruningParams};
use std::fmt::Debug;
use std::fs;
use std::io;
use std::path::PathBuf;
use structopt::StructOpt;
/// The `export-blocks` command used to export blocks.
#[derive(Debug, StructOpt, Clone)]
@@ -65,8 +62,8 @@ pub struct ExportBlocksCmd {
impl ExportBlocksCmd {
/// Run the export-blocks command
pub fn run<B, BC, BB>(
self,
pub async fn run<B, BC, BB>(
&self,
config: Configuration,
builder: B,
) -> error::Result<()>
@@ -77,9 +74,10 @@ impl ExportBlocksCmd {
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: std::fmt::Debug,
<BB as BlockT>::Hash: std::str::FromStr,
{
if let DatabaseConfig::Path { ref path, .. } = config.expect_database() {
if let DatabaseConfig::Path { ref path, .. } = &config.database {
info!("DB path: {}", path.display());
}
let from = self.from.as_ref().and_then(|f| f.parse().ok()).unwrap_or(1);
let to = self.to.as_ref().and_then(|t| t.parse().ok());
@@ -90,24 +88,19 @@ impl ExportBlocksCmd {
None => Box::new(io::stdout()),
};
run_until_exit(config, |config| {
Ok(builder(config)?.export_blocks(file, from.into(), to, binary))
})
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
self.pruning_params.update_config(&mut config, &Role::Full, true)?;
config.use_in_memory_keystore()?;
Ok(())
builder(config)?
.export_blocks(file, from.into(), to, binary)
.await
.map_err(Into::into)
}
}
impl CliConfiguration for ExportBlocksCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
fn pruning_params(&self) -> Option<&PruningParams> {
Some(&self.pruning_params)
}
}
substrate/client/cli/src/commands/import_blocks_cmd.rs
+24 -33
View File
@@ -14,21 +14,17 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::error;
use crate::params::ImportParams;
use crate::params::SharedParams;
use crate::CliConfiguration;
use sc_service::{Configuration, ServiceBuilderCommand};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use std::fmt::Debug;
use std::io::{Read, Seek, self};
use std::fs;
use std::io::{self, Read, Seek};
use std::path::PathBuf;
use structopt::StructOpt;
use sc_service::{
Configuration, ServiceBuilderCommand, ChainSpec, Role,
};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use crate::error;
use crate::VersionInfo;
use crate::runtime::run_until_exit;
use crate::params::SharedParams;
use crate::params::ImportParams;
/// The `import-blocks` command used to import blocks.
#[derive(Debug, StructOpt, Clone)]
@@ -59,8 +55,8 @@ impl<T: Read + Seek> ReadPlusSeek for T {}
impl ImportBlocksCmd {
/// Run the import-blocks command
pub fn run<B, BC, BB>(
self,
pub async fn run<B, BC, BB>(
&self,
config: Configuration,
builder: B,
) -> error::Result<()>
@@ -77,27 +73,22 @@ impl ImportBlocksCmd {
let mut buffer = Vec::new();
io::stdin().read_to_end(&mut buffer)?;
Box::new(io::Cursor::new(buffer))
},
}
};
run_until_exit(config, |config| {
Ok(builder(config)?.import_blocks(file, false))
})
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
self.import_params.update_config(&mut config, &Role::Full, self.shared_params.dev)?;
config.use_in_memory_keystore()?;
Ok(())
builder(config)?
.import_blocks(file, false)
.await
.map_err(Into::into)
}
}
impl CliConfiguration for ImportBlocksCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
fn import_params(&self) -> Option<&ImportParams> {
Some(&self.import_params)
}
}
substrate/client/cli/src/commands/mod.rs
+330 -92
View File
@@ -14,35 +14,24 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
mod runcmd;
mod export_blocks_cmd;
mod build_spec_cmd;
mod import_blocks_cmd;
mod check_block_cmd;
mod revert_cmd;
mod export_blocks_cmd;
mod import_blocks_cmd;
mod purge_chain_cmd;
mod revert_cmd;
mod runcmd;
pub use crate::commands::build_spec_cmd::BuildSpecCmd;
pub use crate::commands::check_block_cmd::CheckBlockCmd;
pub use crate::commands::export_blocks_cmd::ExportBlocksCmd;
pub use crate::commands::import_blocks_cmd::ImportBlocksCmd;
pub use crate::commands::purge_chain_cmd::PurgeChainCmd;
pub use crate::commands::revert_cmd::RevertCmd;
pub use crate::commands::runcmd::RunCmd;
use std::fmt::Debug;
use structopt::StructOpt;
use sc_service::{ Configuration, ServiceBuilderCommand, ChainSpec };
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use crate::error;
use crate::VersionInfo;
use crate::params::SharedParams;
pub use crate::commands::runcmd::RunCmd;
pub use crate::commands::export_blocks_cmd::ExportBlocksCmd;
pub use crate::commands::build_spec_cmd::BuildSpecCmd;
pub use crate::commands::import_blocks_cmd::ImportBlocksCmd;
pub use crate::commands::check_block_cmd::CheckBlockCmd;
pub use crate::commands::revert_cmd::RevertCmd;
pub use crate::commands::purge_chain_cmd::PurgeChainCmd;
/// default sub directory to store network config
const DEFAULT_NETWORK_CONFIG_PATH : &'static str = "network";
/// All core commands that are provided by default.
///
/// The core commands are split into multiple subcommands and `Run` is the default subcommand. From
@@ -51,89 +40,338 @@ const DEFAULT_NETWORK_CONFIG_PATH : &'static str = "network";
#[derive(Debug, Clone, StructOpt)]
pub enum Subcommand {
/// Build a spec.json file, outputs to stdout.
BuildSpec(build_spec_cmd::BuildSpecCmd),
BuildSpec(BuildSpecCmd),
/// Export blocks to a file.
ExportBlocks(export_blocks_cmd::ExportBlocksCmd),
ExportBlocks(ExportBlocksCmd),
/// Import blocks from file.
ImportBlocks(import_blocks_cmd::ImportBlocksCmd),
ImportBlocks(ImportBlocksCmd),
/// Validate a single block.
CheckBlock(check_block_cmd::CheckBlockCmd),
CheckBlock(CheckBlockCmd),
/// Revert chain to the previous state.
Revert(revert_cmd::RevertCmd),
Revert(RevertCmd),
/// Remove the whole chain data.
PurgeChain(purge_chain_cmd::PurgeChainCmd),
PurgeChain(PurgeChainCmd),
}
impl Subcommand {
/// Get the shared parameters of a `CoreParams` command.
pub fn get_shared_params(&self) -> &SharedParams {
use Subcommand::*;
// TODO: move to config.rs?
/// Macro that helps implement CliConfiguration on an enum of subcommand automatically
///
/// # Example
///
/// ```
/// # #[macro_use] extern crate sc_cli;
///
/// # struct EmptyVariant {}
///
/// # impl sc_cli::CliConfiguration for EmptyVariant {
/// # fn shared_params(&self) -> &sc_cli::SharedParams { unimplemented!() }
/// # fn chain_id(&self, _: bool) -> sc_cli::Result<String> { Ok("test-chain-id".to_string()) }
/// # }
///
/// # fn main() {
/// enum Subcommand {
/// Variant1(EmptyVariant),
/// Variant2(EmptyVariant),
/// }
///
/// substrate_cli_subcommands!(
/// Subcommand => Variant1, Variant2
/// );
///
/// # use sc_cli::CliConfiguration;
/// # assert_eq!(Subcommand::Variant1(EmptyVariant {}).chain_id(false).unwrap(), "test-chain-id");
///
/// # }
/// ```
///
/// Which will expand to:
///
/// ```ignore
/// impl CliConfiguration for Subcommand {
/// fn base_path(&self) -> Result<Option<PathBuf>> {
/// match self {
/// Subcommand::Variant1(cmd) => cmd.base_path(),
/// Subcommand::Variant2(cmd) => cmd.base_path(),
/// }
/// }
///
/// fn is_dev(&self) -> Result<bool> {
/// match self {
/// Subcommand::Variant1(cmd) => cmd.is_dev(),
/// Subcommand::Variant2(cmd) => cmd.is_dev(),
/// }
/// }
///
/// // ...
/// }
/// ```
#[macro_export]
macro_rules! substrate_cli_subcommands {
($enum:ident => $($variant:ident),*) => {
impl $crate::CliConfiguration for $enum {
fn shared_params(&self) -> &$crate::SharedParams {
match self {
$($enum::$variant(cmd) => cmd.shared_params()),*
}
}
match self {
BuildSpec(params) => &params.shared_params,
ExportBlocks(params) => &params.shared_params,
ImportBlocks(params) => &params.shared_params,
CheckBlock(params) => &params.shared_params,
Revert(params) => &params.shared_params,
PurgeChain(params) => &params.shared_params,
fn import_params(&self) -> Option<&$crate::ImportParams> {
match self {
$($enum::$variant(cmd) => cmd.import_params()),*
}
}
fn pruning_params(&self) -> Option<&$crate::PruningParams> {
match self {
$($enum::$variant(cmd) => cmd.pruning_params()),*
}
}
fn keystore_params(&self) -> Option<&$crate::KeystoreParams> {
match self {
$($enum::$variant(cmd) => cmd.keystore_params()),*
}
}
fn network_params(&self) -> Option<&$crate::NetworkParams> {
match self {
$($enum::$variant(cmd) => cmd.network_params()),*
}
}
fn base_path(&self) -> $crate::Result<::std::option::Option<::std::path::PathBuf>> {
match self {
$($enum::$variant(cmd) => cmd.base_path()),*
}
}
fn is_dev(&self) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.is_dev()),*
}
}
fn role(&self, is_dev: bool) -> $crate::Result<::sc_service::Role> {
match self {
$($enum::$variant(cmd) => cmd.role(is_dev)),*
}
}
fn transaction_pool(&self)
-> $crate::Result<::sc_service::config::TransactionPoolOptions> {
match self {
$($enum::$variant(cmd) => cmd.transaction_pool()),*
}
}
fn network_config(
&self,
chain_spec: &::std::boxed::Box<dyn ::sc_service::ChainSpec>,
is_dev: bool,
net_config_dir: &::std::path::PathBuf,
client_id: &str,
node_name: &str,
node_key: ::sc_service::config::NodeKeyConfig,
) -> $crate::Result<::sc_service::config::NetworkConfiguration> {
match self {
$(
$enum::$variant(cmd) => cmd.network_config(
chain_spec, is_dev, net_config_dir, client_id, node_name, node_key
)
),*
}
}
fn keystore_config(&self, base_path: &::std::path::PathBuf)
-> $crate::Result<::sc_service::config::KeystoreConfig> {
match self {
$($enum::$variant(cmd) => cmd.keystore_config(base_path)),*
}
}
fn database_cache_size(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.database_cache_size()),*
}
}
fn database_config(
&self,
base_path: &::std::path::PathBuf,
cache_size: usize,
) -> $crate::Result<::sc_service::config::DatabaseConfig> {
match self {
$($enum::$variant(cmd) => cmd.database_config(base_path, cache_size)),*
}
}
fn state_cache_size(&self) -> $crate::Result<usize> {
match self {
$($enum::$variant(cmd) => cmd.state_cache_size()),*
}
}
fn state_cache_child_ratio(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.state_cache_child_ratio()),*
}
}
fn pruning(&self, is_dev: bool, role: &::sc_service::Role)
-> $crate::Result<::sc_service::config::PruningMode> {
match self {
$($enum::$variant(cmd) => cmd.pruning(is_dev, role)),*
}
}
fn chain_id(&self, is_dev: bool) -> $crate::Result<String> {
match self {
$($enum::$variant(cmd) => cmd.chain_id(is_dev)),*
}
}
fn init<C: $crate::SubstrateCli>(&self) -> $crate::Result<()> {
match self {
$($enum::$variant(cmd) => cmd.init::<C>()),*
}
}
fn node_name(&self) -> $crate::Result<String> {
match self {
$($enum::$variant(cmd) => cmd.node_name()),*
}
}
fn wasm_method(&self) -> $crate::Result<::sc_service::config::WasmExecutionMethod> {
match self {
$($enum::$variant(cmd) => cmd.wasm_method()),*
}
}
fn execution_strategies(&self, is_dev: bool)
-> $crate::Result<::sc_service::config::ExecutionStrategies> {
match self {
$($enum::$variant(cmd) => cmd.execution_strategies(is_dev)),*
}
}
fn rpc_http(&self) -> $crate::Result<::std::option::Option<::std::net::SocketAddr>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_http()),*
}
}
fn rpc_ws(&self) -> $crate::Result<::std::option::Option<::std::net::SocketAddr>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_ws()),*
}
}
fn rpc_ws_max_connections(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_ws_max_connections()),*
}
}
fn rpc_cors(&self, is_dev: bool)
-> $crate::Result<::std::option::Option<::std::vec::Vec<String>>> {
match self {
$($enum::$variant(cmd) => cmd.rpc_cors(is_dev)),*
}
}
fn prometheus_config(&self)
-> $crate::Result<::std::option::Option<::sc_service::config::PrometheusConfig>> {
match self {
$($enum::$variant(cmd) => cmd.prometheus_config()),*
}
}
fn telemetry_endpoints(
&self,
chain_spec: &Box<dyn ::sc_service::ChainSpec>,
) -> $crate::Result<::std::option::Option<::sc_service::config::TelemetryEndpoints>> {
match self {
$($enum::$variant(cmd) => cmd.telemetry_endpoints(chain_spec)),*
}
}
fn telemetry_external_transport(&self)
-> $crate::Result<::std::option::Option<::sc_service::config::ExtTransport>> {
match self {
$($enum::$variant(cmd) => cmd.telemetry_external_transport()),*
}
}
fn default_heap_pages(&self) -> $crate::Result<::std::option::Option<u64>> {
match self {
$($enum::$variant(cmd) => cmd.default_heap_pages()),*
}
}
fn offchain_worker(&self, role: &::sc_service::Role) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.offchain_worker(role)),*
}
}
fn force_authoring(&self) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.force_authoring()),*
}
}
fn disable_grandpa(&self) -> $crate::Result<bool> {
match self {
$($enum::$variant(cmd) => cmd.disable_grandpa()),*
}
}
fn dev_key_seed(&self, is_dev: bool) -> $crate::Result<::std::option::Option<String>> {
match self {
$($enum::$variant(cmd) => cmd.dev_key_seed(is_dev)),*
}
}
fn tracing_targets(&self) -> $crate::Result<::std::option::Option<String>> {
match self {
$($enum::$variant(cmd) => cmd.tracing_targets()),*
}
}
fn tracing_receiver(&self) -> $crate::Result<::sc_service::TracingReceiver> {
match self {
$($enum::$variant(cmd) => cmd.tracing_receiver()),*
}
}
fn node_key(&self, net_config_dir: &::std::path::PathBuf)
-> $crate::Result<::sc_service::config::NodeKeyConfig> {
match self {
$($enum::$variant(cmd) => cmd.node_key(net_config_dir)),*
}
}
fn max_runtime_instances(&self) -> $crate::Result<::std::option::Option<usize>> {
match self {
$($enum::$variant(cmd) => cmd.max_runtime_instances()),*
}
}
fn log_filters(&self) -> $crate::Result<::std::option::Option<String>> {
match self {
$($enum::$variant(cmd) => cmd.log_filters()),*
}
}
}
}
/// Run any `CoreParams` command.
pub fn run<B, BC, BB>(
self,
config: Configuration,
builder: B,
) -> error::Result<()>
where
B: FnOnce(Configuration) -> Result<BC, sc_service::error::Error>,
BC: ServiceBuilderCommand<Block = BB> + Unpin,
BB: sp_runtime::traits::Block + Debug,
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: std::fmt::Debug,
<BB as BlockT>::Hash: std::str::FromStr,
{
match self {
Subcommand::BuildSpec(cmd) => cmd.run(config),
Subcommand::ExportBlocks(cmd) => cmd.run(config, builder),
Subcommand::ImportBlocks(cmd) => cmd.run(config, builder),
Subcommand::CheckBlock(cmd) => cmd.run(config, builder),
Subcommand::PurgeChain(cmd) => cmd.run(config),
Subcommand::Revert(cmd) => cmd.run(config, builder),
}
}
/// Update and prepare a `Configuration` with command line parameters.
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
match self {
Subcommand::BuildSpec(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::ExportBlocks(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::ImportBlocks(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::CheckBlock(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::PurgeChain(cmd) => cmd.update_config(&mut config, spec_factory, version),
Subcommand::Revert(cmd) => cmd.update_config(&mut config, spec_factory, version),
}
}
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
pub fn init(&self, version: &VersionInfo) -> error::Result<()> {
self.get_shared_params().init(version)
}
}
substrate_cli_subcommands!(
Subcommand => BuildSpec, ExportBlocks, ImportBlocks, CheckBlock, Revert, PurgeChain
);
substrate/client/cli/src/commands/purge_chain_cmd.rs
+13 -26
View File
@@ -14,15 +14,14 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::fmt::Debug;
use std::io::{Write, self};
use std::fs;
use structopt::StructOpt;
use sc_service::{ Configuration, ChainSpec, config::{DatabaseConfig} };
use crate::error;
use crate::VersionInfo;
use crate::params::SharedParams;
use crate::CliConfiguration;
use sc_service::{config::DatabaseConfig, Configuration};
use std::fmt::Debug;
use std::fs;
use std::io::{self, Write};
use structopt::StructOpt;
/// The `purge-chain` command used to remove the whole chain.
#[derive(Debug, StructOpt, Clone)]
@@ -38,11 +37,8 @@ pub struct PurgeChainCmd {
impl PurgeChainCmd {
/// Run the purge command
pub fn run(
self,
config: Configuration,
) -> error::Result<()> {
let db_path = match config.expect_database() {
pub fn run(&self, config: Configuration) -> error::Result<()> {
let db_path = match &config.database {
DatabaseConfig::Path { path, .. } => path,
_ => {
eprintln!("Cannot purge custom database implementation");
@@ -76,22 +72,13 @@ impl PurgeChainCmd {
eprintln!("{:?} did not exist.", &db_path);
Ok(())
},
Err(err) => Result::Err(err.into())
Err(err) => Result::Err(err.into()),
}
}
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
config.use_in_memory_keystore()?;
Ok(())
impl CliConfiguration for PurgeChainCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
}
substrate/client/cli/src/commands/revert_cmd.rs
+13 -26
View File
@@ -14,16 +14,13 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::error;
use crate::params::{BlockNumber, PruningParams, SharedParams};
use crate::CliConfiguration;
use sc_service::{Configuration, ServiceBuilderCommand};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use std::fmt::Debug;
use structopt::StructOpt;
use sc_service::{
Configuration, ServiceBuilderCommand, ChainSpec, Role,
};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use crate::error;
use crate::VersionInfo;
use crate::params::{BlockNumber, SharedParams, PruningParams};
/// The `revert` command used revert the chain to a previous state.
#[derive(Debug, StructOpt, Clone)]
@@ -43,11 +40,7 @@ pub struct RevertCmd {
impl RevertCmd {
/// Run the revert command
pub fn run<B, BC, BB>(
self,
config: Configuration,
builder: B,
) -> error::Result<()>
pub fn run<B, BC, BB>(&self, config: Configuration, builder: B) -> error::Result<()>
where
B: FnOnce(Configuration) -> Result<BC, sc_service::error::Error>,
BC: ServiceBuilderCommand<Block = BB> + Unpin,
@@ -60,20 +53,14 @@ impl RevertCmd {
Ok(())
}
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
self.pruning_params.update_config(&mut config, &Role::Full, true)?;
config.use_in_memory_keystore()?;
impl CliConfiguration for RevertCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
Ok(())
fn pruning_params(&self) -> Option<&PruningParams> {
Some(&self.pruning_params)
}
}
substrate/client/cli/src/commands/runcmd.rs
+226 -383
View File
@@ -14,34 +14,21 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::path::PathBuf;
use std::net::SocketAddr;
use std::fs;
use std::fmt;
use log::info;
use structopt::{StructOpt, clap::arg_enum};
use names::{Generator, Name};
use crate::error::{Error, Result};
use crate::params::ImportParams;
use crate::params::KeystoreParams;
use crate::params::NetworkParams;
use crate::params::SharedParams;
use crate::params::TransactionPoolParams;
use crate::CliConfiguration;
use regex::Regex;
use chrono::prelude::*;
use sc_service::{
AbstractService, Configuration, ChainSpec, Role,
config::{MultiaddrWithPeerId, KeystoreConfig, PrometheusConfig},
config::{MultiaddrWithPeerId, PrometheusConfig, TransactionPoolOptions},
ChainSpec, Role,
};
use sc_telemetry::TelemetryEndpoints;
use crate::VersionInfo;
use crate::error;
use crate::params::ImportParams;
use crate::params::SharedParams;
use crate::params::NetworkConfigurationParams;
use crate::params::TransactionPoolParams;
use crate::runtime::run_service_until_exit;
/// The maximum number of characters for a node name.
const NODE_NAME_MAX_LENGTH: usize = 32;
/// default sub directory for the key store
const DEFAULT_KEYSTORE_CONFIG_PATH : &'static str = "keystore";
use std::net::SocketAddr;
use structopt::{clap::arg_enum, StructOpt};
arg_enum! {
/// Whether off-chain workers are enabled.
@@ -200,7 +187,7 @@ pub struct RunCmd {
#[allow(missing_docs)]
#[structopt(flatten)]
pub network_config: NetworkConfigurationParams,
pub network_params: NetworkParams,
#[allow(missing_docs)]
#[structopt(flatten)]
@@ -242,38 +229,23 @@ pub struct RunCmd {
#[structopt(long = "force-authoring")]
pub force_authoring: bool,
/// Specify custom keystore path.
#[structopt(long = "keystore-path", value_name = "PATH", parse(from_os_str))]
pub keystore_path: Option<PathBuf>,
/// Use interactive shell for entering the password used by the keystore.
#[structopt(
long = "password-interactive",
conflicts_with_all = &[ "password", "password-filename" ]
)]
pub password_interactive: bool,
/// Password used by the keystore.
#[structopt(
long = "password",
conflicts_with_all = &[ "password-interactive", "password-filename" ]
)]
pub password: Option<String>,
/// File that contains the password used by the keystore.
#[structopt(
long = "password-filename",
value_name = "PATH",
parse(from_os_str),
conflicts_with_all = &[ "password-interactive", "password" ]
)]
pub password_filename: Option<PathBuf>,
#[allow(missing_docs)]
#[structopt(flatten)]
pub keystore_params: KeystoreParams,
/// The size of the instances cache for each runtime.
///
/// The default value is 8 and the values higher than 256 are ignored.
#[structopt(long = "max-runtime-instances", default_value = "8")]
pub max_runtime_instances: usize,
#[structopt(long)]
pub max_runtime_instances: Option<usize>,
/// Specify a list of sentry node public addresses.
#[structopt(
long = "sentry-nodes",
value_name = "ADDR",
conflicts_with_all = &[ "sentry" ]
)]
pub sentry_nodes: Vec<MultiaddrWithPeerId>,
}
impl RunCmd {
@@ -281,219 +253,203 @@ impl RunCmd {
pub fn get_keyring(&self) -> Option<sp_keyring::Sr25519Keyring> {
use sp_keyring::Sr25519Keyring::*;
if self.alice { Some(Alice) }
else if self.bob { Some(Bob) }
else if self.charlie { Some(Charlie) }
else if self.dave { Some(Dave) }
else if self.eve { Some(Eve) }
else if self.ferdie { Some(Ferdie) }
else if self.one { Some(One) }
else if self.two { Some(Two) }
else { None }
}
/// Update and prepare a `Configuration` with command line parameters of `RunCmd` and `VersionInfo`.
pub fn update_config<F>(
&self,
mut config: &mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<()>
where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
self.shared_params.update_config(&mut config, spec_factory, version)?;
let password = if self.password_interactive {
#[cfg(not(target_os = "unknown"))]
{
Some(input_keystore_password()?.into())
}
#[cfg(target_os = "unknown")]
None
} else if let Some(ref file) = self.password_filename {
Some(fs::read_to_string(file).map_err(|e| format!("{}", e))?.into())
} else if let Some(ref password) = self.password {
Some(password.clone().into())
if self.alice {
Some(Alice)
} else if self.bob {
Some(Bob)
} else if self.charlie {
Some(Charlie)
} else if self.dave {
Some(Dave)
} else if self.eve {
Some(Eve)
} else if self.ferdie {
Some(Ferdie)
} else if self.one {
Some(One)
} else if self.two {
Some(Two)
} else {
None
};
}
}
}
let path = self.keystore_path.clone().or(
config.in_chain_config_dir(DEFAULT_KEYSTORE_CONFIG_PATH)
);
impl CliConfiguration for RunCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
config.keystore = KeystoreConfig::Path {
path: path.ok_or_else(|| "No `base_path` provided to create keystore path!".to_string())?,
password,
};
fn import_params(&self) -> Option<&ImportParams> {
Some(&self.import_params)
}
let keyring = self.get_keyring();
let is_dev = self.shared_params.dev;
let is_light = self.light;
let is_authority = (self.validator || is_dev || keyring.is_some())
&& !is_light;
let role =
if is_light {
sc_service::Role::Light
} else if is_authority {
sc_service::Role::Authority { sentry_nodes: self.network_config.sentry_nodes.clone() }
} else if !self.sentry.is_empty() {
sc_service::Role::Sentry { validators: self.sentry.clone() }
} else {
sc_service::Role::Full
};
fn network_params(&self) -> Option<&NetworkParams> {
Some(&self.network_params)
}
self.import_params.update_config(&mut config, &role, is_dev)?;
fn keystore_params(&self) -> Option<&KeystoreParams> {
Some(&self.keystore_params)
}
config.name = match (self.name.as_ref(), keyring) {
fn node_name(&self) -> Result<String> {
let name: String = match (self.name.as_ref(), self.get_keyring()) {
(Some(name), _) => name.to_string(),
(_, Some(keyring)) => keyring.to_string(),
(None, None) => generate_node_name(),
(None, None) => crate::generate_node_name(),
};
if let Err(msg) = is_node_name_valid(&config.name) {
return Err(error::Error::Input(
format!("Invalid node name '{}'. Reason: {}. If unsure, use none.",
config.name,
msg,
)
));
}
config.offchain_worker = match (&self.offchain_worker, &role) {
(OffchainWorkerEnabled::WhenValidating, sc_service::Role::Authority { .. }) => true,
is_node_name_valid(&name).map_err(|msg| {
Error::Input(format!(
"Invalid node name '{}'. Reason: {}. If unsure, use none.",
name, msg
));
})?;
Ok(name)
}
fn dev_key_seed(&self, is_dev: bool) -> Result<Option<String>> {
Ok(self.get_keyring().map(|a| format!("//{}", a)).or_else(|| {
if is_dev && !self.light {
Some("//Alice".into())
} else {
None
}
}))
}
fn telemetry_endpoints(
&self,
chain_spec: &Box<dyn ChainSpec>,
) -> Result<Option<TelemetryEndpoints>> {
Ok(if self.no_telemetry {
None
} else if !self.telemetry_endpoints.is_empty() {
Some(
TelemetryEndpoints::new(self.telemetry_endpoints.clone())
.map_err(|e| e.to_string())?,
)
} else {
chain_spec.telemetry_endpoints().clone()
})
}
fn role(&self, is_dev: bool) -> Result<Role> {
let keyring = self.get_keyring();
let is_light = self.light;
let is_authority = (self.validator || is_dev || keyring.is_some()) && !is_light;
Ok(if is_light {
sc_service::Role::Light
} else if is_authority {
sc_service::Role::Authority {
sentry_nodes: self.sentry_nodes.clone(),
}
} else if !self.sentry.is_empty() {
sc_service::Role::Sentry {
validators: self.sentry.clone(),
}
} else {
sc_service::Role::Full
})
}
fn force_authoring(&self) -> Result<bool> {
// Imply forced authoring on --dev
Ok(self.shared_params.dev || self.force_authoring)
}
fn prometheus_config(&self) -> Result<Option<PrometheusConfig>> {
if self.no_prometheus {
Ok(None)
} else {
let prometheus_interface: &str = if self.prometheus_external {
"0.0.0.0"
} else {
"127.0.0.1"
};
Ok(Some(PrometheusConfig::new_with_default_registry(
parse_address(
&format!("{}:{}", prometheus_interface, 9615),
self.prometheus_port,
)?,
)))
}
}
fn disable_grandpa(&self) -> Result<bool> {
Ok(self.no_grandpa)
}
fn rpc_ws_max_connections(&self) -> Result<Option<usize>> {
Ok(self.ws_max_connections)
}
fn rpc_cors(&self, is_dev: bool) -> Result<Option<Vec<String>>> {
Ok(self
.rpc_cors
.clone()
.unwrap_or_else(|| {
if is_dev {
log::warn!("Running in --dev mode, RPC CORS has been disabled.");
Cors::All
} else {
Cors::List(vec![
"http://localhost:*".into(),
"http://127.0.0.1:*".into(),
"https://localhost:*".into(),
"https://127.0.0.1:*".into(),
"https://polkadot.js.org".into(),
])
}
})
.into())
}
fn rpc_http(&self) -> Result<Option<SocketAddr>> {
let rpc_interface: &str =
interface_str(self.rpc_external, self.unsafe_rpc_external, self.validator)?;
Ok(Some(parse_address(
&format!("{}:{}", rpc_interface, 9933),
self.rpc_port,
)?))
}
fn rpc_ws(&self) -> Result<Option<SocketAddr>> {
let ws_interface: &str =
interface_str(self.ws_external, self.unsafe_ws_external, self.validator)?;
Ok(Some(parse_address(
&format!("{}:{}", ws_interface, 9944),
self.ws_port,
)?))
}
fn offchain_worker(&self, role: &Role) -> Result<bool> {
Ok(match (&self.offchain_worker, role) {
(OffchainWorkerEnabled::WhenValidating, Role::Authority { .. }) => true,
(OffchainWorkerEnabled::Always, _) => true,
(OffchainWorkerEnabled::Never, _) => false,
(OffchainWorkerEnabled::WhenValidating, _) => false,
};
config.role = role;
config.disable_grandpa = self.no_grandpa;
let client_id = config.client_id();
let network_path = config
.in_chain_config_dir(crate::commands::DEFAULT_NETWORK_CONFIG_PATH)
.expect("We provided a basepath");
self.network_config.update_config(
&mut config,
network_path,
client_id,
is_dev,
)?;
self.pool_config.update_config(&mut config)?;
config.dev_key_seed = keyring
.map(|a| format!("//{}", a)).or_else(|| {
if is_dev && !is_light {
Some("//Alice".into())
} else {
None
}
});
if config.rpc_http.is_none() || self.rpc_port.is_some() {
let rpc_interface: &str = interface_str(self.rpc_external, self.unsafe_rpc_external, self.validator)?;
config.rpc_http = Some(parse_address(&format!("{}:{}", rpc_interface, 9933), self.rpc_port)?);
}
if config.rpc_ws.is_none() || self.ws_port.is_some() {
let ws_interface: &str = interface_str(self.ws_external, self.unsafe_ws_external, self.validator)?;
config.rpc_ws = Some(parse_address(&format!("{}:{}", ws_interface, 9944), self.ws_port)?);
}
config.rpc_ws_max_connections = self.ws_max_connections;
config.rpc_cors = self.rpc_cors.clone().unwrap_or_else(|| if is_dev {
log::warn!("Running in --dev mode, RPC CORS has been disabled.");
Cors::All
} else {
Cors::List(vec![
"http://localhost:*".into(),
"http://127.0.0.1:*".into(),
"https://localhost:*".into(),
"https://127.0.0.1:*".into(),
"https://polkadot.js.org".into(),
])
}).into();
// Override telemetry
if self.no_telemetry {
config.telemetry_endpoints = None;
} else if !self.telemetry_endpoints.is_empty() {
config.telemetry_endpoints = Some(
TelemetryEndpoints::new(self.telemetry_endpoints.clone()).map_err(|e| e.to_string())?
);
}
// Override prometheus
if self.no_prometheus {
config.prometheus_config = None;
} else if config.prometheus_config.is_none() {
let prometheus_interface: &str = if self.prometheus_external { "0.0.0.0" } else { "127.0.0.1" };
config.prometheus_config = Some(PrometheusConfig::new_with_default_registry(
parse_address(&format!("{}:{}", prometheus_interface, 9615), self.prometheus_port)?,
));
}
config.tracing_targets = self.import_params.tracing_targets.clone().into();
config.tracing_receiver = self.import_params.tracing_receiver.clone().into();
// Imply forced authoring on --dev
config.force_authoring = self.shared_params.dev || self.force_authoring;
config.max_runtime_instances = self.max_runtime_instances.min(256);
Ok(())
})
}
/// Run the command that runs the node.
pub fn run<FNL, FNF, SL, SF>(
self,
config: Configuration,
new_light: FNL,
new_full: FNF,
version: &VersionInfo,
) -> error::Result<()>
where
FNL: FnOnce(Configuration) -> Result<SL, sc_service::error::Error>,
FNF: FnOnce(Configuration) -> Result<SF, sc_service::error::Error>,
SL: AbstractService + Unpin,
SF: AbstractService + Unpin,
{
info!("{}", version.name);
info!("✌️ version {}", config.full_version());
info!("❤️ by {}, {}-{}", version.author, version.copyright_start_year, Local::today().year());
info!("📋 Chain specification: {}", config.expect_chain_spec().name());
info!("🏷 Node name: {}", config.name);
info!("👤 Role: {}", config.display_role());
match config.role {
Role::Light => run_service_until_exit(
config,
new_light,
),
_ => run_service_until_exit(
config,
new_full,
),
}
fn transaction_pool(&self) -> Result<TransactionPoolOptions> {
Ok(self.pool_config.transaction_pool())
}
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
pub fn init(&self, version: &VersionInfo) -> error::Result<()> {
self.shared_params.init(version)
fn max_runtime_instances(&self) -> Result<Option<usize>> {
Ok(self.max_runtime_instances.map(|x| x.min(256)))
}
}
/// Check whether a node name is considered as valid.
pub fn is_node_name_valid(_name: &str) -> Result<(), &str> {
pub fn is_node_name_valid(_name: &str) -> std::result::Result<(), &str> {
let name = _name.to_string();
if name.chars().count() >= NODE_NAME_MAX_LENGTH {
if name.chars().count() >= crate::NODE_NAME_MAX_LENGTH {
return Err("Node name too long");
}
@@ -512,32 +468,10 @@ pub fn is_node_name_valid(_name: &str) -> Result<(), &str> {
Ok(())
}
#[cfg(not(target_os = "unknown"))]
fn input_keystore_password() -> Result<String, String> {
rpassword::read_password_from_tty(Some("Keystore password: "))
.map_err(|e| format!("{:?}", e))
}
fn generate_node_name() -> String {
let result = loop {
let node_name = Generator::with_naming(Name::Numbered).next().unwrap();
let count = node_name.chars().count();
if count < NODE_NAME_MAX_LENGTH {
break node_name
}
};
result
}
fn parse_address(
address: &str,
port: Option<u16>,
) -> Result<SocketAddr, String> {
let mut address: SocketAddr = address.parse().map_err(
|_| format!("Invalid address: {}", address)
)?;
fn parse_address(address: &str, port: Option<u16>) -> std::result::Result<SocketAddr, String> {
let mut address: SocketAddr = address
.parse()
.map_err(|_| format!("Invalid address: {}", address))?;
if let Some(port) = port {
address.set_port(port);
}
@@ -549,16 +483,21 @@ fn interface_str(
is_external: bool,
is_unsafe_external: bool,
is_validator: bool,
) -> Result<&'static str, error::Error> {
) -> Result<&'static str> {
if is_external && is_validator {
return Err(error::Error::Input("--rpc-external and --ws-external options shouldn't be \
return Err(Error::Input(
"--rpc-external and --ws-external options shouldn't be \
used if the node is running as a validator. Use `--unsafe-rpc-external` if you understand \
the risks. See the options description for more information.".to_owned()));
the risks. See the options description for more information."
.to_owned(),
));
}
if is_external || is_unsafe_external {
log::warn!("It isn't safe to expose RPC publicly without a proxy server that filters \
available set of RPC methods.");
log::warn!(
"It isn't safe to expose RPC publicly without a proxy server that filters \
available set of RPC methods."
);
Ok("0.0.0.0")
} else {
@@ -574,8 +513,8 @@ enum TelemetryParsingError {
impl std::error::Error for TelemetryParsingError {}
impl fmt::Display for TelemetryParsingError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
impl std::fmt::Display for TelemetryParsingError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match &*self {
TelemetryParsingError::MissingVerbosity => write!(f, "Verbosity level missing"),
TelemetryParsingError::VerbosityParsingError(e) => write!(f, "{}", e),
@@ -583,13 +522,15 @@ impl fmt::Display for TelemetryParsingError {
}
}
fn parse_telemetry_endpoints(s: &str) -> Result<(String, u8), TelemetryParsingError> {
fn parse_telemetry_endpoints(s: &str) -> std::result::Result<(String, u8), TelemetryParsingError> {
let pos = s.find(' ');
match pos {
None => Err(TelemetryParsingError::MissingVerbosity),
Some(pos_) => {
let url = s[..pos_].to_string();
let verbosity = s[pos_ + 1..].parse().map_err(TelemetryParsingError::VerbosityParsingError)?;
let verbosity = s[pos_ + 1..]
.parse()
.map_err(TelemetryParsingError::VerbosityParsingError)?;
Ok((url, verbosity))
}
}
@@ -617,7 +558,7 @@ impl From<Cors> for Option<Vec<String>> {
}
/// Parse cors origins.
fn parse_cors(s: &str) -> Result<Cors, Box<dyn std::error::Error>> {
fn parse_cors(s: &str) -> std::result::Result<Cors, Box<dyn std::error::Error>> {
let mut is_all = false;
let mut origins = Vec::new();
for part in s.split(',') {
@@ -625,29 +566,21 @@ fn parse_cors(s: &str) -> Result<Cors, Box<dyn std::error::Error>> {
"all" | "*" => {
is_all = true;
break;
},
}
other => origins.push(other.to_owned()),
}
}
Ok(if is_all { Cors::All } else { Cors::List(origins) })
Ok(if is_all {
Cors::All
} else {
Cors::List(origins)
})
}
#[cfg(test)]
mod tests {
use super::*;
use sc_service::{GenericChainSpec, config::DatabaseConfig};
const TEST_VERSION_INFO: &'static VersionInfo = &VersionInfo {
name: "node-test",
version: "0.1.0",
commit: "some_commit",
executable_name: "node-test",
description: "description",
author: "author",
support_url: "http://example.org",
copyright_start_year: 2020,
};
#[test]
fn tests_node_name_good() {
@@ -663,94 +596,4 @@ mod tests {
assert!(is_node_name_valid("www.visit.me").is_err());
assert!(is_node_name_valid("email@domain").is_err());
}
#[test]
fn keystore_path_is_generated_correctly() {
let chain_spec = GenericChainSpec::from_genesis(
"test",
"test-id",
|| (),
Vec::new(),
None,
None,
None,
None::<()>,
);
for keystore_path in vec![None, Some("/keystore/path")] {
let args: Vec<&str> = vec![];
let mut cli = RunCmd::from_iter(args);
cli.keystore_path = keystore_path.clone().map(PathBuf::from);
let mut config = Configuration::default();
config.config_dir = Some(PathBuf::from("/test/path"));
config.chain_spec = Some(Box::new(chain_spec.clone()));
let chain_spec = chain_spec.clone();
cli.update_config(&mut config, move |_| Ok(Box::new(chain_spec)), TEST_VERSION_INFO).unwrap();
let expected_path = match keystore_path {
Some(path) => PathBuf::from(path),
None => PathBuf::from("/test/path/chains/test-id/keystore"),
};
assert_eq!(expected_path, config.keystore.path().unwrap().to_owned());
}
}
#[test]
fn ensure_load_spec_provide_defaults() {
let chain_spec = GenericChainSpec::from_genesis(
"test",
"test-id",
|| (),
vec!["/ip4/127.0.0.1/tcp/30333/p2p/QmdSHZLmwEL5Axz5JvWNE2mmxU7qyd7xHBFpyUfktgAdg7".parse().unwrap()],
Some(TelemetryEndpoints::new(vec![("wss://foo/bar".to_string(), 42)])
.expect("provided url should be valid")),
None,
None,
None::<()>,
);
let args: Vec<&str> = vec![];
let cli = RunCmd::from_iter(args);
let mut config = Configuration::from_version(TEST_VERSION_INFO);
cli.update_config(&mut config, |_| Ok(Box::new(chain_spec)), TEST_VERSION_INFO).unwrap();
assert!(config.chain_spec.is_some());
assert!(!config.network.boot_nodes.is_empty());
assert!(config.telemetry_endpoints.is_some());
}
#[test]
fn ensure_update_config_for_running_node_provides_defaults() {
let chain_spec = GenericChainSpec::from_genesis(
"test",
"test-id",
|| (),
vec![],
None,
None,
None,
None::<()>,
);
let args: Vec<&str> = vec![];
let cli = RunCmd::from_iter(args);
let mut config = Configuration::from_version(TEST_VERSION_INFO);
cli.init(&TEST_VERSION_INFO).unwrap();
cli.update_config(&mut config, |_| Ok(Box::new(chain_spec)), TEST_VERSION_INFO).unwrap();
assert!(config.config_dir.is_some());
assert!(config.database.is_some());
if let Some(DatabaseConfig::Path { ref cache_size, .. }) = config.database {
assert!(cache_size.is_some());
} else {
panic!("invalid config.database variant");
}
assert!(!config.name.is_empty());
assert!(config.network.config_path.is_some());
assert!(!config.network.listen_addresses.is_empty());
}
}
substrate/client/cli/src/config.rs
+462
View File
@@ -0,0 +1,462 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! Configuration trait for a CLI based on substrate
use crate::error::Result;
use crate::{
init_logger, ImportParams, KeystoreParams, NetworkParams, NodeKeyParams,
PruningParams, SharedParams, SubstrateCli,
};
use app_dirs::{AppDataType, AppInfo};
use names::{Generator, Name};
use sc_service::config::{
Configuration, DatabaseConfig, ExecutionStrategies, ExtTransport, KeystoreConfig,
NetworkConfiguration, NodeKeyConfig, PrometheusConfig, PruningMode, Role, TelemetryEndpoints,
TransactionPoolOptions, WasmExecutionMethod,
};
use sc_service::{ChainSpec, TracingReceiver};
use std::future::Future;
use std::net::SocketAddr;
use std::path::PathBuf;
use std::pin::Pin;
use std::sync::Arc;
/// The maximum number of characters for a node name.
pub(crate) const NODE_NAME_MAX_LENGTH: usize = 32;
/// default sub directory to store network config
pub(crate) const DEFAULT_NETWORK_CONFIG_PATH: &'static str = "network";
/// A trait that allows converting an object to a Configuration
pub trait CliConfiguration: Sized {
/// Get the SharedParams for this object
fn shared_params(&self) -> &SharedParams;
/// Get the ImportParams for this object
fn import_params(&self) -> Option<&ImportParams> {
None
}
/// Get the PruningParams for this object
fn pruning_params(&self) -> Option<&PruningParams> {
self.import_params().map(|x| &x.pruning_params)
}
/// Get the KeystoreParams for this object
fn keystore_params(&self) -> Option<&KeystoreParams> {
None
}
/// Get the NetworkParams for this object
fn network_params(&self) -> Option<&NetworkParams> {
None
}
/// Get the NodeKeyParams for this object
fn node_key_params(&self) -> Option<&NodeKeyParams> {
self.network_params()
.map(|x| &x.node_key_params)
}
/// Get the base path of the configuration (if any)
///
/// By default this is retrieved from `SharedParams`.
fn base_path(&self) -> Result<Option<PathBuf>> {
Ok(self.shared_params().base_path())
}
/// Returns `true` if the node is for development or not
///
/// By default this is retrieved from `SharedParams`.
fn is_dev(&self) -> Result<bool> {
Ok(self.shared_params().is_dev())
}
/// Gets the role
///
/// By default this is `Role::Full`.
fn role(&self, _is_dev: bool) -> Result<Role> {
Ok(Role::Full)
}
/// Get the transaction pool options
///
/// By default this is `TransactionPoolOptions::default()`.
fn transaction_pool(&self) -> Result<TransactionPoolOptions> {
Ok(Default::default())
}
/// Get the network configuration
///
/// By default this is retrieved from `NetworkParams` if it is available otherwise it creates
/// a default `NetworkConfiguration` based on `node_name`, `client_id`, `node_key` and
/// `net_config_dir`.
fn network_config(
&self,
chain_spec: &Box<dyn ChainSpec>,
is_dev: bool,
net_config_dir: &PathBuf,
client_id: &str,
node_name: &str,
node_key: NodeKeyConfig,
) -> Result<NetworkConfiguration> {
Ok(if let Some(network_params) = self.network_params() {
network_params.network_config(
chain_spec,
is_dev,
net_config_dir,
client_id,
node_name,
node_key,
)
} else {
NetworkConfiguration::new(
node_name,
client_id,
node_key,
net_config_dir,
)
})
}
/// Get the keystore configuration.
///
/// Bu default this is retrieved from `KeystoreParams` if it is available. Otherwise it uses
/// `KeystoreConfig::InMemory`.
fn keystore_config(&self, base_path: &PathBuf) -> Result<KeystoreConfig> {
self.keystore_params()
.map(|x| x.keystore_config(base_path))
.unwrap_or(Ok(KeystoreConfig::InMemory))
}
/// Get the database cache size.
///
/// By default this is retrieved from `ImportParams` if it is available. Otherwise its `None`.
fn database_cache_size(&self) -> Result<Option<usize>> {
Ok(self.import_params()
.map(|x| x.database_cache_size())
.unwrap_or(Default::default()))
}
/// Get the database configuration.
///
/// By default this is retrieved from `SharedParams`
fn database_config(&self, base_path: &PathBuf, cache_size: usize) -> Result<DatabaseConfig> {
Ok(self.shared_params().database_config(base_path, cache_size))
}
/// Get the state cache size.
///
/// By default this is retrieved from `ImportParams` if it is available. Otherwise its `0`.
fn state_cache_size(&self) -> Result<usize> {
Ok(self.import_params()
.map(|x| x.state_cache_size())
.unwrap_or(Default::default()))
}
/// Get the state cache child ratio (if any).
///
/// By default this is `None`.
fn state_cache_child_ratio(&self) -> Result<Option<usize>> {
Ok(Default::default())
}
/// Get the pruning mode.
///
/// By default this is retrieved from `PruningMode` if it is available. Otherwise its
/// `PruningMode::default()`.
fn pruning(&self, is_dev: bool, role: &Role) -> Result<PruningMode> {
self.pruning_params()
.map(|x| x.pruning(is_dev, role))
.unwrap_or(Ok(Default::default()))
}
/// Get the chain ID (string).
///
/// By default this is retrieved from `SharedParams`.
fn chain_id(&self, is_dev: bool) -> Result<String> {
Ok(self.shared_params().chain_id(is_dev))
}
/// Get the name of the node.
///
/// By default a random name is generated.
fn node_name(&self) -> Result<String> {
Ok(generate_node_name())
}
/// Get the WASM execution method.
///
/// By default this is retrieved from `ImportParams` if it is available. Otherwise its
/// `WasmExecutionMethod::default()`.
fn wasm_method(&self) -> Result<WasmExecutionMethod> {
Ok(self.import_params()
.map(|x| x.wasm_method())
.unwrap_or(Default::default()))
}
/// Get the execution strategies.
///
/// By default this is retrieved from `ImportParams` if it is available. Otherwise its
/// `ExecutionStrategies::default()`.
fn execution_strategies(&self, is_dev: bool) -> Result<ExecutionStrategies> {
Ok(self.import_params()
.map(|x| x.execution_strategies(is_dev))
.unwrap_or(Default::default()))
}
/// Get the RPC HTTP address (`None` if disabled).
///
/// By default this is `None`.
fn rpc_http(&self) -> Result<Option<SocketAddr>> {
Ok(Default::default())
}
/// Get the RPC websocket address (`None` if disabled).
///
/// By default this is `None`.
fn rpc_ws(&self) -> Result<Option<SocketAddr>> {
Ok(Default::default())
}
/// Get the RPC websockets maximum connections (`None` if unlimited).
///
/// By default this is `None`.
fn rpc_ws_max_connections(&self) -> Result<Option<usize>> {
Ok(Default::default())
}
/// Get the RPC cors (`None` if disabled)
///
/// By default this is `None`.
fn rpc_cors(&self, _is_dev: bool) -> Result<Option<Vec<String>>> {
Ok(Some(Vec::new()))
}
/// Get the prometheus configuration (`None` if disabled)
///
/// By default this is `None`.
fn prometheus_config(&self) -> Result<Option<PrometheusConfig>> {
Ok(Default::default())
}
/// Get the telemetry endpoints (if any)
///
/// By default this is retrieved from the chain spec loaded by `load_spec`.
fn telemetry_endpoints(
&self,
chain_spec: &Box<dyn ChainSpec>,
) -> Result<Option<TelemetryEndpoints>> {
Ok(chain_spec.telemetry_endpoints().clone())
}
/// Get the telemetry external transport
///
/// By default this is `None`.
fn telemetry_external_transport(&self) -> Result<Option<ExtTransport>> {
Ok(Default::default())
}
/// Get the default value for heap pages
///
/// By default this is `None`.
fn default_heap_pages(&self) -> Result<Option<u64>> {
Ok(Default::default())
}
/// Returns `Ok(true)` if offchain worker should be used
///
/// By default this is `false`.
fn offchain_worker(&self, _role: &Role) -> Result<bool> {
Ok(Default::default())
}
/// Returns `Ok(true)` if authoring should be forced
///
/// By default this is `false`.
fn force_authoring(&self) -> Result<bool> {
Ok(Default::default())
}
/// Returns `Ok(true)` if grandpa should be disabled
///
/// By default this is `false`.
fn disable_grandpa(&self) -> Result<bool> {
Ok(Default::default())
}
/// Get the development key seed from the current object
///
/// By default this is `None`.
fn dev_key_seed(&self, _is_dev: bool) -> Result<Option<String>> {
Ok(Default::default())
}
/// Get the tracing targets from the current object (if any)
///
/// By default this is retrieved from `ImportParams` if it is available. Otherwise its
/// `None`.
fn tracing_targets(&self) -> Result<Option<String>> {
Ok(self.import_params()
.map(|x| x.tracing_targets())
.unwrap_or(Default::default()))
}
/// Get the TracingReceiver value from the current object
///
/// By default this is retrieved from `ImportParams` if it is available. Otherwise its
/// `TracingReceiver::default()`.
fn tracing_receiver(&self) -> Result<TracingReceiver> {
Ok(self.import_params()
.map(|x| x.tracing_receiver())
.unwrap_or(Default::default()))
}
/// Get the node key from the current object
///
/// By default this is retrieved from `NodeKeyParams` if it is available. Otherwise its
/// `NodeKeyConfig::default()`.
fn node_key(&self, net_config_dir: &PathBuf) -> Result<NodeKeyConfig> {
self.node_key_params()
.map(|x| x.node_key(net_config_dir))
.unwrap_or(Ok(Default::default()))
}
/// Get maximum runtime instances
///
/// By default this is `None`.
fn max_runtime_instances(&self) -> Result<Option<usize>> {
Ok(Default::default())
}
/// Activate or not the automatic announcing of blocks after import
///
/// By default this is `false`.
fn announce_block(&self) -> Result<bool> {
Ok(true)
}
/// Create a Configuration object from the current object
fn create_configuration<C: SubstrateCli>(
&self,
cli: &C,
task_executor: Arc<dyn Fn(Pin<Box<dyn Future<Output = ()> + Send>>) + Send + Sync>,
) -> Result<Configuration> {
let is_dev = self.is_dev()?;
let chain_id = self.chain_id(is_dev)?;
let chain_spec = cli.load_spec(chain_id.as_str())?;
let config_dir = self
.base_path()?
.unwrap_or_else(|| {
app_dirs::get_app_root(
AppDataType::UserData,
&AppInfo {
name: C::executable_name(),
author: C::author(),
},
)
.expect("app directories exist on all supported platforms; qed")
})
.join("chains")
.join(chain_spec.id());
let net_config_dir = config_dir.join(DEFAULT_NETWORK_CONFIG_PATH);
let client_id = C::client_id();
let database_cache_size = self.database_cache_size()?.unwrap_or(128);
let node_key = self.node_key(&net_config_dir)?;
let role = self.role(is_dev)?;
let max_runtime_instances = self.max_runtime_instances()?.unwrap_or(8);
Ok(Configuration {
impl_name: C::impl_name(),
impl_version: C::impl_version(),
task_executor,
transaction_pool: self.transaction_pool()?,
network: self.network_config(
&chain_spec,
is_dev,
&net_config_dir,
client_id.as_str(),
self.node_name()?.as_str(),
node_key,
)?,
keystore: self.keystore_config(&config_dir)?,
database: self.database_config(&config_dir, database_cache_size)?,
state_cache_size: self.state_cache_size()?,
state_cache_child_ratio: self.state_cache_child_ratio()?,
pruning: self.pruning(is_dev, &role)?,
wasm_method: self.wasm_method()?,
execution_strategies: self.execution_strategies(is_dev)?,
rpc_http: self.rpc_http()?,
rpc_ws: self.rpc_ws()?,
rpc_ws_max_connections: self.rpc_ws_max_connections()?,
rpc_cors: self.rpc_cors(is_dev)?,
prometheus_config: self.prometheus_config()?,
telemetry_endpoints: self.telemetry_endpoints(&chain_spec)?,
telemetry_external_transport: self.telemetry_external_transport()?,
default_heap_pages: self.default_heap_pages()?,
offchain_worker: self.offchain_worker(&role)?,
force_authoring: self.force_authoring()?,
disable_grandpa: self.disable_grandpa()?,
dev_key_seed: self.dev_key_seed(is_dev)?,
tracing_targets: self.tracing_targets()?,
tracing_receiver: self.tracing_receiver()?,
chain_spec,
max_runtime_instances,
announce_block: self.announce_block()?,
role,
})
}
/// Get the filters for the logging.
///
/// By default this is retrieved from `SharedParams`.
fn log_filters(&self) -> Result<Option<String>> {
Ok(self.shared_params().log_filters())
}
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
fn init<C: SubstrateCli>(&self) -> Result<()> {
let logger_pattern = self.log_filters()?.unwrap_or_default();
sp_panic_handler::set(C::support_url(), C::impl_version());
fdlimit::raise_fd_limit();
init_logger(logger_pattern.as_str());
Ok(())
}
}
/// Generate a valid random name for the node
pub fn generate_node_name() -> String {
loop {
let node_name = Generator::with_naming(Name::Numbered)
.next()
.expect("RNG is available on all supported platforms; qed");
let count = node_name.chars().count();
if count < NODE_NAME_MAX_LENGTH {
return node_name;
}
};
}
substrate/client/cli/src/lib.rs
+157 -114
View File
@@ -19,137 +19,175 @@
#![warn(missing_docs)]
#![warn(unused_extern_crates)]
mod params;
mod arg_enums;
mod error;
mod runtime;
mod commands;
mod config;
mod error;
mod params;
mod runner;
pub use sc_service::config::VersionInfo;
use std::io::Write;
use regex::Regex;
use structopt::{StructOpt, clap::{self, AppSettings}};
pub use structopt;
pub use params::*;
pub use commands::*;
pub use arg_enums::*;
pub use commands::*;
pub use config::*;
pub use error::*;
use log::info;
use lazy_static::lazy_static;
pub use crate::runtime::{run_until_exit, run_service_until_exit};
use log::info;
pub use params::*;
use regex::Regex;
pub use runner::*;
use sc_service::{ChainSpec, Configuration};
use std::future::Future;
use std::io::Write;
use std::pin::Pin;
use std::sync::Arc;
pub use structopt;
use structopt::{
clap::{self, AppSettings},
StructOpt,
};
/// Helper function used to parse the command line arguments. This is the equivalent of
/// `structopt`'s `from_iter()` except that it takes a `VersionInfo` argument to provide the name of
/// the application, author, "about" and version. It will also set `AppSettings::GlobalVersion`.
/// Substrate client CLI
///
/// To allow running the node without subcommand, tt also sets a few more settings:
/// `AppSettings::ArgsNegateSubcommands` and `AppSettings::SubcommandsNegateReqs`.
/// This trait needs to be defined on the root structopt of the application. It will provide the
/// implementation name, version, executable name, description, author, support_url, copyright start
/// year and most importantly: how to load the chain spec.
///
/// Gets the struct from the command line arguments. Print the
/// error message and quit the program in case of failure.
pub fn from_args<T>(version: &VersionInfo) -> T
where
T: StructOpt + Sized,
{
from_iter::<T, _>(&mut std::env::args_os(), version)
}
/// StructOpt must not be in scope to use from_args (or the similar methods). This trait provides
/// its own implementation that will fill the necessary field based on the trait's functions.
pub trait SubstrateCli: Sized {
/// Implementation name.
fn impl_name() -> &'static str;
/// Helper function used to parse the command line arguments. This is the equivalent of
/// `structopt`'s `from_iter()` except that it takes a `VersionInfo` argument to provide the name of
/// the application, author, "about" and version. It will also set `AppSettings::GlobalVersion`.
///
/// To allow running the node without subcommand, tt also sets a few more settings:
/// `AppSettings::ArgsNegateSubcommands` and `AppSettings::SubcommandsNegateReqs`.
///
/// Gets the struct from any iterator such as a `Vec` of your making.
/// Print the error message and quit the program in case of failure.
pub fn from_iter<T, I>(iter: I, version: &VersionInfo) -> T
where
T: StructOpt + Sized,
I: IntoIterator,
I::Item: Into<std::ffi::OsString> + Clone,
{
let app = T::clap();
/// Implementation version.
///
/// By default this will look like this: 2.0.0-b950f731c-x86_64-linux-gnu where the hash is the
/// short commit hash of the commit of in the Git repository.
fn impl_version() -> &'static str;
let mut full_version = sc_service::config::full_version_from_strs(
version.version,
version.commit
);
full_version.push_str("\n");
/// Executable file name.
fn executable_name() -> &'static str;
let app = app
.name(version.executable_name)
.author(version.author)
.about(version.description)
.version(full_version.as_str())
.settings(&[
AppSettings::GlobalVersion,
AppSettings::ArgsNegateSubcommands,
AppSettings::SubcommandsNegateReqs,
]);
/// Executable file description.
fn description() -> &'static str;
T::from_clap(&app.get_matches_from(iter))
}
/// Executable file author.
fn author() -> &'static str;
/// Helper function used to parse the command line arguments. This is the equivalent of
/// `structopt`'s `from_iter()` except that it takes a `VersionInfo` argument to provide the name of
/// the application, author, "about" and version. It will also set `AppSettings::GlobalVersion`.
///
/// To allow running the node without subcommand, tt also sets a few more settings:
/// `AppSettings::ArgsNegateSubcommands` and `AppSettings::SubcommandsNegateReqs`.
///
/// Gets the struct from any iterator such as a `Vec` of your making.
/// Print the error message and quit the program in case of failure.
///
/// **NOTE:** This method WILL NOT exit when `--help` or `--version` (or short versions) are
/// used. It will return a [`clap::Error`], where the [`kind`] is a
/// [`ErrorKind::HelpDisplayed`] or [`ErrorKind::VersionDisplayed`] respectively. You must call
/// [`Error::exit`] or perform a [`std::process::exit`].
pub fn try_from_iter<T, I>(iter: I, version: &VersionInfo) -> clap::Result<T>
where
T: StructOpt + Sized,
I: IntoIterator,
I::Item: Into<std::ffi::OsString> + Clone,
{
let app = T::clap();
/// Support URL.
fn support_url() -> &'static str;
let mut full_version = sc_service::config::full_version_from_strs(
version.version,
version.commit,
);
full_version.push_str("\n");
/// Copyright starting year (x-current year)
fn copyright_start_year() -> i32;
let app = app
.name(version.executable_name)
.author(version.author)
.about(version.description)
.version(full_version.as_str());
/// Chain spec factory
fn load_spec(&self, id: &str) -> std::result::Result<Box<dyn ChainSpec>, String>;
let matches = app.get_matches_from_safe(iter)?;
/// Helper function used to parse the command line arguments. This is the equivalent of
/// `structopt`'s `from_iter()` except that it takes a `VersionInfo` argument to provide the name of
/// the application, author, "about" and version. It will also set `AppSettings::GlobalVersion`.
///
/// To allow running the node without subcommand, tt also sets a few more settings:
/// `AppSettings::ArgsNegateSubcommands` and `AppSettings::SubcommandsNegateReqs`.
///
/// Gets the struct from the command line arguments. Print the
/// error message and quit the program in case of failure.
fn from_args() -> Self
where
Self: StructOpt + Sized,
{
<Self as SubstrateCli>::from_iter(&mut std::env::args_os())
}
Ok(T::from_clap(&matches))
}
/// Helper function used to parse the command line arguments. This is the equivalent of
/// `structopt`'s `from_iter()` except that it takes a `VersionInfo` argument to provide the name of
/// the application, author, "about" and version. It will also set `AppSettings::GlobalVersion`.
///
/// To allow running the node without subcommand, it also sets a few more settings:
/// `AppSettings::ArgsNegateSubcommands` and `AppSettings::SubcommandsNegateReqs`.
///
/// Gets the struct from any iterator such as a `Vec` of your making.
/// Print the error message and quit the program in case of failure.
fn from_iter<I>(iter: I) -> Self
where
Self: StructOpt + Sized,
I: IntoIterator,
I::Item: Into<std::ffi::OsString> + Clone,
{
let app = <Self as StructOpt>::clap();
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
pub fn init(logger_pattern: &str, version: &VersionInfo) -> error::Result<()> {
let full_version = sc_service::config::full_version_from_strs(
version.version,
version.commit
);
sp_panic_handler::set(version.support_url, &full_version);
let mut full_version = Self::impl_version().to_string();
full_version.push_str("\n");
fdlimit::raise_fd_limit();
init_logger(logger_pattern);
let app = app
.name(Self::executable_name())
.author(Self::author())
.about(Self::description())
.version(full_version.as_str())
.settings(&[
AppSettings::GlobalVersion,
AppSettings::ArgsNegateSubcommands,
AppSettings::SubcommandsNegateReqs,
]);
Ok(())
<Self as StructOpt>::from_clap(&app.get_matches_from(iter))
}
/// Helper function used to parse the command line arguments. This is the equivalent of
/// `structopt`'s `from_iter()` except that it takes a `VersionInfo` argument to provide the name of
/// the application, author, "about" and version. It will also set `AppSettings::GlobalVersion`.
///
/// To allow running the node without subcommand, it also sets a few more settings:
/// `AppSettings::ArgsNegateSubcommands` and `AppSettings::SubcommandsNegateReqs`.
///
/// Gets the struct from any iterator such as a `Vec` of your making.
/// Print the error message and quit the program in case of failure.
///
/// **NOTE:** This method WILL NOT exit when `--help` or `--version` (or short versions) are
/// used. It will return a [`clap::Error`], where the [`kind`] is a
/// [`ErrorKind::HelpDisplayed`] or [`ErrorKind::VersionDisplayed`] respectively. You must call
/// [`Error::exit`] or perform a [`std::process::exit`].
fn try_from_iter<I>(iter: I) -> clap::Result<Self>
where
Self: StructOpt + Sized,
I: IntoIterator,
I::Item: Into<std::ffi::OsString> + Clone,
{
let app = <Self as StructOpt>::clap();
let mut full_version = Self::impl_version().to_string();
full_version.push_str("\n");
let app = app
.name(Self::executable_name())
.author(Self::author())
.about(Self::description())
.version(full_version.as_str());
let matches = app.get_matches_from_safe(iter)?;
Ok(<Self as StructOpt>::from_clap(&matches))
}
/// Returns the client ID: `{impl_name}/v{impl_version}`
fn client_id() -> String {
format!("{}/v{}", Self::impl_name(), Self::impl_version())
}
/// Only create a Configuration for the command provided in argument
fn create_configuration<T: CliConfiguration>(
&self,
command: &T,
task_executor: Arc<dyn Fn(Pin<Box<dyn Future<Output = ()> + Send>>) + Send + Sync>,
) -> error::Result<Configuration> {
command.create_configuration(self, task_executor)
}
/// Create a runner for the command provided in argument. This will create a Configuration and
/// a tokio runtime
fn create_runner<T: CliConfiguration>(&self, command: &T) -> error::Result<Runner<Self>> {
command.init::<Self>()?;
Runner::new(self, command)
}
}
/// Initialize the logger
@@ -177,15 +215,20 @@ pub fn init_logger(pattern: &str) {
builder.format(move |buf, record| {
let now = time::now();
let timestamp =
time::strftime("%Y-%m-%d %H:%M:%S", &now)
.expect("Error formatting log timestamp");
time::strftime("%Y-%m-%d %H:%M:%S", &now).expect("Error formatting log timestamp");
let mut output = if log::max_level() <= log::LevelFilter::Info {
format!("{} {}", Colour::Black.bold().paint(timestamp), record.args())
format!(
"{} {}",
Colour::Black.bold().paint(timestamp),
record.args(),
)
} else {
let name = ::std::thread::current()
.name()
.map_or_else(Default::default, |x| format!("{}", Colour::Blue.bold().paint(x)));
.map_or_else(Default::default, |x| {
format!("{}", Colour::Blue.bold().paint(x))
});
let millis = (now.tm_nsec as f32 / 1000000.0).floor() as usize;
let timestamp = format!("{}.{}", timestamp, millis);
format!(
substrate/client/cli/src/params/import_params.rs
+44 -30
View File
@@ -14,16 +14,16 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use structopt::StructOpt;
use sc_service::{Configuration, config::DatabaseConfig};
use crate::error;
use crate::arg_enums::{
WasmExecutionMethod, TracingReceiver, ExecutionStrategy, DEFAULT_EXECUTION_BLOCK_CONSTRUCTION,
ExecutionStrategy, TracingReceiver, WasmExecutionMethod, DEFAULT_EXECUTION_BLOCK_CONSTRUCTION,
DEFAULT_EXECUTION_IMPORT_BLOCK, DEFAULT_EXECUTION_OFFCHAIN_WORKER, DEFAULT_EXECUTION_OTHER,
DEFAULT_EXECUTION_SYNCING
DEFAULT_EXECUTION_SYNCING,
};
use crate::params::PruningParams;
use crate::Result;
use sc_client_api::execution_extensions::ExecutionStrategies;
use sc_service::{PruningMode, Role};
use structopt::StructOpt;
/// Parameters for block import.
#[derive(Debug, StructOpt, Clone)]
@@ -52,11 +52,11 @@ pub struct ImportParams {
#[allow(missing_docs)]
#[structopt(flatten)]
pub execution_strategies: ExecutionStrategies,
pub execution_strategies: ExecutionStrategiesParams,
/// Limit the memory the database cache can use.
#[structopt(long = "db-cache", value_name = "MiB", default_value = "128")]
pub database_cache_size: u32,
#[structopt(long = "db-cache", value_name = "MiB")]
pub database_cache_size: Option<usize>,
/// Specify the state cache size.
#[structopt(long = "state-cache-size", value_name = "Bytes", default_value = "67108864")]
@@ -78,25 +78,31 @@ pub struct ImportParams {
}
impl ImportParams {
/// Put block import CLI params into `config` object.
pub fn update_config(
/// Receiver to process tracing messages.
pub fn tracing_receiver(&self) -> sc_service::TracingReceiver {
self.tracing_receiver.clone().into()
}
/// Comma separated list of targets for tracing.
pub fn tracing_targets(&self) -> Option<String> {
self.tracing_targets.clone()
}
/// Specify the state cache size.
pub fn state_cache_size(&self) -> usize {
self.state_cache_size
}
/// Get the WASM execution method from the parameters
pub fn wasm_method(&self) -> sc_service::config::WasmExecutionMethod {
self.wasm_method.into()
}
/// Get execution strategies for the parameters
pub fn execution_strategies(
&self,
mut config: &mut Configuration,
role: &sc_service::Role,
is_dev: bool,
) -> error::Result<()> {
use sc_client_api::execution_extensions::ExecutionStrategies;
if let Some(DatabaseConfig::Path { ref mut cache_size, .. }) = config.database {
*cache_size = Some(self.database_cache_size);
}
config.state_cache_size = self.state_cache_size;
self.pruning_params.update_config(&mut config, role, self.unsafe_pruning)?;
config.wasm_method = self.wasm_method.into();
) -> ExecutionStrategies {
let exec = &self.execution_strategies;
let exec_all_or = |strat: ExecutionStrategy, default: ExecutionStrategy| {
exec.execution.unwrap_or(if strat == default && is_dev {
@@ -106,7 +112,7 @@ impl ImportParams {
}).into()
};
config.execution_strategies = ExecutionStrategies {
ExecutionStrategies {
syncing: exec_all_or(exec.execution_syncing, DEFAULT_EXECUTION_SYNCING),
importing: exec_all_or(exec.execution_import_block, DEFAULT_EXECUTION_IMPORT_BLOCK),
block_construction:
@@ -114,15 +120,23 @@ impl ImportParams {
offchain_worker:
exec_all_or(exec.execution_offchain_worker, DEFAULT_EXECUTION_OFFCHAIN_WORKER),
other: exec_all_or(exec.execution_other, DEFAULT_EXECUTION_OTHER),
};
}
}
Ok(())
/// Get the pruning mode from the parameters
pub fn pruning(&self, unsafe_pruning: bool, role: &Role) -> Result<PruningMode> {
self.pruning_params.pruning(unsafe_pruning, role)
}
/// Limit the memory the database cache can use.
pub fn database_cache_size(&self) -> Option<usize> {
self.database_cache_size
}
}
/// Execution strategies parameters.
#[derive(Debug, StructOpt, Clone)]
pub struct ExecutionStrategies {
pub struct ExecutionStrategiesParams {
/// The means of execution used when calling into the runtime while syncing blocks.
#[structopt(
long = "execution-syncing",
substrate/client/cli/src/params/keystore_params.rs
+92
View File
@@ -0,0 +1,92 @@
// Copyright 2018-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::error::Result;
use sc_service::config::KeystoreConfig;
use std::fs;
use std::path::PathBuf;
use structopt::StructOpt;
/// default sub directory for the key store
const DEFAULT_KEYSTORE_CONFIG_PATH: &'static str = "keystore";
/// Parameters of the keystore
#[derive(Debug, StructOpt, Clone)]
pub struct KeystoreParams {
/// Specify custom keystore path.
#[structopt(long = "keystore-path", value_name = "PATH", parse(from_os_str))]
pub keystore_path: Option<PathBuf>,
/// Use interactive shell for entering the password used by the keystore.
#[structopt(
long = "password-interactive",
conflicts_with_all = &[ "password", "password-filename" ]
)]
pub password_interactive: bool,
/// Password used by the keystore.
#[structopt(
long = "password",
conflicts_with_all = &[ "password-interactive", "password-filename" ]
)]
pub password: Option<String>,
/// File that contains the password used by the keystore.
#[structopt(
long = "password-filename",
value_name = "PATH",
parse(from_os_str),
conflicts_with_all = &[ "password-interactive", "password" ]
)]
pub password_filename: Option<PathBuf>,
}
impl KeystoreParams {
/// Get the keystore configuration for the parameters
pub fn keystore_config(&self, base_path: &PathBuf) -> Result<KeystoreConfig> {
let password = if self.password_interactive {
#[cfg(not(target_os = "unknown"))]
{
Some(input_keystore_password()?.into())
}
#[cfg(target_os = "unknown")]
None
} else if let Some(ref file) = self.password_filename {
Some(
fs::read_to_string(file)
.map_err(|e| format!("{}", e))?
.into(),
)
} else if let Some(ref password) = self.password {
Some(password.clone().into())
} else {
None
};
let path = self
.keystore_path
.clone()
.unwrap_or(base_path.join(DEFAULT_KEYSTORE_CONFIG_PATH));
Ok(KeystoreConfig::Path { path, password })
}
}
#[cfg(not(target_os = "unknown"))]
fn input_keystore_password() -> Result<String> {
rpassword::read_password_from_tty(Some("Keystore password: "))
.map_err(|e| format!("{:?}", e).into())
}
substrate/client/cli/src/params/mod.rs
+9 -7
View File
@@ -15,21 +15,23 @@
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
mod import_params;
mod transaction_pool_params;
mod shared_params;
mod keystore_params;
mod network_params;
mod node_key_params;
mod network_configuration_params;
mod pruning_params;
mod shared_params;
mod transaction_pool_params;
use std::str::FromStr;
use std::fmt::Debug;
use std::str::FromStr;
pub use crate::params::import_params::*;
pub use crate::params::transaction_pool_params::*;
pub use crate::params::shared_params::*;
pub use crate::params::keystore_params::*;
pub use crate::params::network_params::*;
pub use crate::params::node_key_params::*;
pub use crate::params::network_configuration_params::*;
pub use crate::params::pruning_params::*;
pub use crate::params::shared_params::*;
pub use crate::params::transaction_pool_params::*;
/// Wrapper type of `String` that holds an unsigned integer of arbitrary size, formatted as a decimal.
#[derive(Debug, Clone)]
substrate/client/cli/src/params/network_configuration_params.rs → substrate/client/cli/src/params/network_params.rs
+53 -61
View File
@@ -14,21 +14,20 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::path::PathBuf;
use crate::params::node_key_params::NodeKeyParams;
use sc_network::{
config::{NetworkConfiguration, NodeKeyConfig, NonReservedPeerMode, TransportConfig},
multiaddr::Protocol,
};
use sc_service::{ChainSpec, config::{Multiaddr, MultiaddrWithPeerId}};
use std::iter;
use std::net::Ipv4Addr;
use std::path::PathBuf;
use structopt::StructOpt;
use sc_network::{
config::{MultiaddrWithPeerId, NonReservedPeerMode, TransportConfig}, multiaddr::Protocol, Multiaddr,
};
use sc_service::Configuration;
use crate::error;
use crate::params::node_key_params::NodeKeyParams;
/// Parameters used to create the network configuration.
#[derive(Debug, StructOpt, Clone)]
pub struct NetworkConfigurationParams {
pub struct NetworkParams {
/// Specify a list of bootnodes.
#[structopt(long = "bootnodes", value_name = "ADDR")]
pub bootnodes: Vec<MultiaddrWithPeerId>,
@@ -44,14 +43,6 @@ pub struct NetworkConfigurationParams {
#[structopt(long = "reserved-only")]
pub reserved_only: bool,
/// Specify a list of sentry node public addresses.
#[structopt(
long = "sentry-nodes",
value_name = "ADDR",
conflicts_with_all = &[ "sentry" ]
)]
pub sentry_nodes: Vec<MultiaddrWithPeerId>,
/// Listen on this multiaddress.
#[structopt(long = "listen-addr", value_name = "LISTEN_ADDR")]
pub listen_addr: Vec<Multiaddr>,
@@ -87,7 +78,11 @@ pub struct NetworkConfigurationParams {
///
/// This allows downloading announced blocks from multiple peers. Decrease to save
/// traffic and risk increased latency.
#[structopt(long = "max-parallel-downloads", value_name = "COUNT", default_value = "5")]
#[structopt(
long = "max-parallel-downloads",
value_name = "COUNT",
default_value = "5"
)]
pub max_parallel_downloads: u32,
#[allow(missing_docs)]
@@ -99,53 +94,50 @@ pub struct NetworkConfigurationParams {
pub use_yamux_flow_control: bool,
}
impl NetworkConfigurationParams {
impl NetworkParams {
/// Fill the given `NetworkConfiguration` by looking at the cli parameters.
pub fn update_config(
pub fn network_config(
&self,
mut config: &mut Configuration,
config_path: PathBuf,
client_id: String,
chain_spec: &Box<dyn ChainSpec>,
is_dev: bool,
) -> error::Result<()> {
config.network.boot_nodes.extend(self.bootnodes.clone());
config.network.config_path = Some(config_path.clone());
config.network.net_config_path = Some(config_path.clone());
net_config_path: &PathBuf,
client_id: &str,
node_name: &str,
node_key: NodeKeyConfig,
) -> NetworkConfiguration {
let port = self.port.unwrap_or(30333);
let mut listen_addresses = vec![iter::once(Protocol::Ip4(Ipv4Addr::new(0, 0, 0, 0)))
.chain(iter::once(Protocol::Tcp(port)))
.collect()];
config.network.reserved_nodes.extend(self.reserved_nodes.clone());
if self.reserved_only {
config.network.non_reserved_mode = NonReservedPeerMode::Deny;
listen_addresses.extend(self.listen_addr.iter().cloned());
let mut boot_nodes = chain_spec.boot_nodes().to_vec();
boot_nodes.extend(self.bootnodes.clone());
NetworkConfiguration {
boot_nodes,
net_config_path: net_config_path.clone(),
reserved_nodes: self.reserved_nodes.clone(),
non_reserved_mode: if self.reserved_only {
NonReservedPeerMode::Deny
} else {
NonReservedPeerMode::Accept
},
listen_addresses,
public_addresses: Vec::new(),
node_key,
node_name: node_name.to_string(),
client_version: client_id.to_string(),
in_peers: self.in_peers,
out_peers: self.out_peers,
transport: TransportConfig::Normal {
enable_mdns: !is_dev && !self.no_mdns,
allow_private_ipv4: !self.no_private_ipv4,
wasm_external_transport: None,
use_yamux_flow_control: self.use_yamux_flow_control,
},
max_parallel_downloads: self.max_parallel_downloads,
}
config.network.listen_addresses.extend(self.listen_addr.iter().cloned());
if config.network.listen_addresses.is_empty() {
let port = match self.port {
Some(port) => port,
None => 30333,
};
config.network.listen_addresses = vec![
iter::once(Protocol::Ip4(Ipv4Addr::new(0, 0, 0, 0)))
.chain(iter::once(Protocol::Tcp(port)))
.collect()
];
}
config.network.client_version = client_id;
self.node_key_params.update_config(&mut config, Some(&config_path))?;
config.network.in_peers = self.in_peers;
config.network.out_peers = self.out_peers;
config.network.transport = TransportConfig::Normal {
enable_mdns: !is_dev && !self.no_mdns,
allow_private_ipv4: !self.no_private_ipv4,
wasm_external_transport: None,
use_yamux_flow_control: self.use_yamux_flow_control,
};
config.network.max_parallel_downloads = self.max_parallel_downloads;
Ok(())
}
}
substrate/client/cli/src/params/node_key_params.rs
+49 -65
View File
@@ -14,14 +14,13 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::{path::PathBuf, str::FromStr};
use structopt::StructOpt;
use sc_service::Configuration;
use sc_network::config::NodeKeyConfig;
use sp_core::H256;
use std::{path::PathBuf, str::FromStr};
use structopt::StructOpt;
use crate::error;
use crate::arg_enums::NodeKeyType;
use crate::error;
/// The file name of the node's Ed25519 secret key inside the chain-specific
/// network config directory, if neither `--node-key` nor `--node-key-file`
@@ -93,31 +92,23 @@ pub struct NodeKeyParams {
impl NodeKeyParams {
/// Create a `NodeKeyConfig` from the given `NodeKeyParams` in the context
/// of an optional network config storage directory.
pub fn update_config<'a>(
&self,
mut config: &'a mut Configuration,
net_config_path: Option<&PathBuf>,
) -> error::Result<&'a NodeKeyConfig> {
config.network.node_key = match self.node_key_type {
pub fn node_key(&self, net_config_dir: &PathBuf) -> error::Result<NodeKeyConfig> {
Ok(match self.node_key_type {
NodeKeyType::Ed25519 => {
let secret = if let Some(node_key) = self.node_key.as_ref() {
parse_ed25519_secret(node_key)?
} else {
let path = self.node_key_file.clone()
.or_else(|| net_config_path.map(|d| d.join(NODE_KEY_ED25519_FILE)));
let path = self
.node_key_file
.clone()
.unwrap_or_else(|| net_config_dir.join(NODE_KEY_ED25519_FILE));
if let Some(path) = path {
sc_network::config::Secret::File(path)
} else {
sc_network::config::Secret::New
}
sc_network::config::Secret::File(path)
};
NodeKeyConfig::Ed25519(secret)
}
};
Ok(&config.network.node_key)
})
}
}
@@ -128,114 +119,107 @@ fn invalid_node_key(e: impl std::fmt::Display) -> error::Error {
/// Parse a Ed25519 secret key from a hex string into a `sc_network::Secret`.
fn parse_ed25519_secret(hex: &str) -> error::Result<sc_network::config::Ed25519Secret> {
H256::from_str(&hex).map_err(invalid_node_key).and_then(|bytes|
sc_network::config::identity::ed25519::SecretKey::from_bytes(bytes)
.map(sc_network::config::Secret::Input)
.map_err(invalid_node_key))
H256::from_str(&hex)
.map_err(invalid_node_key)
.and_then(|bytes| {
sc_network::config::identity::ed25519::SecretKey::from_bytes(bytes)
.map(sc_network::config::Secret::Input)
.map_err(invalid_node_key)
})
}
#[cfg(test)]
mod tests {
use sc_network::config::identity::ed25519;
use super::*;
use sc_network::config::identity::ed25519;
#[test]
fn test_node_key_config_input() {
fn secret_input(net_config_dir: Option<&PathBuf>) -> error::Result<()> {
fn secret_input(net_config_dir: &PathBuf) -> error::Result<()> {
NodeKeyType::variants().iter().try_for_each(|t| {
let mut config = Configuration::default();
let node_key_type = NodeKeyType::from_str(t).unwrap();
let sk = match node_key_type {
NodeKeyType::Ed25519 => ed25519::SecretKey::generate().as_ref().to_vec()
NodeKeyType::Ed25519 => ed25519::SecretKey::generate().as_ref().to_vec(),
};
let params = NodeKeyParams {
node_key_type,
node_key: Some(format!("{:x}", H256::from_slice(sk.as_ref()))),
node_key_file: None
node_key_file: None,
};
params.update_config(&mut config, net_config_dir).and_then(|c| match c {
params.node_key(net_config_dir).and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::Input(ref ski))
if node_key_type == NodeKeyType::Ed25519 &&
&sk[..] == ski.as_ref() => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
if node_key_type == NodeKeyType::Ed25519 && &sk[..] == ski.as_ref() =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(secret_input(None).is_ok());
assert!(secret_input(Some(&PathBuf::from_str("x").unwrap())).is_ok());
assert!(secret_input(&PathBuf::from_str("x").unwrap()).is_ok());
}
#[test]
fn test_node_key_config_file() {
fn secret_file(net_config_dir: Option<&PathBuf>) -> error::Result<()> {
fn secret_file(net_config_dir: &PathBuf) -> error::Result<()> {
NodeKeyType::variants().iter().try_for_each(|t| {
let mut config = Configuration::default();
let node_key_type = NodeKeyType::from_str(t).unwrap();
let tmp = tempfile::Builder::new().prefix("alice").tempdir()?;
let file = tmp.path().join(format!("{}_mysecret", t)).to_path_buf();
let params = NodeKeyParams {
node_key_type,
node_key: None,
node_key_file: Some(file.clone())
node_key_file: Some(file.clone()),
};
params.update_config(&mut config, net_config_dir).and_then(|c| match c {
params.node_key(net_config_dir).and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::File(ref f))
if node_key_type == NodeKeyType::Ed25519 && f == &file => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
if node_key_type == NodeKeyType::Ed25519 && f == &file =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(secret_file(None).is_ok());
assert!(secret_file(Some(&PathBuf::from_str("x").unwrap())).is_ok());
assert!(secret_file(&PathBuf::from_str("x").unwrap()).is_ok());
}
#[test]
fn test_node_key_config_default() {
fn with_def_params<F>(f: F) -> error::Result<()>
where
F: Fn(NodeKeyParams) -> error::Result<()>
F: Fn(NodeKeyParams) -> error::Result<()>,
{
NodeKeyType::variants().iter().try_for_each(|t| {
let node_key_type = NodeKeyType::from_str(t).unwrap();
f(NodeKeyParams {
node_key_type,
node_key: None,
node_key_file: None
node_key_file: None,
})
})
}
fn no_config_dir() -> error::Result<()> {
with_def_params(|params| {
let mut config = Configuration::default();
let typ = params.node_key_type;
params.update_config(&mut config, None)
.and_then(|c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::New)
if typ == NodeKeyType::Ed25519 => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
})
})
}
fn some_config_dir(net_config_dir: &PathBuf) -> error::Result<()> {
with_def_params(|params| {
let mut config = Configuration::default();
let dir = PathBuf::from(net_config_dir.clone());
let typ = params.node_key_type;
params.update_config(&mut config, Some(net_config_dir))
params
.node_key(net_config_dir)
.and_then(move |c| match c {
NodeKeyConfig::Ed25519(sc_network::config::Secret::File(ref f))
if typ == NodeKeyType::Ed25519 &&
f == &dir.join(NODE_KEY_ED25519_FILE) => Ok(()),
_ => Err(error::Error::Input("Unexpected node key config".into()))
})
if typ == NodeKeyType::Ed25519
&& f == &dir.join(NODE_KEY_ED25519_FILE) =>
{
Ok(())
}
_ => Err(error::Error::Input("Unexpected node key config".into())),
})
})
}
assert!(no_config_dir().is_ok());
assert!(some_config_dir(&PathBuf::from_str("x").unwrap()).is_ok());
}
}
substrate/client/cli/src/params/pruning_params.rs
+12 -19
View File
@@ -14,10 +14,9 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use structopt::StructOpt;
use sc_service::{Configuration, PruningMode};
use crate::error;
use sc_service::{PruningMode, Role};
use structopt::StructOpt;
/// Parameters to define the pruning mode
#[derive(Debug, StructOpt, Clone)]
@@ -32,18 +31,13 @@ pub struct PruningParams {
}
impl PruningParams {
/// Put block pruning CLI params into `config` object.
pub fn update_config(
&self,
mut config: &mut Configuration,
role: &sc_service::Role,
unsafe_pruning: bool,
) -> error::Result<()> {
/// Get the pruning value from the parameters
pub fn pruning(&self, unsafe_pruning: bool, role: &Role) -> error::Result<PruningMode> {
// by default we disable pruning if the node is an authority (i.e.
// `ArchiveAll`), otherwise we keep state for the last 256 blocks. if the
// node is an authority and pruning is enabled explicitly, then we error
// unless `unsafe_pruning` is set.
config.pruning = match &self.pruning {
Ok(match &self.pruning {
Some(ref s) if s == "archive" => PruningMode::ArchiveAll,
None if role.is_network_authority() => PruningMode::ArchiveAll,
None => PruningMode::default(),
@@ -51,16 +45,15 @@ impl PruningParams {
if role.is_network_authority() && !unsafe_pruning {
return Err(error::Error::Input(
"Validators should run with state pruning disabled (i.e. archive). \
You can ignore this check with `--unsafe-pruning`.".to_string()
You can ignore this check with `--unsafe-pruning`."
.to_string(),
));
}
PruningMode::keep_blocks(s.parse()
.map_err(|_| error::Error::Input("Invalid pruning mode specified".to_string()))?
)
},
};
Ok(())
PruningMode::keep_blocks(s.parse().map_err(|_| {
error::Error::Input("Invalid pruning mode specified".to_string())
})?)
}
})
}
}
substrate/client/cli/src/params/shared_params.rs
+46 -65
View File
@@ -14,18 +14,12 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sc_service::config::DatabaseConfig;
use std::path::PathBuf;
use structopt::StructOpt;
use app_dirs::{AppInfo, AppDataType};
use sc_service::{
Configuration, config::DatabaseConfig, ChainSpec,
};
use crate::VersionInfo;
use crate::error;
/// default sub directory to store database
const DEFAULT_DB_CONFIG_PATH : &'static str = "db";
const DEFAULT_DB_CONFIG_PATH: &'static str = "db";
/// Shared parameters used by all `CoreParams`.
#[derive(Debug, StructOpt, Clone)]
@@ -39,7 +33,12 @@ pub struct SharedParams {
pub dev: bool,
/// Specify custom base path.
#[structopt(long = "base-path", short = "d", value_name = "PATH", parse(from_os_str))]
#[structopt(
long = "base-path",
short = "d",
value_name = "PATH",
parse(from_os_str)
)]
pub base_path: Option<PathBuf>,
/// Sets a custom logging filter. Syntax is <target>=<level>, e.g. -lsync=debug.
@@ -51,62 +50,44 @@ pub struct SharedParams {
}
impl SharedParams {
/// Load spec to `Configuration` from `SharedParams` and spec factory.
pub fn update_config<'a, F>(
&self,
mut config: &'a mut Configuration,
spec_factory: F,
version: &VersionInfo,
) -> error::Result<&'a dyn ChainSpec> where
F: FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
{
let chain_key = match self.chain {
/// Specify custom base path.
pub fn base_path(&self) -> Option<PathBuf> {
self.base_path.clone()
}
/// Specify the development chain.
pub fn is_dev(&self) -> bool {
self.dev
}
/// Get the chain spec for the parameters provided
pub fn chain_id(&self, is_dev: bool) -> String {
match self.chain {
Some(ref chain) => chain.clone(),
None => if self.dev { "dev".into() } else { "".into() }
};
let spec = spec_factory(&chain_key)?;
config.network.boot_nodes = spec.boot_nodes().to_vec();
config.telemetry_endpoints = spec.telemetry_endpoints().clone();
config.chain_spec = Some(spec);
if config.config_dir.is_none() {
config.config_dir = Some(base_path(self, version));
}
if config.database.is_none() {
config.database = Some(DatabaseConfig::Path {
path: config
.in_chain_config_dir(DEFAULT_DB_CONFIG_PATH)
.expect("We provided a base_path/config_dir."),
cache_size: None,
});
}
Ok(config.expect_chain_spec())
}
/// Initialize substrate. This must be done only once.
///
/// This method:
///
/// 1. Set the panic handler
/// 2. Raise the FD limit
/// 3. Initialize the logger
pub fn init(&self, version: &VersionInfo) -> error::Result<()> {
crate::init(self.log.as_ref().map(|v| v.as_ref()).unwrap_or(""), version)
}
}
fn base_path(cli: &SharedParams, version: &VersionInfo) -> PathBuf {
cli.base_path.clone()
.unwrap_or_else(||
app_dirs::get_app_root(
AppDataType::UserData,
&AppInfo {
name: version.executable_name,
author: version.author
None => {
if is_dev {
"dev".into()
} else {
"".into()
}
).expect("app directories exist on all supported platforms; qed")
)
}
}
}
/// Get the database configuration object for the parameters provided
pub fn database_config(
&self,
base_path: &PathBuf,
cache_size: usize,
) -> DatabaseConfig {
DatabaseConfig::Path {
path: base_path.join(DEFAULT_DB_CONFIG_PATH),
cache_size,
}
}
/// Get the filters for the logging
pub fn log_filters(&self) -> Option<String> {
self.log.clone()
}
}
substrate/client/cli/src/params/transaction_pool_params.rs
+10 -11
View File
@@ -14,9 +14,8 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use sc_service::config::TransactionPoolOptions;
use structopt::StructOpt;
use sc_service::Configuration;
use crate::error;
/// Parameters used to create the pool configuration.
#[derive(Debug, StructOpt, Clone)]
@@ -24,6 +23,7 @@ pub struct TransactionPoolParams {
/// Maximum number of transactions in the transaction pool.
#[structopt(long = "pool-limit", value_name = "COUNT", default_value = "8192")]
pub pool_limit: usize,
/// Maximum number of kilobytes of all transactions stored in the pool.
#[structopt(long = "pool-kbytes", value_name = "COUNT", default_value = "20480")]
pub pool_kbytes: usize,
@@ -31,19 +31,18 @@ pub struct TransactionPoolParams {
impl TransactionPoolParams {
/// Fill the given `PoolConfiguration` by looking at the cli parameters.
pub fn update_config(
&self,
config: &mut Configuration,
) -> error::Result<()> {
pub fn transaction_pool(&self) -> TransactionPoolOptions {
let mut opts = TransactionPoolOptions::default();
// ready queue
config.transaction_pool.ready.count = self.pool_limit;
config.transaction_pool.ready.total_bytes = self.pool_kbytes * 1024;
opts.ready.count = self.pool_limit;
opts.ready.total_bytes = self.pool_kbytes * 1024;
// future queue
let factor = 10;
config.transaction_pool.future.count = self.pool_limit / factor;
config.transaction_pool.future.total_bytes = self.pool_kbytes * 1024 / factor;
opts.future.count = self.pool_limit / factor;
opts.future.total_bytes = self.pool_kbytes * 1024 / factor;
Ok(())
opts
}
}
substrate/client/cli/src/runner.rs
+237
View File
@@ -0,0 +1,237 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::CliConfiguration;
use crate::Result;
use crate::SubstrateCli;
use crate::Subcommand;
use chrono::prelude::*;
use futures::pin_mut;
use futures::select;
use futures::{future, future::FutureExt, Future};
use log::info;
use sc_service::{AbstractService, Configuration, Role, ServiceBuilderCommand};
use sp_runtime::traits::{Block as BlockT, Header as HeaderT};
use sp_utils::metrics::{TOKIO_THREADS_ALIVE, TOKIO_THREADS_TOTAL};
use std::fmt::Debug;
use std::marker::PhantomData;
use std::sync::Arc;
#[cfg(target_family = "unix")]
async fn main<F, E>(func: F) -> std::result::Result<(), Box<dyn std::error::Error>>
where
F: Future<Output = std::result::Result<(), E>> + future::FusedFuture,
E: 'static + std::error::Error,
{
use tokio::signal::unix::{signal, SignalKind};
let mut stream_int = signal(SignalKind::interrupt())?;
let mut stream_term = signal(SignalKind::terminate())?;
let t1 = stream_int.recv().fuse();
let t2 = stream_term.recv().fuse();
let t3 = func;
pin_mut!(t1, t2, t3);
select! {
_ = t1 => {},
_ = t2 => {},
res = t3 => res?,
}
Ok(())
}
#[cfg(not(unix))]
async fn main<F, E>(func: F) -> std::result::Result<(), Box<dyn std::error::Error>>
where
F: Future<Output = std::result::Result<(), E>> + future::FusedFuture,
E: 'static + std::error::Error,
{
use tokio::signal::ctrl_c;
let t1 = ctrl_c().fuse();
let t2 = func;
pin_mut!(t1, t2);
select! {
_ = t1 => {},
res = t2 => res?,
}
Ok(())
}
/// Build a tokio runtime with all features
pub fn build_runtime() -> std::result::Result<tokio::runtime::Runtime, std::io::Error> {
tokio::runtime::Builder::new()
.thread_name("main-tokio-")
.threaded_scheduler()
.on_thread_start(||{
TOKIO_THREADS_ALIVE.inc();
TOKIO_THREADS_TOTAL.inc();
})
.on_thread_stop(||{
TOKIO_THREADS_ALIVE.dec();
})
.enable_all()
.build()
}
fn run_until_exit<FUT, ERR>(mut tokio_runtime: tokio::runtime::Runtime, future: FUT) -> Result<()>
where
FUT: Future<Output = std::result::Result<(), ERR>> + future::Future,
ERR: 'static + std::error::Error,
{
let f = future.fuse();
pin_mut!(f);
tokio_runtime.block_on(main(f)).map_err(|e| e.to_string())?;
Ok(())
}
/// A Substrate CLI runtime that can be used to run a node or a command
pub struct Runner<C: SubstrateCli> {
config: Configuration,
tokio_runtime: tokio::runtime::Runtime,
phantom: PhantomData<C>,
}
impl<C: SubstrateCli> Runner<C> {
/// Create a new runtime with the command provided in argument
pub fn new<T: CliConfiguration>(cli: &C, command: &T) -> Result<Runner<C>> {
let tokio_runtime = build_runtime()?;
let task_executor = {
let runtime_handle = tokio_runtime.handle().clone();
Arc::new(move |fut| {
runtime_handle.spawn(fut);
})
};
Ok(Runner {
config: command.create_configuration(cli, task_executor)?,
tokio_runtime,
phantom: PhantomData,
})
}
/// A helper function that runs an `AbstractService` with tokio and stops if the process receives
/// the signal `SIGTERM` or `SIGINT`.
pub fn run_node<FNL, FNF, SL, SF>(self, new_light: FNL, new_full: FNF) -> Result<()>
where
FNL: FnOnce(Configuration) -> sc_service::error::Result<SL>,
FNF: FnOnce(Configuration) -> sc_service::error::Result<SF>,
SL: AbstractService + Unpin,
SF: AbstractService + Unpin,
{
info!("{}", C::impl_name());
info!("✌️ version {}", C::impl_version());
info!(
"❤️ by {}, {}-{}",
C::author(),
C::copyright_start_year(),
Local::today().year(),
);
info!("📋 Chain specification: {}", self.config.chain_spec.name());
info!("🏷 Node name: {}", self.config.network.node_name);
info!("👤 Role: {}", self.config.display_role());
match self.config.role {
Role::Light => self.run_service_until_exit(new_light),
_ => self.run_service_until_exit(new_full),
}
}
/// A helper function that runs a future with tokio and stops if the process receives the signal
/// `SIGTERM` or `SIGINT`.
pub fn run_subcommand<B, BC, BB>(self, subcommand: &Subcommand, builder: B) -> Result<()>
where
B: FnOnce(Configuration) -> sc_service::error::Result<BC>,
BC: ServiceBuilderCommand<Block = BB> + Unpin,
BB: sp_runtime::traits::Block + Debug,
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: Debug,
<BB as BlockT>::Hash: std::str::FromStr,
{
match subcommand {
Subcommand::BuildSpec(cmd) => cmd.run(self.config),
Subcommand::ExportBlocks(cmd) => {
run_until_exit(self.tokio_runtime, cmd.run(self.config, builder))
}
Subcommand::ImportBlocks(cmd) => {
run_until_exit(self.tokio_runtime, cmd.run(self.config, builder))
}
Subcommand::CheckBlock(cmd) => {
run_until_exit(self.tokio_runtime, cmd.run(self.config, builder))
}
Subcommand::Revert(cmd) => cmd.run(self.config, builder),
Subcommand::PurgeChain(cmd) => cmd.run(self.config),
}
}
fn run_service_until_exit<T, F>(mut self, service_builder: F) -> Result<()>
where
F: FnOnce(Configuration) -> std::result::Result<T, sc_service::error::Error>,
T: AbstractService + Unpin,
{
let service = service_builder(self.config)?;
let informant_future = sc_informant::build(&service, sc_informant::OutputFormat::Coloured);
let _informant_handle = self.tokio_runtime.spawn(informant_future);
// we eagerly drop the service so that the internal exit future is fired,
// but we need to keep holding a reference to the global telemetry guard
// and drop the runtime first.
let _telemetry = service.telemetry();
let f = service.fuse();
pin_mut!(f);
self.tokio_runtime
.block_on(main(f))
.map_err(|e| e.to_string())?;
drop(self.tokio_runtime);
Ok(())
}
/// A helper function that runs a command with the configuration of this node
pub fn sync_run(self, runner: impl FnOnce(Configuration) -> Result<()>) -> Result<()> {
runner(self.config)
}
/// A helper function that runs a future with tokio and stops if the process receives
/// the signal SIGTERM or SIGINT
pub fn async_run<FUT>(self, runner: impl FnOnce(Configuration) -> FUT) -> Result<()>
where
FUT: Future<Output = Result<()>>,
{
run_until_exit(self.tokio_runtime, runner(self.config))
}
/// Get an immutable reference to the node Configuration
pub fn config(&self) -> &Configuration {
&self.config
}
/// Get a mutable reference to the node Configuration
pub fn config_mut(&mut self) -> &Configuration {
&mut self.config
}
}
substrate/client/cli/src/runtime.rs
-149
View File
@@ -1,149 +0,0 @@
// Copyright 2017-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::sync::Arc;
use futures::{Future, future, future::FutureExt};
use futures::select;
use futures::pin_mut;
use sc_service::{AbstractService, Configuration};
use sp_utils::metrics::{TOKIO_THREADS_ALIVE, TOKIO_THREADS_TOTAL};
use crate::error;
#[cfg(target_family = "unix")]
async fn main<F, E>(func: F) -> Result<(), Box<dyn std::error::Error>>
where
F: Future<Output = Result<(), E>> + future::FusedFuture,
E: 'static + std::error::Error,
{
use tokio::signal::unix::{signal, SignalKind};
let mut stream_int = signal(SignalKind::interrupt())?;
let mut stream_term = signal(SignalKind::terminate())?;
let t1 = stream_int.recv().fuse();
let t2 = stream_term.recv().fuse();
let t3 = func;
pin_mut!(t1, t2, t3);
select! {
_ = t1 => {},
_ = t2 => {},
res = t3 => res?,
}
Ok(())
}
#[cfg(not(unix))]
async fn main<F, E>(func: F) -> Result<(), Box<dyn std::error::Error>>
where
F: Future<Output = Result<(), E>> + future::FusedFuture,
E: 'static + std::error::Error,
{
use tokio::signal::ctrl_c;
let t1 = ctrl_c().fuse();
let t2 = func;
pin_mut!(t1, t2);
select! {
_ = t1 => {},
res = t2 => res?,
}
Ok(())
}
fn build_runtime() -> Result<tokio::runtime::Runtime, std::io::Error> {
tokio::runtime::Builder::new()
.thread_name("main-tokio-")
.threaded_scheduler()
.on_thread_start(||{
TOKIO_THREADS_ALIVE.inc();
TOKIO_THREADS_TOTAL.inc();
})
.on_thread_stop(||{
TOKIO_THREADS_ALIVE.dec();
})
.enable_all()
.build()
}
/// A helper function that runs a future with tokio and stops if the process receives the signal
/// SIGTERM or SIGINT
pub fn run_until_exit<FUT, ERR, F>(
mut config: Configuration,
future_builder: F,
) -> error::Result<()>
where
F: FnOnce(Configuration) -> error::Result<FUT>,
FUT: Future<Output = Result<(), ERR>> + future::Future,
ERR: 'static + std::error::Error,
{
let mut runtime = build_runtime()?;
config.task_executor = {
let runtime_handle = runtime.handle().clone();
Some(Arc::new(move |fut| { runtime_handle.spawn(fut); }))
};
let f = future_builder(config)?;
let f = f.fuse();
pin_mut!(f);
runtime.block_on(main(f)).map_err(|e| e.to_string())?;
Ok(())
}
/// A helper function that runs an `AbstractService` with tokio and stops if the process receives
/// the signal SIGTERM or SIGINT
pub fn run_service_until_exit<T, F>(
mut config: Configuration,
service_builder: F,
) -> error::Result<()>
where
F: FnOnce(Configuration) -> Result<T, sc_service::error::Error>,
T: AbstractService + Unpin,
{
let mut runtime = build_runtime()?;
config.task_executor = {
let runtime_handle = runtime.handle().clone();
Some(Arc::new(move |fut| { runtime_handle.spawn(fut); }))
};
let service = service_builder(config)?;
let informant_future = sc_informant::build(&service, sc_informant::OutputFormat::Coloured);
let _informant_handle = runtime.spawn(informant_future);
// we eagerly drop the service so that the internal exit future is fired,
// but we need to keep holding a reference to the global telemetry guard
// and drop the runtime first.
let _telemetry = service.telemetry();
let f = service.fuse();
pin_mut!(f);
runtime.block_on(main(f)).map_err(|e| e.to_string())?;
drop(runtime);
Ok(())
}
substrate/client/db/src/lib.rs
+2 -2
View File
@@ -284,8 +284,8 @@ pub enum DatabaseSettingsSrc {
Path {
/// Path to the database.
path: PathBuf,
/// Cache size in bytes. If `None` default is used.
cache_size: Option<usize>,
/// Cache size in bytes.
cache_size: usize,
},
/// Use a custom already-open database.
substrate/client/db/src/light.rs
+45 -45
View File
@@ -963,7 +963,7 @@ pub(crate) mod tests {
fn run_checks(db: &LightStorage<Block>, max: u64, checks: &[(u64, Option<Vec<AuthorityId>>)]) {
for (at, expected) in checks.iter().take_while(|(at, _)| *at <= max) {
let actual = get_authorities(db.cache(), BlockId::Number(*at));
let actual = authorities(db.cache(), BlockId::Number(*at));
assert_eq!(*expected, actual);
}
}
@@ -978,7 +978,7 @@ pub(crate) mod tests {
map
}
fn get_authorities(cache: &dyn BlockchainCache<Block>, at: BlockId<Block>) -> Option<Vec<AuthorityId>> {
fn authorities(cache: &dyn BlockchainCache<Block>, at: BlockId<Block>) -> Option<Vec<AuthorityId>> {
cache.get_at(&well_known_cache_keys::AUTHORITIES, &at).unwrap_or(None)
.and_then(|(_, _, val)| Decode::decode(&mut &val[..]).ok())
}
@@ -1026,9 +1026,9 @@ pub(crate) mod tests {
// ... -> B2(1) -> B2_1(1) -> B2_2(2)
// => the cache ignores all writes before best finalized block
let hash2_1 = insert_non_best_block(&db, make_authorities(vec![auth1()]), || default_header(&hash2, 3));
assert_eq!(None, get_authorities(db.cache(), BlockId::Hash(hash2_1)));
assert_eq!(None, authorities(db.cache(), BlockId::Hash(hash2_1)));
let hash2_2 = insert_non_best_block(&db, make_authorities(vec![auth1(), auth2()]), || default_header(&hash2_1, 4));
assert_eq!(None, get_authorities(db.cache(), BlockId::Hash(hash2_2)));
assert_eq!(None, authorities(db.cache(), BlockId::Hash(hash2_2)));
}
let (hash7, hash8, hash6_1, hash6_2, hash6_1_1, hash6_1_2) = {
@@ -1040,55 +1040,55 @@ pub(crate) mod tests {
let hash7 = insert_block(&db, make_authorities(vec![auth3()]), || default_header(&hash6, 7));
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
let hash8 = insert_block(&db, make_authorities(vec![auth3()]), || default_header(&hash7, 8));
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
let hash6_1 = insert_block(&db, make_authorities(vec![auth4()]), || default_header(&hash6, 7));
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
let hash6_1_1 = insert_non_best_block(&db, make_authorities(vec![auth5()]), || default_header(&hash6_1, 8));
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
let hash6_1_2 = insert_non_best_block(&db, make_authorities(vec![auth6()]), || default_header(&hash6_1, 8));
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_2)), Some(vec![auth6()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_2)), Some(vec![auth6()]));
let hash6_2 = insert_block(&db, make_authorities(vec![auth4()]), || default_header(&hash6_1, 8));
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_2)), Some(vec![auth6()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_2)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), Some(vec![auth3()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_2)), Some(vec![auth6()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_2)), Some(vec![auth4()]));
(hash7, hash8, hash6_1, hash6_2, hash6_1_1, hash6_1_2)
};
@@ -1097,27 +1097,27 @@ pub(crate) mod tests {
// finalize block hash6_1
db.finalize_header(BlockId::Hash(hash6_1)).unwrap();
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), None);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), None);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_2)), Some(vec![auth6()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_2)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), None);
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), None);
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_1)), Some(vec![auth5()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_2)), Some(vec![auth6()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_2)), Some(vec![auth4()]));
// finalize block hash6_2
db.finalize_header(BlockId::Hash(hash6_2)).unwrap();
assert_eq!(
get_authorities(db.cache(), BlockId::Hash(hash6)),
authorities(db.cache(), BlockId::Hash(hash6)),
Some(vec![auth1(), auth2()]),
);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash7)), None);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash8)), None);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_1)), None);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_1_2)), None);
assert_eq!(get_authorities(db.cache(), BlockId::Hash(hash6_2)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash7)), None);
assert_eq!(authorities(db.cache(), BlockId::Hash(hash8)), None);
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1)), Some(vec![auth4()]));
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_1)), None);
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_1_2)), None);
assert_eq!(authorities(db.cache(), BlockId::Hash(hash6_2)), Some(vec![auth4()]));
}
}
substrate/client/db/src/upgrade.rs
+1 -1
View File
@@ -166,7 +166,7 @@ mod tests {
state_cache_size: 0,
state_cache_child_ratio: None,
pruning: PruningMode::ArchiveAll,
source: DatabaseSettingsSrc::Path { path: db_path.to_owned(), cache_size: None },
source: DatabaseSettingsSrc::Path { path: db_path.to_owned(), cache_size: 128 },
}, DatabaseType::Full).map(|_| ())
}
substrate/client/db/src/utils.rs
+14 -16
View File
@@ -227,24 +227,22 @@ pub fn open_database<Block: BlockT>(
// and now open database assuming that it has the latest version
let mut db_config = DatabaseConfig::with_columns(NUM_COLUMNS);
if let Some(cache_size) = cache_size {
let state_col_budget = (*cache_size as f64 * 0.9) as usize;
let other_col_budget = (cache_size - state_col_budget) / (NUM_COLUMNS as usize - 1);
let mut memory_budget = std::collections::HashMap::new();
for i in 0..NUM_COLUMNS {
if i == crate::columns::STATE {
memory_budget.insert(i, state_col_budget);
} else {
memory_budget.insert(i, other_col_budget);
}
}
db_config.memory_budget = memory_budget;
}
let state_col_budget = (*cache_size as f64 * 0.9) as usize;
let other_col_budget = (cache_size - state_col_budget) / (NUM_COLUMNS as usize - 1);
let mut memory_budget = std::collections::HashMap::new();
let path = path.to_str()
.ok_or_else(|| sp_blockchain::Error::Backend("Invalid database path".into()))?;
for i in 0..NUM_COLUMNS {
if i == crate::columns::STATE {
memory_budget.insert(i, state_col_budget);
} else {
memory_budget.insert(i, other_col_budget);
}
}
db_config.memory_budget = memory_budget;
Arc::new(Database::open(&db_config, &path).map_err(db_err)?)
},
#[cfg(not(any(feature = "kvdb-rocksdb", test)))]
substrate/client/executor/src/wasm_runtime.rs
+6
View File
@@ -40,6 +40,12 @@ pub enum WasmExecutionMethod {
Compiled,
}
impl Default for WasmExecutionMethod {
fn default() -> WasmExecutionMethod {
WasmExecutionMethod::Interpreted
}
}
/// A Wasm runtime object along with its cached runtime version.
struct VersionedRuntime {
/// Runtime code hash.
substrate/client/network/src/config.rs
+35 -17
View File
@@ -307,10 +307,8 @@ impl From<multiaddr::Error> for ParseErr {
/// Network service configuration.
#[derive(Clone, Debug)]
pub struct NetworkConfiguration {
/// Directory path to store general network configuration. None means nothing will be saved.
pub config_path: Option<PathBuf>,
/// Directory path to store network-specific configuration. None means nothing will be saved.
pub net_config_path: Option<PathBuf>,
pub net_config_path: PathBuf,
/// Multiaddresses to listen for incoming connections.
pub listen_addresses: Vec<Multiaddr>,
/// Multiaddresses to advertise. Detected automatically if empty.
@@ -337,21 +335,26 @@ pub struct NetworkConfiguration {
pub max_parallel_downloads: u32,
}
impl Default for NetworkConfiguration {
fn default() -> Self {
impl NetworkConfiguration {
/// Create new default configuration
pub fn new<SN: Into<String>, SV: Into<String>>(
node_name: SN,
client_version: SV,
node_key: NodeKeyConfig,
net_config_path: &PathBuf,
) -> Self {
NetworkConfiguration {
config_path: None,
net_config_path: None,
net_config_path: net_config_path.clone(),
listen_addresses: Vec::new(),
public_addresses: Vec::new(),
boot_nodes: Vec::new(),
node_key: NodeKeyConfig::Ed25519(Secret::New),
node_key,
in_peers: 25,
out_peers: 75,
reserved_nodes: Vec::new(),
non_reserved_mode: NonReservedPeerMode::Accept,
client_version: "unknown".into(),
node_name: "unknown".into(),
client_version: client_version.into(),
node_name: node_name.into(),
transport: TransportConfig::Normal {
enable_mdns: false,
allow_private_ipv4: true,
@@ -364,30 +367,39 @@ impl Default for NetworkConfiguration {
}
impl NetworkConfiguration {
/// Create a new instance of default settings.
pub fn new() -> Self {
Self::default()
}
/// Create new default configuration for localhost-only connection with random port (useful for testing)
pub fn new_local() -> NetworkConfiguration {
let mut config = NetworkConfiguration::new();
let mut config = NetworkConfiguration::new(
"test-node",
"test-client",
Default::default(),
&std::env::current_dir().expect("current directory must exist"),
);
config.listen_addresses = vec![
iter::once(multiaddr::Protocol::Ip4(Ipv4Addr::new(127, 0, 0, 1)))
.chain(iter::once(multiaddr::Protocol::Tcp(0)))
.collect()
];
config
}
/// Create new default configuration for localhost-only connection with random port (useful for testing)
pub fn new_memory() -> NetworkConfiguration {
let mut config = NetworkConfiguration::new();
let mut config = NetworkConfiguration::new(
"test-node",
"test-client",
Default::default(),
&std::env::current_dir().expect("current directory must exist"),
);
config.listen_addresses = vec![
iter::once(multiaddr::Protocol::Ip4(Ipv4Addr::new(127, 0, 0, 1)))
.chain(iter::once(multiaddr::Protocol::Tcp(0)))
.collect()
];
config
}
}
@@ -452,6 +464,12 @@ pub enum NodeKeyConfig {
Ed25519(Secret<ed25519::SecretKey>)
}
impl Default for NodeKeyConfig {
fn default() -> NodeKeyConfig {
NodeKeyConfig::Ed25519(Secret::New)
}
}
/// The options for obtaining a Ed25519 secret key.
pub type Ed25519Secret = Secret<ed25519::SecretKey>;
substrate/client/network/src/service.rs
+1 -4
View File
@@ -56,7 +56,6 @@ use std::{
collections::{HashMap, HashSet},
fs, io,
marker::PhantomData,
path::Path,
pin::Pin,
str,
sync::{atomic::{AtomicBool, AtomicUsize, Ordering}, Arc},
@@ -175,9 +174,7 @@ impl<B: BlockT + 'static, H: ExHashT> NetworkWorker<B, H> {
pub fn new(params: Params<B, H>) -> Result<NetworkWorker<B, H>, Error> {
let (to_worker, from_worker) = tracing_unbounded("mpsc_network_worker");
if let Some(ref path) = params.network_config.net_config_path {
fs::create_dir_all(Path::new(path))?;
}
fs::create_dir_all(&params.network_config.net_config_path)?;
// List of multiaddresses that we know in the network.
let mut known_addresses = Vec::new();
substrate/client/network/test/src/lib.rs
+20 -10
View File
@@ -600,14 +600,19 @@ pub trait TestNetFactory: Sized {
let listen_addr = build_multiaddr![Memory(rand::random::<u64>())];
let mut network_config = NetworkConfiguration::new(
"test-node",
"test-client",
Default::default(),
&std::env::current_dir().expect("current directory must exist"),
);
network_config.transport = TransportConfig::MemoryOnly;
network_config.listen_addresses = vec![listen_addr.clone()];
let network = NetworkWorker::new(sc_network::config::Params {
role: Role::Full,
executor: None,
network_config: NetworkConfiguration {
listen_addresses: vec![listen_addr.clone()],
transport: TransportConfig::MemoryOnly,
..NetworkConfiguration::default()
},
network_config,
chain: client.clone(),
finality_proof_provider: self.make_finality_proof_provider(
PeersClient::Full(client.clone(), backend.clone()),
@@ -671,14 +676,19 @@ pub trait TestNetFactory: Sized {
let listen_addr = build_multiaddr![Memory(rand::random::<u64>())];
let mut network_config = NetworkConfiguration::new(
"test-node",
"test-client",
Default::default(),
&std::env::current_dir().expect("current directory must exist"),
);
network_config.transport = TransportConfig::MemoryOnly;
network_config.listen_addresses = vec![listen_addr.clone()];
let network = NetworkWorker::new(sc_network::config::Params {
role: Role::Light,
executor: None,
network_config: NetworkConfiguration {
listen_addresses: vec![listen_addr.clone()],
transport: TransportConfig::MemoryOnly,
..NetworkConfiguration::default()
},
network_config,
chain: client.clone(),
finality_proof_provider: self.make_finality_proof_provider(
PeersClient::Light(client.clone(), backend.clone())
substrate/client/service/Cargo.toml
-1
View File
@@ -32,7 +32,6 @@ exit-future = "0.2.0"
serde = "1.0.101"
serde_json = "1.0.41"
sysinfo = "0.12.0"
target_info = "0.1.0"
sc-keystore = { version = "2.0.0-alpha.5", path = "../keystore" }
sp-io = { version = "2.0.0-alpha.5", path = "../../primitives/io" }
sp-runtime = { version = "2.0.0-alpha.5", path = "../../primitives/runtime" }
substrate/client/service/src/builder.rs
+13 -15
View File
@@ -168,7 +168,6 @@ fn new_full_parts<TBl, TRtApi, TExecDisp>(
password.clone()
)?,
KeystoreConfig::InMemory => Keystore::new_in_memory(),
KeystoreConfig::None => return Err("No keystore config provided!".into()),
};
let tasks_builder = TaskManagerBuilder::new();
@@ -179,7 +178,7 @@ fn new_full_parts<TBl, TRtApi, TExecDisp>(
config.max_runtime_instances,
);
let chain_spec = config.expect_chain_spec();
let chain_spec = &config.chain_spec;
let fork_blocks = get_extension::<sc_client::ForkBlocks<TBl>>(chain_spec.extensions())
.cloned()
.unwrap_or_default();
@@ -194,11 +193,11 @@ fn new_full_parts<TBl, TRtApi, TExecDisp>(
state_cache_child_ratio:
config.state_cache_child_ratio.map(|v| (v, 100)),
pruning: config.pruning.clone(),
source: match config.expect_database() {
source: match &config.database {
DatabaseConfig::Path { path, cache_size } =>
sc_client_db::DatabaseSettingsSrc::Path {
path: path.clone(),
cache_size: cache_size.clone().map(|u| u as usize),
cache_size: *cache_size,
},
DatabaseConfig::Custom(db) =>
sc_client_db::DatabaseSettingsSrc::Custom(db.clone()),
@@ -213,7 +212,7 @@ fn new_full_parts<TBl, TRtApi, TExecDisp>(
sc_client_db::new_client(
db_config,
executor,
config.expect_chain_spec().as_storage_builder(),
chain_spec.as_storage_builder(),
fork_blocks,
bad_blocks,
extensions,
@@ -289,7 +288,6 @@ impl ServiceBuilder<(), (), (), (), (), (), (), (), (), (), ()> {
password.clone()
)?,
KeystoreConfig::InMemory => Keystore::new_in_memory(),
KeystoreConfig::None => return Err("No keystore config provided!".into()),
};
let executor = NativeExecutor::<TExecDisp>::new(
@@ -304,11 +302,11 @@ impl ServiceBuilder<(), (), (), (), (), (), (), (), (), (), ()> {
state_cache_child_ratio:
config.state_cache_child_ratio.map(|v| (v, 100)),
pruning: config.pruning.clone(),
source: match config.expect_database() {
source: match &config.database {
DatabaseConfig::Path { path, cache_size } =>
sc_client_db::DatabaseSettingsSrc::Path {
path: path.clone(),
cache_size: cache_size.clone().map(|u| u as usize),
cache_size: *cache_size,
},
DatabaseConfig::Custom(db) =>
sc_client_db::DatabaseSettingsSrc::Custom(db.clone()),
@@ -329,7 +327,7 @@ impl ServiceBuilder<(), (), (), (), (), (), (), (), (), (), ()> {
let remote_blockchain = backend.remote_blockchain();
let client = Arc::new(sc_client::light::new_light(
backend.clone(),
config.expect_chain_spec().as_storage_builder(),
config.chain_spec.as_storage_builder(),
executor,
Box::new(tasks_builder.spawn_handle()),
config.prometheus_config.as_ref().map(|config| config.registry.clone()),
@@ -778,9 +776,9 @@ ServiceBuilder<
let import_queue = Box::new(import_queue);
let chain_info = client.chain_info();
let chain_spec = config.expect_chain_spec();
let chain_spec = &config.chain_spec;
let version = config.full_version();
let version = config.impl_version;
info!("📦 Highest known block at #{}", chain_info.best_number);
telemetry!(
SUBSTRATE_INFO;
@@ -958,7 +956,7 @@ ServiceBuilder<
};
let metrics = MetricsService::with_prometheus(
&registry,
&config.name,
&config.network.node_name,
&config.impl_version,
role_bits,
)?;
@@ -1097,10 +1095,10 @@ ServiceBuilder<
let telemetry = config.telemetry_endpoints.clone().map(|endpoints| {
let is_authority = config.role.is_authority();
let network_id = network.local_peer_id().to_base58();
let name = config.name.clone();
let name = config.network.node_name.clone();
let impl_name = config.impl_name.to_owned();
let version = version.clone();
let chain_name = config.expect_chain_spec().name().to_owned();
let chain_name = config.chain_spec.name().to_owned();
let telemetry_connection_sinks_ = telemetry_connection_sinks.clone();
let telemetry = sc_telemetry::init_telemetry(sc_telemetry::TelemetryConfig {
endpoints,
@@ -1152,7 +1150,7 @@ ServiceBuilder<
Ok(Service {
client,
task_manager: tasks_builder.into_task_manager(config.task_executor.ok_or(Error::TaskExecutorRequired)?),
task_manager: tasks_builder.into_task_manager(config.task_executor),
network,
network_status_sinks,
select_chain,
substrate/client/service/src/config.rs
+10 -153
View File
@@ -18,60 +18,35 @@
pub use sc_client::ExecutionStrategies;
pub use sc_client_db::{kvdb::KeyValueDB, PruningMode};
pub use sc_network::{Multiaddr, config::{MultiaddrWithPeerId, ExtTransport, NetworkConfiguration, Role}};
pub use sc_network::Multiaddr;
pub use sc_network::config::{ExtTransport, MultiaddrWithPeerId, NetworkConfiguration, Role, NodeKeyConfig};
pub use sc_executor::WasmExecutionMethod;
use std::{future::Future, path::{PathBuf, Path}, pin::Pin, net::SocketAddr, sync::Arc};
pub use sc_transaction_pool::txpool::Options as TransactionPoolOptions;
use sc_chain_spec::ChainSpec;
use sp_core::crypto::Protected;
use target_info::Target;
use sc_telemetry::TelemetryEndpoints;
pub use sc_telemetry::TelemetryEndpoints;
use prometheus_endpoint::Registry;
/// Executable version. Used to pass version information from the root crate.
#[derive(Clone)]
pub struct VersionInfo {
/// Implementation name.
pub name: &'static str,
/// Implementation version.
pub version: &'static str,
/// SCM Commit hash.
pub commit: &'static str,
/// Executable file name.
pub executable_name: &'static str,
/// Executable file description.
pub description: &'static str,
/// Executable file author.
pub author: &'static str,
/// Support URL.
pub support_url: &'static str,
/// Copyright starting year (x-current year)
pub copyright_start_year: i32,
}
/// Service configuration.
pub struct Configuration {
/// Implementation name
pub impl_name: &'static str,
/// Implementation version
/// Implementation version (see sc-cli to see an example of format)
pub impl_version: &'static str,
/// Git commit if any.
pub impl_commit: &'static str,
/// Node role.
pub role: Role,
/// How to spawn background tasks. Mandatory, otherwise creating a `Service` will error.
pub task_executor: Option<Arc<dyn Fn(Pin<Box<dyn Future<Output = ()> + Send>>) + Send + Sync>>,
pub task_executor: Arc<dyn Fn(Pin<Box<dyn Future<Output = ()> + Send>>) + Send + Sync>,
/// Extrinsic pool configuration.
pub transaction_pool: TransactionPoolOptions,
/// Network configuration.
pub network: NetworkConfiguration,
/// Path to the base configuration directory.
pub config_dir: Option<PathBuf>,
/// Configuration for the keystore.
pub keystore: KeystoreConfig,
/// Configuration for the database.
pub database: Option<DatabaseConfig>,
pub database: DatabaseConfig,
/// Size of internal state cache in Bytes
pub state_cache_size: usize,
/// Size in percent of cache size dedicated to child tries
@@ -79,9 +54,7 @@ pub struct Configuration {
/// Pruning settings.
pub pruning: PruningMode,
/// Chain configuration.
pub chain_spec: Option<Box<dyn ChainSpec>>,
/// Node name.
pub name: String,
pub chain_spec: Box<dyn ChainSpec>,
/// Wasm execution method.
pub wasm_method: WasmExecutionMethod,
/// Execution strategies.
@@ -130,8 +103,6 @@ pub struct Configuration {
/// Configuration of the client keystore.
#[derive(Clone)]
pub enum KeystoreConfig {
/// No config supplied.
None,
/// Keystore at a path on-disk. Recommended for native nodes.
Path {
/// The path of the keystore.
@@ -147,8 +118,8 @@ impl KeystoreConfig {
/// Returns the path for the keystore.
pub fn path(&self) -> Option<&Path> {
match self {
Self::Path { path, .. } => Some(&path),
Self::None | Self::InMemory => None,
Self::Path { path, .. } => Some(path),
Self::InMemory => None,
}
}
}
@@ -161,7 +132,7 @@ pub enum DatabaseConfig {
/// Path to the database.
path: PathBuf,
/// Cache Size for internal database in MiB
cache_size: Option<u32>,
cache_size: usize,
},
/// A custom implementation of an already-open database.
@@ -190,123 +161,9 @@ impl PrometheusConfig {
}
}
impl Default for Configuration {
/// Create a default config
fn default() -> Self {
Configuration {
impl_name: "parity-substrate",
impl_version: "0.0.0",
impl_commit: "",
chain_spec: None,
config_dir: None,
name: Default::default(),
role: Role::Full,
task_executor: None,
transaction_pool: Default::default(),
network: Default::default(),
keystore: KeystoreConfig::None,
database: None,
state_cache_size: Default::default(),
state_cache_child_ratio: Default::default(),
pruning: PruningMode::default(),
wasm_method: WasmExecutionMethod::Interpreted,
execution_strategies: Default::default(),
rpc_http: None,
rpc_ws: None,
rpc_ws_max_connections: None,
rpc_cors: Some(vec![]),
prometheus_config: None,
telemetry_endpoints: None,
telemetry_external_transport: None,
default_heap_pages: None,
offchain_worker: Default::default(),
force_authoring: false,
disable_grandpa: false,
dev_key_seed: None,
tracing_targets: Default::default(),
tracing_receiver: Default::default(),
max_runtime_instances: 8,
announce_block: true,
}
}
}
impl Configuration {
/// Create a default config using `VersionInfo`
pub fn from_version(version: &VersionInfo) -> Self {
let mut config = Configuration::default();
config.impl_name = version.name;
config.impl_version = version.version;
config.impl_commit = version.commit;
config
}
/// Returns full version string of this configuration.
pub fn full_version(&self) -> String {
full_version_from_strs(self.impl_version, self.impl_commit)
}
/// Implementation id and version.
pub fn client_id(&self) -> String {
format!("{}/v{}", self.impl_name, self.full_version())
}
/// Generate a PathBuf to sub in the chain configuration directory
/// if given
pub fn in_chain_config_dir(&self, sub: &str) -> Option<PathBuf> {
self.config_dir.clone().map(|mut path| {
path.push("chains");
path.push(self.expect_chain_spec().id());
path.push(sub);
path
})
}
/// Return a reference to the `ChainSpec` of this `Configuration`.
///
/// ### Panics
///
/// This method panic if the `chain_spec` is `None`
pub fn expect_chain_spec(&self) -> &dyn ChainSpec {
&**self.chain_spec.as_ref().expect("chain_spec must be specified")
}
/// Return a reference to the `DatabaseConfig` of this `Configuration`.
///
/// ### Panics
///
/// This method panic if the `database` is `None`
pub fn expect_database(&self) -> &DatabaseConfig {
self.database.as_ref().expect("database must be specified")
}
/// Returns a string displaying the node role.
pub fn display_role(&self) -> String {
self.role.to_string()
}
/// Use in memory keystore config when it is not required at all.
///
/// This function returns an error if the keystore is already set to something different than
/// `KeystoreConfig::None`.
pub fn use_in_memory_keystore(&mut self) -> Result<(), String> {
match &mut self.keystore {
cfg @ KeystoreConfig::None => { *cfg = KeystoreConfig::InMemory; Ok(()) },
_ => Err("Keystore config specified when it should not be!".into()),
}
}
}
/// Returns platform info
pub fn platform() -> String {
let env = Target::env();
let env_dash = if env.is_empty() { "" } else { "-" };
format!("{}-{}{}{}", Target::arch(), Target::os(), env_dash, env)
}
/// Returns full version string, using supplied version and commit.
pub fn full_version_from_strs(impl_version: &str, impl_commit: &str) -> String {
let commit_dash = if impl_commit.is_empty() { "" } else { "-" };
format!("{}{}{}-{}", impl_version, commit_dash, impl_commit, platform())
}
substrate/client/service/src/lib.rs
+3 -1
View File
@@ -61,7 +61,8 @@ pub use self::builder::{
};
pub use config::{Configuration, Role, PruningMode, DatabaseConfig};
pub use sc_chain_spec::{
ChainSpec, GenericChainSpec, Properties, RuntimeGenesis, Extension as ChainSpecExtension
ChainSpec, GenericChainSpec, Properties, RuntimeGenesis, Extension as ChainSpecExtension,
NoExtension,
};
pub use sp_transaction_pool::{TransactionPool, InPoolTransaction, error::IntoPoolError};
pub use sc_transaction_pool::txpool::Options as TransactionPoolOptions;
@@ -72,6 +73,7 @@ pub use sc_executor::NativeExecutionDispatch;
pub use std::{ops::Deref, result::Result, sync::Arc};
#[doc(hidden)]
pub use sc_network::config::{FinalityProofProvider, OnDemand, BoxFinalityProofRequestBuilder};
pub use sc_tracing::TracingReceiver;
pub use task_manager::{TaskManagerBuilder, SpawnTaskHandle};
use task_manager::TaskManager;
substrate/client/service/test/src/lib.rs
+24 -35
View File
@@ -38,7 +38,7 @@ use sc_service::{
Error,
};
use sc_network::{multiaddr, Multiaddr, NetworkStateInfo};
use sc_network::config::{NetworkConfiguration, TransportConfig, NodeKeyConfig, Secret, NonReservedPeerMode};
use sc_network::config::{NetworkConfiguration, TransportConfig};
use sp_runtime::{generic::BlockId, traits::Block as BlockT};
use sp_transaction_pool::TransactionPool;
@@ -143,57 +143,46 @@ fn node_config<G: RuntimeGenesis + 'static, E: ChainSpecExtension + Clone + 'sta
{
let root = root.path().join(format!("node-{}", index));
let config_path = Some(root.join("network"));
let net_config_path = config_path.clone();
let net_config_path = root.join("network");
let network_config = NetworkConfiguration {
config_path,
net_config_path,
listen_addresses: vec! [
iter::once(multiaddr::Protocol::Ip4(Ipv4Addr::new(127, 0, 0, 1)))
.chain(iter::once(multiaddr::Protocol::Tcp(base_port + index as u16)))
.collect()
],
public_addresses: vec![],
boot_nodes: vec![],
node_key: NodeKeyConfig::Ed25519(Secret::New),
in_peers: 50,
out_peers: 450,
reserved_nodes: vec![],
non_reserved_mode: NonReservedPeerMode::Accept,
client_version: "network/test/0.1".to_owned(),
node_name: "unknown".to_owned(),
transport: TransportConfig::Normal {
enable_mdns: false,
allow_private_ipv4: true,
wasm_external_transport: None,
use_yamux_flow_control: true,
},
max_parallel_downloads: NetworkConfiguration::default().max_parallel_downloads,
let mut network_config = NetworkConfiguration::new(
format!("Node {}", index),
"network/test/0.1",
Default::default(), &net_config_path,
);
network_config.listen_addresses.push(
iter::once(multiaddr::Protocol::Ip4(Ipv4Addr::new(127, 0, 0, 1)))
.chain(iter::once(multiaddr::Protocol::Tcp(base_port + index as u16)))
.collect()
);
network_config.transport = TransportConfig::Normal {
enable_mdns: false,
allow_private_ipv4: true,
wasm_external_transport: None,
use_yamux_flow_control: true,
};
Configuration {
impl_name: "network-test-impl",
impl_version: "0.1",
impl_commit: "",
role,
task_executor: Some(task_executor),
task_executor,
transaction_pool: Default::default(),
network: network_config,
keystore: KeystoreConfig::Path {
path: root.join("key"),
password: None
},
config_dir: Some(root.clone()),
database: Some(DatabaseConfig::Path {
database: DatabaseConfig::Path {
path: root.join("db"),
cache_size: None
}),
cache_size: 128,
},
state_cache_size: 16777216,
state_cache_child_ratio: None,
pruning: Default::default(),
chain_spec: Some(Box::new((*spec).clone())),
name: format!("Node {}", index),
chain_spec: Box::new((*spec).clone()),
wasm_method: sc_service::config::WasmExecutionMethod::Interpreted,
execution_strategies: Default::default(),
rpc_http: None,
substrate/client/src/client.rs
+2 -2
View File
@@ -3024,7 +3024,7 @@ pub(crate) mod tests {
pruning: PruningMode::ArchiveAll,
source: DatabaseSettingsSrc::Path {
path: tmp.path().into(),
cache_size: None,
cache_size: 128,
}
},
u64::max_value(),
@@ -3226,7 +3226,7 @@ pub(crate) mod tests {
pruning: PruningMode::keep_blocks(1),
source: DatabaseSettingsSrc::Path {
path: tmp.path().into(),
cache_size: None,
cache_size: 128,
}
},
u64::max_value(),
substrate/docs/CHANGELOG.md
+5 -1
View File
@@ -6,6 +6,10 @@ The format is based on [Keep a Changelog].
## Unreleased
### Changed
* `sc_rpc::system::SystemInfo.impl_version` now returns the full version (2.0.0-alpha.2-b950f731c-x86_64-linux-gnu) instead of the short version (1.0.0) (#5271)
## 2.0.0-alpha.4 -> 2.0.0-alpha.5
Runtime
@@ -51,4 +55,4 @@ API
* client/authority-discovery: Instrument code with Prometheus (#5195)
* Don't include `:code` by default in storage proofs (#5179)
* client/network-gossip: Merge GossipEngine and GossipEngineInner (#5042)
* Introduce `on_runtime_upgrade` (#5058)
* Introduce `on_runtime_upgrade` (#5058)
substrate/docs/CODEOWNERS
+4
View File
@@ -90,3 +90,7 @@
# Prometheus endpoint
/utils/prometheus/ @mxinden
# CLI API
/client/cli @cecton
/client/cli-derive @cecton
substrate/utils/browser/src/lib.rs
+49 -19
View File
@@ -17,8 +17,10 @@
use futures01::sync::mpsc as mpsc01;
use log::{debug, info};
use std::sync::Arc;
use sc_network::config::TransportConfig;
use sc_service::{
AbstractService, RpcSession, Role, Configuration, config::{DatabaseConfig, KeystoreConfig},
AbstractService, RpcSession, Role, Configuration,
config::{DatabaseConfig, KeystoreConfig, NetworkConfiguration},
GenericChainSpec, RuntimeGenesis
};
use wasm_bindgen::prelude::*;
@@ -43,29 +45,57 @@ where
let name = chain_spec.name().to_string();
let transport = ExtTransport::new(ffi::websocket_transport());
let mut config = Configuration::default();
config.network.boot_nodes = chain_spec.boot_nodes().to_vec();
config.telemetry_endpoints = chain_spec.telemetry_endpoints().clone();
config.chain_spec = Some(Box::new(chain_spec));
config.network.transport = sc_network::config::TransportConfig::Normal {
let mut network = NetworkConfiguration::new(
format!("{} (Browser)", name),
"unknown",
Default::default(),
&std::env::current_dir().expect("current directory must exist"),
);
network.boot_nodes = chain_spec.boot_nodes().to_vec();
network.transport = TransportConfig::Normal {
wasm_external_transport: Some(transport.clone()),
allow_private_ipv4: true,
enable_mdns: false,
use_yamux_flow_control: true,
};
config.task_executor = Some(Arc::new(move |fut| {
wasm_bindgen_futures::spawn_local(fut)
}));
config.telemetry_external_transport = Some(transport);
config.role = Role::Light;
config.name = format!("{} (Browser)", name);
config.database = Some({
info!("Opening Indexed DB database '{}'...", name);
let db = kvdb_web::Database::open(name, 10)
.await?;
DatabaseConfig::Custom(Arc::new(db))
});
config.keystore = KeystoreConfig::InMemory;
let config = Configuration {
network,
telemetry_endpoints: chain_spec.telemetry_endpoints().clone(),
chain_spec: Box::new(chain_spec),
task_executor: Arc::new(move |fut| wasm_bindgen_futures::spawn_local(fut)),
telemetry_external_transport: Some(transport),
role: Role::Light,
database: {
info!("Opening Indexed DB database '{}'...", name);
let db = kvdb_web::Database::open(name, 10).await?;
DatabaseConfig::Custom(Arc::new(db))
},
keystore: KeystoreConfig::InMemory,
default_heap_pages: Default::default(),
dev_key_seed: Default::default(),
disable_grandpa: Default::default(),
execution_strategies: Default::default(),
force_authoring: Default::default(),
impl_name: "parity-substrate",
impl_version: "0.0.0",
offchain_worker: Default::default(),
prometheus_config: Default::default(),
pruning: Default::default(),
rpc_cors: Default::default(),
rpc_http: Default::default(),
rpc_ws: Default::default(),
rpc_ws_max_connections: Default::default(),
state_cache_child_ratio: Default::default(),
state_cache_size: Default::default(),
tracing_receiver: Default::default(),
tracing_targets: Default::default(),
transaction_pool: Default::default(),
wasm_method: Default::default(),
max_runtime_instances: 8,
announce_block: true,
};
Ok(config)
}
substrate/utils/build-script-utils/Cargo.toml
+1
View File
@@ -9,6 +9,7 @@ repository = "https://github.com/paritytech/substrate/"
description = "Crate with utility functions for `build.rs` scripts."
[dependencies]
platforms = "0.2.1"
[package.metadata.docs.rs]
targets = ["x86_64-unknown-linux-gnu"]
substrate/utils/build-script-utils/src/git.rs
+123
View File
@@ -0,0 +1,123 @@
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::{env, fs, fs::File, io, io::Read, path::PathBuf};
/// Make sure the calling `build.rs` script is rerun when `.git/HEAD` changed.
///
/// The file is searched from the `CARGO_MANIFEST_DIR` upwards. If the file can not be found,
/// a warning is generated.
pub fn rerun_if_git_head_changed() {
let mut manifest_dir = PathBuf::from(
env::var("CARGO_MANIFEST_DIR").expect("`CARGO_MANIFEST_DIR` is always set by cargo."),
);
let manifest_dir_copy = manifest_dir.clone();
while manifest_dir.parent().is_some() {
match get_git_paths(&manifest_dir) {
Err(err) => {
eprintln!("cargo:warning=Unable to read the Git repository: {}", err);
return;
}
Ok(None) => {}
Ok(Some(paths)) => {
for p in paths {
println!("cargo:rerun-if-changed={}", p.display());
}
return;
}
}
manifest_dir.pop();
}
println!(
"cargo:warning=Could not find `.git/HEAD` searching from `{}` upwards!",
manifest_dir_copy.display(),
);
}
// Code taken from https://github.com/rustyhorde/vergen/blob/8d522db8c8e16e26c0fc9ea8e6b0247cbf5cca84/src/output/envvar.rs
fn get_git_paths(path: &PathBuf) -> Result<Option<Vec<PathBuf>>, io::Error> {
let git_dir_or_file = path.join(".git");
if let Ok(metadata) = fs::metadata(&git_dir_or_file) {
if metadata.is_dir() {
// Echo the HEAD path
let git_head_path = git_dir_or_file.join("HEAD");
// Determine where HEAD points and echo that path also.
let mut f = File::open(&git_head_path)?;
let mut git_head_contents = String::new();
let _ = f.read_to_string(&mut git_head_contents)?;
let ref_vec: Vec<&str> = git_head_contents.split(": ").collect();
if ref_vec.len() == 2 {
let current_head_file = ref_vec[1];
let git_refs_path = PathBuf::from(".git").join(current_head_file);
Ok(Some(vec![git_head_path, git_refs_path]))
} else {
Err(io::Error::new(
io::ErrorKind::Other,
"You are most likely in a detached HEAD state",
))
}
} else if metadata.is_file() {
// We are in a worktree, so find out where the actual worktrees/<name>/HEAD file is.
let mut git_file = File::open(&git_dir_or_file)?;
let mut git_contents = String::new();
let _ = git_file.read_to_string(&mut git_contents)?;
let dir_vec: Vec<&str> = git_contents.split(": ").collect();
let git_path = dir_vec[1].trim();
// Echo the HEAD psth
let git_head_path = PathBuf::from(git_path).join("HEAD");
// Find out what the full path to the .git dir is.
let mut actual_git_dir = PathBuf::from(git_path);
actual_git_dir.pop();
actual_git_dir.pop();
// Determine where HEAD points and echo that path also.
let mut f = File::open(&git_head_path)?;
let mut git_head_contents = String::new();
let _ = f.read_to_string(&mut git_head_contents)?;
let ref_vec: Vec<&str> = git_head_contents.split(": ").collect();
if ref_vec.len() == 2 {
let current_head_file = ref_vec[1];
let git_refs_path = actual_git_dir.join(current_head_file);
Ok(Some(vec![git_head_path, git_refs_path]))
} else {
Err(io::Error::new(
io::ErrorKind::Other,
"You are most likely in a detached HEAD state",
))
}
} else {
Err(io::Error::new(
io::ErrorKind::Other,
"Invalid .git format (Not a directory or a file)",
))
}
} else {
Ok(None)
}
}
substrate/utils/build-script-utils/src/lib.rs
+4 -25
View File
@@ -16,29 +16,8 @@
//! Crate with utility functions for `build.rs` scripts.
use std::{env, path::PathBuf};
mod version;
mod git;
/// Make sure the calling `build.rs` script is rerun when `.git/HEAD` changed.
///
/// The file is searched from the `CARGO_MANIFEST_DIR` upwards. If the file can not be found,
/// a warning is generated.
pub fn rerun_if_git_head_changed() {
let mut manifest_dir = PathBuf::from(
env::var("CARGO_MANIFEST_DIR").expect("`CARGO_MANIFEST_DIR` is always set by cargo.")
);
let manifest_dir_copy = manifest_dir.clone();
while manifest_dir.parent().is_some() {
if manifest_dir.join(".git/HEAD").exists() {
println!("cargo:rerun-if-changed={}", manifest_dir.join(".git/HEAD").display());
return
}
manifest_dir.pop();
}
println!(
"cargo:warning=Could not find `.git/HEAD` searching from `{}` upwards!",
manifest_dir_copy.display(),
);
}
pub use git::*;
pub use version::*;
substrate/utils/build-script-utils/src/version.rs
+59
View File
@@ -0,0 +1,59 @@
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use platforms::*;
use std::process::Command;
/// Generate the `cargo:` key output
pub fn generate_cargo_keys() {
let output = Command::new("git")
.args(&["rev-parse", "--short", "HEAD"])
.output();
match output {
Ok(o) if o.status.success() => {
let sha = String::from_utf8_lossy(&o.stdout).trim().to_owned();
println!("cargo:rustc-env=SUBSTRATE_CLI_IMPL_VERSION={}", get_version(sha.as_str()))
}
Ok(o) => eprintln!("cargo:warning=Git command failed with status: {}", o.status),
Err(err) => eprintln!("cargo:warning=Failed to execute git command: {}", err),
}
}
fn get_platform() -> String {
let env_dash = if TARGET_ENV.is_some() { "-" } else { "" };
format!(
"{}-{}{}{}",
TARGET_ARCH.as_str(),
TARGET_OS.as_str(),
env_dash,
TARGET_ENV.map(|x| x.as_str()).unwrap_or(""),
)
}
fn get_version(impl_commit: &str) -> String {
let commit_dash = if impl_commit.is_empty() { "" } else { "-" };
format!(
"{}{}{}-{}",
std::env::var("CARGO_PKG_VERSION").unwrap_or_default(),
commit_dash,
impl_commit,
get_platform(),
)
}
substrate/utils/frame/benchmarking-cli/src/command.rs
+141
View File
@@ -0,0 +1,141 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use crate::BenchmarkCmd;
use codec::{Decode, Encode};
use frame_benchmarking::{Analysis, BenchmarkBatch};
use sc_cli::{SharedParams, CliConfiguration, ExecutionStrategy, Result};
use sc_client::StateMachine;
use sc_client_db::BenchmarkingState;
use sc_executor::NativeExecutor;
use sp_externalities::Extensions;
use sc_service::{Configuration, NativeExecutionDispatch};
use sp_runtime::{
traits::{Block as BlockT, Header as HeaderT, NumberFor},
};
use sp_core::{tasks, testing::KeyStore, traits::KeystoreExt};
use std::fmt::Debug;
impl BenchmarkCmd {
/// Runs the command and benchmarks the chain.
pub fn run<BB, ExecDispatch>(&self, config: Configuration) -> Result<()>
where
BB: BlockT + Debug,
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: std::fmt::Debug,
<BB as BlockT>::Hash: std::str::FromStr,
ExecDispatch: NativeExecutionDispatch + 'static,
{
let spec = config.chain_spec;
let wasm_method = self.wasm_method.into();
let strategy = self.execution.unwrap_or(ExecutionStrategy::Native);
let genesis_storage = spec.build_storage()?;
let mut changes = Default::default();
let cache_size = Some(self.database_cache_size as usize);
let state = BenchmarkingState::<BB>::new(genesis_storage, cache_size)?;
let executor = NativeExecutor::<ExecDispatch>::new(
wasm_method,
None, // heap pages
2, // The runtime instances cache size.
);
let mut extensions = Extensions::default();
extensions.register(KeystoreExt(KeyStore::new()));
let result = StateMachine::<_, _, NumberFor<BB>, _>::new(
&state,
None,
&mut changes,
&executor,
"Benchmark_dispatch_benchmark",
&(
&self.pallet,
&self.extrinsic,
self.lowest_range_values.clone(),
self.highest_range_values.clone(),
self.steps.clone(),
self.repeat,
).encode(),
extensions,
&sp_state_machine::backend::BackendRuntimeCode::new(&state).runtime_code()?,
tasks::executor(),
)
.execute(strategy.into())
.map_err(|e| format!("Error executing runtime benchmark: {:?}", e))?;
let results = <std::result::Result<Vec<BenchmarkBatch>, String> as Decode>::decode(&mut &result[..])
.map_err(|e| format!("Failed to decode benchmark results: {:?}", e))?;
match results {
Ok(batches) => for batch in batches.into_iter() {
// Print benchmark metadata
println!(
"Pallet: {:?}, Extrinsic: {:?}, Lowest values: {:?}, Highest values: {:?}, Steps: {:?}, Repeat: {:?}",
String::from_utf8(batch.pallet).expect("Encoded from String; qed"),
String::from_utf8(batch.benchmark).expect("Encoded from String; qed"),
self.lowest_range_values,
self.highest_range_values,
self.steps,
self.repeat,
);
if self.raw_data {
// Print the table header
batch.results[0].0.iter().for_each(|param| print!("{:?},", param.0));
print!("extrinsic_time,storage_root_time\n");
// Print the values
batch.results.iter().for_each(|result| {
let parameters = &result.0;
parameters.iter().for_each(|param| print!("{:?},", param.1));
// Print extrinsic time and storage root time
print!("{:?},{:?}\n", result.1, result.2);
});
println!();
}
// Conduct analysis.
if !self.no_median_slopes {
if let Some(analysis) = Analysis::median_slopes(&batch.results) {
println!("Median Slopes Analysis\n========\n{}", analysis);
}
}
if !self.no_min_squares {
if let Some(analysis) = Analysis::min_squares_iqr(&batch.results) {
println!("Min Squares Analysis\n========\n{}", analysis);
}
}
},
Err(error) => eprintln!("Error: {:?}", error),
}
Ok(())
}
}
impl CliConfiguration for BenchmarkCmd {
fn shared_params(&self) -> &SharedParams {
&self.shared_params
}
fn chain_id(&self, _is_dev: bool) -> Result<String> {
Ok(match self.shared_params.chain {
Some(ref chain) => chain.clone(),
None => "dev".into(),
})
}
}
substrate/utils/frame/benchmarking-cli/src/lib.rs
+3 -139
View File
@@ -14,21 +14,10 @@
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
mod command;
use sc_cli::{ExecutionStrategy, WasmExecutionMethod};
use std::fmt::Debug;
use sp_runtime::{traits::{Block as BlockT, Header as HeaderT, NumberFor}};
use sc_client::StateMachine;
use sc_cli::{ExecutionStrategy, WasmExecutionMethod, VersionInfo};
use sc_client_db::BenchmarkingState;
use sc_service::{Configuration, ChainSpec};
use sc_executor::{NativeExecutor, NativeExecutionDispatch};
use codec::{Encode, Decode};
use frame_benchmarking::{BenchmarkBatch, Analysis};
use sp_core::{
tasks,
traits::KeystoreExt,
testing::KeyStore,
};
use sp_externalities::Extensions;
/// The `benchmark` command used to benchmark FRAME Pallets.
#[derive(Debug, structopt::StructOpt, Clone)]
@@ -96,128 +85,3 @@ pub struct BenchmarkCmd {
#[structopt(long = "db-cache", value_name = "MiB", default_value = "128")]
pub database_cache_size: u32,
}
impl BenchmarkCmd {
/// Initialize
pub fn init(&self, version: &sc_cli::VersionInfo) -> sc_cli::Result<()> {
self.shared_params.init(version)
}
/// Runs the command and benchmarks the chain.
pub fn run<BB, ExecDispatch>(
self,
config: Configuration,
) -> sc_cli::Result<()>
where
BB: BlockT + Debug,
<<<BB as BlockT>::Header as HeaderT>::Number as std::str::FromStr>::Err: std::fmt::Debug,
<BB as BlockT>::Hash: std::str::FromStr,
ExecDispatch: NativeExecutionDispatch + 'static,
{
let spec = config.chain_spec.expect("chain_spec is always Some");
let wasm_method = self.wasm_method.into();
let strategy = self.execution.unwrap_or(ExecutionStrategy::Native);
let genesis_storage = spec.build_storage()?;
let mut changes = Default::default();
let cache_size = Some(self.database_cache_size as usize);
let state = BenchmarkingState::<BB>::new(genesis_storage, cache_size)?;
let executor = NativeExecutor::<ExecDispatch>::new(
wasm_method,
None, // heap pages
2, // The runtime instances cache size.
);
let mut extensions = Extensions::default();
extensions.register(KeystoreExt(KeyStore::new()));
let result = StateMachine::<_, _, NumberFor<BB>, _>::new(
&state,
None,
&mut changes,
&executor,
"Benchmark_dispatch_benchmark",
&(
&self.pallet,
&self.extrinsic,
self.lowest_range_values.clone(),
self.highest_range_values.clone(),
self.steps.clone(),
self.repeat,
).encode(),
extensions,
&sp_state_machine::backend::BackendRuntimeCode::new(&state).runtime_code()?,
tasks::executor(),
)
.execute(strategy.into())
.map_err(|e| format!("Error executing runtime benchmark: {:?}", e))?;
let results = <Result<Vec<BenchmarkBatch>, String> as Decode>::decode(&mut &result[..])
.map_err(|e| format!("Failed to decode benchmark results: {:?}", e))?;
match results {
Ok(batches) => for batch in batches.into_iter() {
// Print benchmark metadata
println!(
"Pallet: {:?}, Extrinsic: {:?}, Lowest values: {:?}, Highest values: {:?}, Steps: {:?}, Repeat: {:?}",
String::from_utf8(batch.pallet).expect("Encoded from String; qed"),
String::from_utf8(batch.benchmark).expect("Encoded from String; qed"),
self.lowest_range_values,
self.highest_range_values,
self.steps,
self.repeat,
);
if self.raw_data {
// Print the table header
batch.results[0].0.iter().for_each(|param| print!("{:?},", param.0));
print!("extrinsic_time,storage_root_time\n");
// Print the values
batch.results.iter().for_each(|result| {
let parameters = &result.0;
parameters.iter().for_each(|param| print!("{:?},", param.1));
// Print extrinsic time and storage root time
print!("{:?},{:?}\n", result.1, result.2);
});
print!("\n");
}
// Conduct analysis.
if !self.no_median_slopes {
if let Some(analysis) = Analysis::median_slopes(&batch.results) {
println!("Median Slopes Analysis\n========\n{}", analysis);
}
}
if !self.no_min_squares {
if let Some(analysis) = Analysis::min_squares_iqr(&batch.results) {
println!("Min Squares Analysis\n========\n{}", analysis);
}
}
},
Err(error) => eprintln!("Error: {:?}", error),
}
Ok(())
}
/// Update and prepare a `Configuration` with command line parameters
pub fn update_config(
&self,
mut config: &mut Configuration,
spec_factory: impl FnOnce(&str) -> Result<Box<dyn ChainSpec>, String>,
_version: &VersionInfo,
) -> sc_cli::Result<()>
{
// Configure chain spec.
let chain_key = self.shared_params.chain.clone().unwrap_or("dev".into());
let spec = spec_factory(&chain_key)?;
config.chain_spec = Some(spec);
// Make sure to configure keystore.
config.use_in_memory_keystore()?;
Ok(())
}
}