Files
pezkuwi-subxt/polkadot/cli/src/command.rs
T
Javier Viola ea5dbd0475 introduce malus + zombienet based integration tests (#4131)
* test/malus: craft the first maliciously disputing actor

* initial draft

* Add Dockerfile and instructions how to use it to build malus image locally

* Forgot one flag for the build cmd

* we are not docker specific, we are happy to use any containerruntime

* shuffle things around

* add initial tera based integration test

* chores

* fixins

* simple setup to start

* other samples (WIP)

* add Docker version with cargo-chef

* update substarte and small change of orders in commands in the container file

* metrics one

* fmt

* minor

* fixin

* fix metric names

* -d

* add open gauge

* fmt

* spellcheck

* fix test

* adjust to changed error messages

* refactor, more malus impls

* more malus changes

* foo

* minor cleanup

* suggest garbage candidate

* chore

* fix suggest garabge malus

* malus: back garbage candidate

* cargo lock

* re-introduce metrics

* chore: cargo fmt

* undoe 1.54.0 output, CI uses 1.53.0 rustc

* update location of js types

* Fix trybuild

* add tag to image name also; this will be replaced in the prod version

* Tests fixed

* add some fix me

* add dockerfile for ci

* Add docker file for malus for ci

* use variables in .toml file

* add chnages for malus test

* some fixes

* some more fixes

* Update .gitlab-ci.yml

* add local build for polkadot and malus

* some fixes

* enable disputes feature in CI

* ok, ok

* rename: MsgFilter -> MessageInterceptor

* remove TODO that would not have worked

* intercept

* refactor

* fix README and containers

* fix

* chore: cargo fmt

* avoid some more malus-$VARIANT references

* fix argument order

* chore: add about

* Update sanity check in relay chain selection

* fix order, add dispute-unavailable-block malus

* fixup: avoid underflow issue

* it's all u32

* fix conditional use

* Revert "it's all u32"

This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e.

* Revert "fixup: avoid underflow issue"

This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47.

* Revert "Update sanity check in relay chain selection"

This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d.

* update the malus bin

* Update node/malus/integrationtests/0003-dispute-unavailable-block.feature

Co-authored-by: Andronik Ordian <write@reusable.software>

* add some FIXME reminders

* update path to index.js

* Update .gitlab-ci.yml

* Update node/malus/integrationtests/0001-dispute-valid-block.toml

* try 1: make malus test run

* chore: cargo fmt

* temporary fix

* use subcommand syntax from latest gurke

* cargo +nightly fmt

* add collator to a a test

* docs: add env vars to README

* update ci to run dispute-valid-block test

* needs the polkadot image

* Fix path for nodejs container

* post merge fix

* download proper dir  containg configs for malus test

* update the malus ci job

* rm a whitespace

* temp build for malus

* use correct build command for temp malus

* remove subcommands for now

* set max validators per core in the default HostConfig

* tabs

* update beefy

* fixup

* fixup II

* make one variant compile

* make other variants compile

* revert changes to chain_spec

* fmt

* build malus image from polkadot-test-malus again

* revert unrelated changes

* try fixing build-malus job

* Revert "remove subcommands for now"

This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e.

* try fixing build-malus job II

* MVP working dispute-ancestor

* renames

* fix PVF execution on malus

* fix test

* fix typo

* fmt

* checkmate

* try something

* make it actually work

* some tweaks to 01 feature test

* fmt

* sleep a bit more

* complete wococoization

* some tweaks to 01 feature test

* typo fix

* use correct metric names

* fix

* ffs

* .

* try some rearrangement

* Attempt to wait till initial node bootstrap in test

* Fix test syntax

* Run malus tests with v2 script

* Proper symlink created

* simnet v14

* add zombienet tests

* add zombie net test - draft

* add more tests to dispute suite

* add within to fix tests

* replace test directory and start test migration

* migrate all the tests

* add timeout to tests

* reduce debug

* make easy to test in dev

* set appropriated debug

* use image from ci

* fix config for test

* set images from ci

* fix config

* add COLIMAGE env

* tweek tests

* reduce debug

* typo

* wip, migrate old test to zombie-net

* adjunt test config for zombie-net

* run mauls 0001 test only

* clean and setup smoke-test in zombie-net

* add extra time to assertinons

* clean code to merge and improve README

* add info to access logs

* improved readme

* merge master and resolve conflicts

* Update zombienet_tests/README.md

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* clean and consolidate zombienet name

* change runner in gitlab

* add comment explain why we use wococo

* change tag for runner

* remove unused tests

* remove dup Dockerfile and update description

* fmt

* fix compilation post-merge

* fmt

* cut me Some slack

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2021-11-20 14:03:28 +00:00

483 lines
15 KiB
Rust

// Copyright 2017-2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
use crate::cli::{Cli, Subcommand};
use futures::future::TryFutureExt;
use log::info;
use sc_cli::{Role, RuntimeVersion, SubstrateCli};
use service::{self, IdentifyVariant};
use sp_core::crypto::Ss58AddressFormatRegistry;
#[derive(thiserror::Error, Debug)]
pub enum Error {
#[error(transparent)]
PolkadotService(#[from] service::Error),
#[error(transparent)]
SubstrateCli(#[from] sc_cli::Error),
#[error(transparent)]
SubstrateService(#[from] sc_service::Error),
#[error("Other: {0}")]
Other(String),
}
impl std::convert::From<String> for Error {
fn from(s: String) -> Self {
Self::Other(s)
}
}
type Result<T> = std::result::Result<T, Error>;
fn get_exec_name() -> Option<String> {
std::env::current_exe()
.ok()
.and_then(|pb| pb.file_name().map(|s| s.to_os_string()))
.and_then(|s| s.into_string().ok())
}
impl SubstrateCli for Cli {
fn impl_name() -> String {
"Parity Polkadot".into()
}
fn impl_version() -> String {
env!("SUBSTRATE_CLI_IMPL_VERSION").into()
}
fn description() -> String {
env!("CARGO_PKG_DESCRIPTION").into()
}
fn author() -> String {
env!("CARGO_PKG_AUTHORS").into()
}
fn support_url() -> String {
"https://github.com/paritytech/polkadot/issues/new".into()
}
fn copyright_start_year() -> i32 {
2017
}
fn executable_name() -> String {
"polkadot".into()
}
fn load_spec(&self, id: &str) -> std::result::Result<Box<dyn sc_service::ChainSpec>, String> {
let id = if id == "" {
let n = get_exec_name().unwrap_or_default();
["polkadot", "kusama", "westend", "rococo"]
.iter()
.cloned()
.find(|&chain| n.starts_with(chain))
.unwrap_or("polkadot")
} else {
id
};
Ok(match id {
"kusama" => Box::new(service::chain_spec::kusama_config()?),
#[cfg(feature = "kusama-native")]
"kusama-dev" => Box::new(service::chain_spec::kusama_development_config()?),
#[cfg(feature = "kusama-native")]
"kusama-local" => Box::new(service::chain_spec::kusama_local_testnet_config()?),
#[cfg(feature = "kusama-native")]
"kusama-staging" => Box::new(service::chain_spec::kusama_staging_testnet_config()?),
#[cfg(not(feature = "kusama-native"))]
name if name.starts_with("kusama-") && !name.ends_with(".json") =>
Err(format!("`{}` only supported with `kusama-native` feature enabled.", name))?,
"polkadot" => Box::new(service::chain_spec::polkadot_config()?),
#[cfg(feature = "polkadot-native")]
"polkadot-dev" | "dev" => Box::new(service::chain_spec::polkadot_development_config()?),
#[cfg(feature = "polkadot-native")]
"polkadot-local" => Box::new(service::chain_spec::polkadot_local_testnet_config()?),
#[cfg(feature = "polkadot-native")]
"polkadot-staging" => Box::new(service::chain_spec::polkadot_staging_testnet_config()?),
"rococo" => Box::new(service::chain_spec::rococo_config()?),
#[cfg(feature = "rococo-native")]
"rococo-dev" => Box::new(service::chain_spec::rococo_development_config()?),
#[cfg(feature = "rococo-native")]
"rococo-local" => Box::new(service::chain_spec::rococo_local_testnet_config()?),
#[cfg(feature = "rococo-native")]
"rococo-staging" => Box::new(service::chain_spec::rococo_staging_testnet_config()?),
#[cfg(not(feature = "rococo-native"))]
name if name.starts_with("rococo-") && !name.ends_with(".json") =>
Err(format!("`{}` only supported with `rococo-native` feature enabled.", name))?,
"westend" => Box::new(service::chain_spec::westend_config()?),
#[cfg(feature = "westend-native")]
"westend-dev" => Box::new(service::chain_spec::westend_development_config()?),
#[cfg(feature = "westend-native")]
"westend-local" => Box::new(service::chain_spec::westend_local_testnet_config()?),
#[cfg(feature = "westend-native")]
"westend-staging" => Box::new(service::chain_spec::westend_staging_testnet_config()?),
#[cfg(not(feature = "westend-native"))]
name if name.starts_with("westend-") && !name.ends_with(".json") =>
Err(format!("`{}` only supported with `westend-native` feature enabled.", name))?,
"wococo" => Box::new(service::chain_spec::wococo_config()?),
#[cfg(feature = "rococo-native")]
"wococo-dev" => Box::new(service::chain_spec::wococo_development_config()?),
#[cfg(feature = "rococo-native")]
"wococo-local" => Box::new(service::chain_spec::wococo_local_testnet_config()?),
#[cfg(not(feature = "rococo-native"))]
name if name.starts_with("wococo-") =>
Err(format!("`{}` only supported with `rococo-native` feature enabled.", name))?,
path => {
let path = std::path::PathBuf::from(path);
let chain_spec = Box::new(service::PolkadotChainSpec::from_json_file(path.clone())?)
as Box<dyn service::ChainSpec>;
// When `force_*` is given or the file name starts with the name of one of the known chains,
// we use the chain spec for the specific chain.
if self.run.force_rococo || chain_spec.is_rococo() || chain_spec.is_wococo() {
Box::new(service::RococoChainSpec::from_json_file(path)?)
} else if self.run.force_kusama || chain_spec.is_kusama() {
Box::new(service::KusamaChainSpec::from_json_file(path)?)
} else if self.run.force_westend || chain_spec.is_westend() {
Box::new(service::WestendChainSpec::from_json_file(path)?)
} else {
chain_spec
}
},
})
}
fn native_runtime_version(spec: &Box<dyn service::ChainSpec>) -> &'static RuntimeVersion {
#[cfg(feature = "kusama-native")]
if spec.is_kusama() {
return &service::kusama_runtime::VERSION
}
#[cfg(feature = "westend-native")]
if spec.is_westend() {
return &service::westend_runtime::VERSION
}
#[cfg(feature = "rococo-native")]
if spec.is_rococo() || spec.is_wococo() {
return &service::rococo_runtime::VERSION
}
#[cfg(not(all(
feature = "rococo-native",
feature = "westend-native",
feature = "kusama-native"
)))]
let _ = spec;
#[cfg(feature = "polkadot-native")]
{
return &service::polkadot_runtime::VERSION
}
#[cfg(not(feature = "polkadot-native"))]
panic!("No runtime feature (polkadot, kusama, westend, rococo) is enabled")
}
}
fn set_default_ss58_version(spec: &Box<dyn service::ChainSpec>) {
let ss58_version = if spec.is_kusama() {
Ss58AddressFormatRegistry::KusamaAccount
} else if spec.is_westend() {
Ss58AddressFormatRegistry::SubstrateAccount
} else {
Ss58AddressFormatRegistry::PolkadotAccount
}
.into();
sp_core::crypto::set_default_ss58_version(ss58_version);
}
const DEV_ONLY_ERROR_PATTERN: &'static str =
"can only use subcommand with --chain [polkadot-dev, kusama-dev, westend-dev, rococo-dev, wococo-dev], got ";
fn ensure_dev(spec: &Box<dyn service::ChainSpec>) -> std::result::Result<(), String> {
if spec.is_dev() {
Ok(())
} else {
Err(format!("{}{}", DEV_ONLY_ERROR_PATTERN, spec.id()))
}
}
/// Launch a node, accepting arguments just like a regular node,
/// accepts an alternative overseer generator, to adjust behavior
/// for integration tests as needed.
#[cfg(feature = "malus")]
pub fn run_node(run: Cli, overseer_gen: impl service::OverseerGen) -> Result<()> {
run_node_inner(run, overseer_gen)
}
fn run_node_inner(cli: Cli, overseer_gen: impl service::OverseerGen) -> Result<()> {
let runner = cli.create_runner(&cli.run.base).map_err(Error::from)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
let grandpa_pause = if cli.run.grandpa_pause.is_empty() {
None
} else {
Some((cli.run.grandpa_pause[0], cli.run.grandpa_pause[1]))
};
if chain_spec.is_kusama() {
info!("----------------------------");
info!("This chain is not in any way");
info!(" endorsed by the ");
info!(" KUSAMA FOUNDATION ");
info!("----------------------------");
}
let jaeger_agent = cli.run.jaeger_agent;
runner.run_node_until_exit(move |config| async move {
let role = config.role.clone();
match role {
Role::Light => Err(Error::Other("Light client not enabled".into())),
_ => service::build_full(
config,
service::IsCollator::No,
grandpa_pause,
cli.run.beefy,
jaeger_agent,
None,
overseer_gen,
)
.map(|full| full.task_manager)
.map_err(Into::into),
}
})
}
/// Parses polkadot specific CLI arguments and run the service.
pub fn run() -> Result<()> {
let cli = Cli::from_args();
match &cli.subcommand {
None => run_node_inner(cli, service::RealOverseerGen),
Some(Subcommand::BuildSpec(cmd)) => {
let runner = cli.create_runner(cmd)?;
Ok(runner.sync_run(|config| cmd.run(config.chain_spec, config.network))?)
},
Some(Subcommand::CheckBlock(cmd)) => {
let runner = cli.create_runner(cmd).map_err(Error::SubstrateCli)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
runner.async_run(|mut config| {
let (client, _, import_queue, task_manager) =
service::new_chain_ops(&mut config, None)?;
Ok((cmd.run(client, import_queue).map_err(Error::SubstrateCli), task_manager))
})
},
Some(Subcommand::ExportBlocks(cmd)) => {
let runner = cli.create_runner(cmd)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
Ok(runner.async_run(|mut config| {
let (client, _, _, task_manager) =
service::new_chain_ops(&mut config, None).map_err(Error::PolkadotService)?;
Ok((cmd.run(client, config.database).map_err(Error::SubstrateCli), task_manager))
})?)
},
Some(Subcommand::ExportState(cmd)) => {
let runner = cli.create_runner(cmd)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
Ok(runner.async_run(|mut config| {
let (client, _, _, task_manager) = service::new_chain_ops(&mut config, None)?;
Ok((cmd.run(client, config.chain_spec).map_err(Error::SubstrateCli), task_manager))
})?)
},
Some(Subcommand::ImportBlocks(cmd)) => {
let runner = cli.create_runner(cmd)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
Ok(runner.async_run(|mut config| {
let (client, _, import_queue, task_manager) =
service::new_chain_ops(&mut config, None)?;
Ok((cmd.run(client, import_queue).map_err(Error::SubstrateCli), task_manager))
})?)
},
Some(Subcommand::PurgeChain(cmd)) => {
let runner = cli.create_runner(cmd)?;
Ok(runner.sync_run(|config| cmd.run(config.database))?)
},
Some(Subcommand::Revert(cmd)) => {
let runner = cli.create_runner(cmd)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
Ok(runner.async_run(|mut config| {
let (client, backend, _, task_manager) = service::new_chain_ops(&mut config, None)?;
Ok((cmd.run(client, backend).map_err(Error::SubstrateCli), task_manager))
})?)
},
Some(Subcommand::PvfPrepareWorker(cmd)) => {
let mut builder = sc_cli::LoggerBuilder::new("");
builder.with_colors(false);
let _ = builder.init();
#[cfg(target_os = "android")]
{
return Err(sc_cli::Error::Input(
"PVF preparation workers are not supported under this platform".into(),
)
.into())
}
#[cfg(not(target_os = "android"))]
{
polkadot_node_core_pvf::prepare_worker_entrypoint(&cmd.socket_path);
Ok(())
}
},
Some(Subcommand::PvfExecuteWorker(cmd)) => {
let mut builder = sc_cli::LoggerBuilder::new("");
builder.with_colors(false);
let _ = builder.init();
#[cfg(target_os = "android")]
{
return Err(sc_cli::Error::Input(
"PVF execution workers are not supported under this platform".into(),
)
.into())
}
#[cfg(not(target_os = "android"))]
{
polkadot_node_core_pvf::execute_worker_entrypoint(&cmd.socket_path);
Ok(())
}
},
Some(Subcommand::Benchmark(cmd)) => {
let runner = cli.create_runner(cmd)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
ensure_dev(chain_spec).map_err(Error::Other)?;
#[cfg(feature = "kusama-native")]
if chain_spec.is_kusama() {
return Ok(runner.sync_run(|config| {
cmd.run::<service::kusama_runtime::Block, service::KusamaExecutorDispatch>(
config,
)
.map_err(|e| Error::SubstrateCli(e))
})?)
}
#[cfg(feature = "westend-native")]
if chain_spec.is_westend() {
return Ok(runner.sync_run(|config| {
cmd.run::<service::westend_runtime::Block, service::WestendExecutorDispatch>(
config,
)
.map_err(|e| Error::SubstrateCli(e))
})?)
}
// else we assume it is polkadot.
#[cfg(feature = "polkadot-native")]
{
return Ok(runner.sync_run(|config| {
cmd.run::<service::polkadot_runtime::Block, service::PolkadotExecutorDispatch>(
config,
)
.map_err(|e| Error::SubstrateCli(e))
})?)
}
#[cfg(not(feature = "polkadot-native"))]
panic!("No runtime feature (polkadot, kusama, westend, rococo) is enabled")
},
Some(Subcommand::Key(cmd)) => Ok(cmd.run(&cli)?),
#[cfg(feature = "try-runtime")]
Some(Subcommand::TryRuntime(cmd)) => {
let runner = cli.create_runner(cmd)?;
let chain_spec = &runner.config().chain_spec;
set_default_ss58_version(chain_spec);
use sc_service::TaskManager;
let registry = &runner.config().prometheus_config.as_ref().map(|cfg| &cfg.registry);
let task_manager = TaskManager::new(runner.config().tokio_handle.clone(), *registry)
.map_err(|e| Error::SubstrateService(sc_service::Error::Prometheus(e)))?;
ensure_dev(chain_spec).map_err(Error::Other)?;
#[cfg(feature = "kusama-native")]
if chain_spec.is_kusama() {
return runner.async_run(|config| {
Ok((
cmd.run::<service::kusama_runtime::Block, service::KusamaExecutorDispatch>(
config,
)
.map_err(Error::SubstrateCli),
task_manager,
))
})
}
#[cfg(feature = "westend-native")]
if chain_spec.is_westend() {
return runner.async_run(|config| {
Ok((
cmd.run::<service::westend_runtime::Block, service::WestendExecutorDispatch>(
config,
)
.map_err(Error::SubstrateCli),
task_manager,
))
})
}
// else we assume it is polkadot.
#[cfg(feature = "polkadot-native")]
{
return runner.async_run(|config| {
Ok((
cmd.run::<service::polkadot_runtime::Block, service::PolkadotExecutorDispatch>(
config,
)
.map_err(Error::SubstrateCli),
task_manager,
))
})
}
#[cfg(not(feature = "polkadot-native"))]
panic!("No runtime feature (polkadot, kusama, westend, rococo) is enabled")
},
#[cfg(not(feature = "try-runtime"))]
Some(Subcommand::TryRuntime) => Err(Error::Other(
"TryRuntime wasn't enabled when building the node. \
You can enable it with `--features try-runtime`."
.into(),
)
.into()),
}?;
Ok(())
}