Zombienet: add new test collator and integration test (#4797)

* WIP: Wasm compilation perf

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Undying collator WIP

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update test with undying

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Correctly compute post hash

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update helper

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* squash bugs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add --pov-size cli parameter

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix strings

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add pov-size param to export genesis state

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix graveyard size

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* docs + fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix PVF bug and switch to u8 graves

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Build/publish undying collator as colander img

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add undying-collator to colander image

* Fix transaction overflow

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* warn fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add parachain id for export genesis cli

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix the build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* scale test up

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* default parachain id

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add PVF complexity param

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Hash on each iteration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update pvf metric histogram buckets

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Ladi attempt to fix tests

* Fix test

* Fix typos

* Fix pvf typo

* CI integration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock missing

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix clap merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add zombienet image back

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Use collator image from env

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet version

* update test to check pvf prep/exec

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* delete file

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use default bootnode in upgrade test

* FIx tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add some stress -  pvf exec times up to 2s

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix name

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Dial down pvf execution time < 2s

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 1100

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* try again

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add connectivity check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes and refactor folder struct

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* change toml name to match

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* impl Feedback for Review 😎

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spell check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Zombienet: add disputes test (#4859)

* Zombienet disputes test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* CI: add zombienet-parachain-disputes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Bump zombienet

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* reduce duration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update zombienet_tests/functional/0002-parachains-disputes.feature

fix test

* Update zombienet_tests/functional/0002-parachains-disputes.feature

fix fix

* more logs and set collator image

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix cargo lock damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* merge fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet image

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* debugging CI run - scale down test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix cargo merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* default command fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Revert "debugging CI run - scale down test"

This reverts commit eec2ba7e65ede3f929e2f0a8fe44930df2119450.

* more review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
This commit is contained in:
Andrei Sandu
2022-02-28 12:27:01 +02:00
committed by GitHub
parent 00ce69aae8
commit 0fa91e7043
23 changed files with 1588 additions and 8 deletions
+69 -7
View File
@@ -27,7 +27,7 @@ variables:
CI_IMAGE: "paritytech/ci-linux:production"
DOCKER_OS: "debian:stretch"
ARCH: "x86_64"
ZOMBIENET_IMAGE: "docker.io/paritytech/zombienet:v1.2.3"
ZOMBIENET_IMAGE: "docker.io/paritytech/zombienet:v1.2.14"
VAULT_SERVER_URL: "https://vault.parity-mgmt-vault.parity.io"
VAULT_AUTH_PATH: "gitlab-parity-io-jwt"
VAULT_AUTH_ROLE: "cicd_gitlab_parity_${CI_PROJECT_NAME}"
@@ -258,7 +258,7 @@ spellcheck:
$(git diff --diff-filter=AM --name-only $(git merge-base ${CI_COMMIT_SHA} ${CI_DEFAULT_BRANCH} -- :^bridges))
allow_failure: true
build-adder-collator:
build-test-collators:
stage: test
<<: *collect-artifacts
<<: *docker-env
@@ -266,13 +266,16 @@ build-adder-collator:
<<: *test-refs
script:
- time cargo build --profile testnet --verbose -p test-parachain-adder-collator
- time cargo build --profile testnet --verbose -p test-parachain-undying-collator
- sccache -s
# pack artifacts
- mkdir -p ./artifacts
- mv ./target/testnet/adder-collator ./artifacts/.
- mv ./target/testnet/undying-collator ./artifacts/.
- echo -n "${CI_COMMIT_REF_NAME}" > ./artifacts/VERSION
- echo -n "${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}" > ./artifacts/EXTRATAG
- echo "adder-collator version = $(cat ./artifacts/VERSION) (EXTRATAG = $(cat ./artifacts/EXTRATAG))"
- echo "undying-collator version = $(cat ./artifacts/VERSION) (EXTRATAG = $(cat ./artifacts/EXTRATAG))"
- cp -r scripts/* ./artifacts
build-malus:
@@ -432,7 +435,7 @@ publish-polkadot-image:
dotenv: ./artifacts/parachains.env
expire_in: 1 days
publish-adder-collator-image:
publish-test-collators-image:
# service image for Simnet
stage: build
<<: *build-push-image
@@ -443,7 +446,7 @@ publish-adder-collator-image:
DOCKERFILE: dockerfiles/collator_injected.Dockerfile
IMAGE_NAME: docker.io/paritypr/colander
needs:
- job: build-adder-collator
- job: build-test-collators
artifacts: true
after_script:
- buildah logout --all
@@ -614,9 +617,9 @@ zombienet-tests-parachains-smoke-test:
needs:
- job: publish-polkadot-image
- job: publish-malus-image
- job: publish-adder-collator-image
- job: publish-test-collators-image
variables:
GH_DIR: "https://github.com/paritytech/polkadot/tree/${CI_COMMIT_SHORT_SHA}/zombienet_tests/parachains"
GH_DIR: "https://github.com/paritytech/polkadot/tree/${CI_COMMIT_SHORT_SHA}/zombienet_tests/smoke"
before_script:
- echo "Zombie-net Tests Config"
- echo "${ZOMBIENET_IMAGE}"
@@ -636,6 +639,65 @@ zombienet-tests-parachains-smoke-test:
tags:
- zombienet-polkadot-integration-test
zombienet-tests-parachains-pvf:
stage: deploy
image: "${ZOMBIENET_IMAGE}"
<<: *kubernetes-env
<<: *zombienet-refs
needs:
- job: publish-polkadot-image
- job: publish-test-collators-image
variables:
GH_DIR: "https://github.com/paritytech/polkadot/tree/${CI_COMMIT_SHORT_SHA}/zombienet_tests/functional"
before_script:
- echo "Zombie-net Tests Config"
- echo "${ZOMBIENET_IMAGE}"
- echo "${PARACHAINS_IMAGE_NAME} ${PARACHAINS_IMAGE_TAG}"
- echo "COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}"
- echo "${GH_DIR}"
- export DEBUG=zombie,zombie::network-node
- export ZOMBIENET_INTEGRATION_TEST_IMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}
- export MALUS_IMAGE=${MALUS_IMAGE_NAME}:${MALUS_IMAGE_TAG}
- export COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}
script:
- /home/nonroot/zombie-net/scripts/run-test-env-manager.sh
--github-remote-dir="${GH_DIR}"
--test="0001-parachains-pvf.feature"
allow_failure: true
retry: 2
tags:
- zombienet-polkadot-integration-test
zombienet-tests-parachains-disputes:
stage: deploy
image: "${ZOMBIENET_IMAGE}"
<<: *kubernetes-env
<<: *zombienet-refs
needs:
- job: publish-polkadot-image
- job: publish-test-collators-image
- job: publish-malus-image
variables:
GH_DIR: "https://github.com/paritytech/polkadot/tree/${CI_COMMIT_SHORT_SHA}/zombienet_tests/functional"
before_script:
- echo "Zombie-net Tests Config"
- echo "${ZOMBIENET_IMAGE_NAME}"
- echo "${PARACHAINS_IMAGE_NAME} ${PARACHAINS_IMAGE_TAG}"
- echo "${MALUS_IMAGE_NAME} ${MALUS_IMAGE_TAG}"
- echo "${GH_DIR}"
- export DEBUG=zombie,zombie::network-node
- export ZOMBIENET_INTEGRATION_TEST_IMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}
- export MALUS_IMAGE=${MALUS_IMAGE_NAME}:${MALUS_IMAGE_TAG}
- export COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}
script:
- /home/nonroot/zombie-net/scripts/run-test-env-manager.sh
--github-remote-dir="${GH_DIR}"
--test="0002-parachains-disputes.feature"
allow_failure: true
retry: 2
tags:
- zombienet-polkadot-integration-test
zombienet-tests-malus-dispute-valid:
stage: deploy
image: "${ZOMBIENET_IMAGE}"
@@ -644,7 +706,7 @@ zombienet-tests-malus-dispute-valid:
needs:
- job: publish-polkadot-image
- job: publish-malus-image
- job: publish-adder-collator-image
- job: publish-test-collators-image
variables:
GH_DIR: "https://github.com/paritytech/polkadot/tree/${CI_COMMIT_SHORT_SHA}/node/malus/integrationtests"
before_script:
+40
View File
@@ -11096,6 +11096,46 @@ dependencies = [
"substrate-wasm-builder",
]
[[package]]
name = "test-parachain-undying"
version = "0.9.17"
dependencies = [
"dlmalloc",
"log",
"parity-scale-codec",
"polkadot-parachain",
"sp-io",
"sp-std",
"substrate-wasm-builder",
"tiny-keccak",
]
[[package]]
name = "test-parachain-undying-collator"
version = "0.9.17"
dependencies = [
"clap",
"futures 0.3.21",
"futures-timer",
"log",
"parity-scale-codec",
"polkadot-cli",
"polkadot-node-core-pvf",
"polkadot-node-primitives",
"polkadot-node-subsystem",
"polkadot-parachain",
"polkadot-primitives",
"polkadot-service",
"polkadot-test-service",
"sc-cli",
"sc-service",
"sp-core",
"sp-keyring",
"substrate-test-utils",
"test-parachain-undying",
"tokio",
]
[[package]]
name = "test-parachains"
version = "0.9.17"
+2
View File
@@ -105,6 +105,8 @@ members = [
"parachain/test-parachains/adder",
"parachain/test-parachains/adder/collator",
"parachain/test-parachains/halt",
"parachain/test-parachains/undying",
"parachain/test-parachains/undying/collator",
"utils/staking-miner",
"utils/remote-ext-tests/bags-list",
"utils/generate-bags",
@@ -87,7 +87,7 @@ impl SubstrateCli for Cli {
}
fn executable_name() -> String {
"polkadot".into()
"adder-collator".into()
}
fn load_spec(&self, id: &str) -> std::result::Result<Box<dyn sc_service::ChainSpec>, String> {
@@ -0,0 +1,28 @@
[package]
name = "test-parachain-undying"
version = "0.9.17"
authors = ["Parity Technologies <admin@parity.io>"]
description = "Test parachain for zombienet integration tests"
edition = "2021"
build = "build.rs"
[dependencies]
parachain = { package = "polkadot-parachain", path = "../../", default-features = false, features = [ "wasm-api" ] }
parity-scale-codec = { version = "2.3.1", default-features = false, features = ["derive"] }
sp-std = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
tiny-keccak = { version = "2.0.2", features = ["keccak"] }
dlmalloc = { version = "0.2.3", features = [ "global" ] }
log = { version = "0.4.14", default-features = false }
# We need to make sure the global allocator is disabled until we have support of full substrate externalities
sp-io = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false, features = [ "disable_allocator" ] }
[build-dependencies]
substrate-wasm-builder = { git = "https://github.com/paritytech/substrate", branch = "master" }
[features]
default = [ "std" ]
std = [
"parachain/std",
"sp-std/std",
]
@@ -0,0 +1,21 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
use substrate_wasm_builder::WasmBuilder;
fn main() {
WasmBuilder::new().with_current_project().export_heap_base().build()
}
@@ -0,0 +1,47 @@
[package]
name = "test-parachain-undying-collator"
version = "0.9.17"
authors = ["Parity Technologies <admin@parity.io>"]
description = "Collator for the undying test parachain"
edition = "2021"
[[bin]]
name = "undying-collator"
path = "src/main.rs"
[[bin]]
name = "undying_collator_puppet_worker"
path = "bin/puppet_worker.rs"
[dependencies]
parity-scale-codec = { version = "2.3.1", default-features = false, features = ["derive"] }
clap = { version = "3.0", features = ["derive"] }
futures = "0.3.19"
futures-timer = "3.0.2"
log = "0.4.13"
test-parachain-undying = { path = ".." }
polkadot-primitives = { path = "../../../../primitives" }
polkadot-cli = { path = "../../../../cli" }
polkadot-service = { path = "../../../../node/service", features = ["rococo-native"] }
polkadot-node-primitives = { path = "../../../../node/primitives" }
polkadot-node-subsystem = { path = "../../../../node/subsystem" }
sc-cli = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
# This one is tricky. Even though it is not used directly by the collator, we still need it for the
# `puppet_worker` binary, which is required for the integration test. However, this shouldn't be
# a big problem since it is used transitively anyway.
polkadot-node-core-pvf = { path = "../../../../node/core/pvf" }
[dev-dependencies]
polkadot-parachain = { path = "../../.." }
polkadot-test-service = { path = "../../../../node/test/service" }
substrate-test-utils = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
tokio = { version = "1.15", features = ["macros"] }
@@ -0,0 +1,17 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
polkadot_node_core_pvf::decl_puppet_worker_main!();
@@ -0,0 +1,135 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Polkadot CLI library.
use clap::Parser;
use sc_cli::{RuntimeVersion, SubstrateCli};
/// Sub-commands supported by the collator.
#[derive(Debug, Parser)]
pub enum Subcommand {
/// Export the genesis state of the parachain.
#[clap(name = "export-genesis-state")]
ExportGenesisState(ExportGenesisStateCommand),
/// Export the genesis wasm of the parachain.
#[clap(name = "export-genesis-wasm")]
ExportGenesisWasm(ExportGenesisWasmCommand),
}
/// Command for exporting the genesis state of the parachain
#[derive(Debug, Parser)]
pub struct ExportGenesisStateCommand {
/// Id of the parachain this collator collates for.
#[clap(long, default_value = "100")]
pub parachain_id: u32,
/// The target raw PoV size in bytes. Minimum value is 64.
#[clap(long, default_value = "1024")]
pub pov_size: usize,
/// The PVF execution complexity. Actually specifies how many iterations/signatures
/// we compute per block.
#[clap(long, default_value = "1")]
pub pvf_complexity: u32,
}
/// Command for exporting the genesis wasm file.
#[derive(Debug, Parser)]
pub struct ExportGenesisWasmCommand {}
#[allow(missing_docs)]
#[derive(Debug, Parser)]
pub struct RunCmd {
#[allow(missing_docs)]
#[clap(flatten)]
pub base: sc_cli::RunCmd,
/// Id of the parachain this collator collates for.
#[clap(long, default_value = "2000")]
pub parachain_id: u32,
/// The target raw PoV size in bytes. Minimum value is 64.
#[clap(long, default_value = "1024")]
pub pov_size: usize,
/// The PVF execution complexity. Actually specifies how many iterations/signatures
/// we compute per block.
#[clap(long, default_value = "1")]
pub pvf_complexity: u32,
}
#[allow(missing_docs)]
#[derive(Debug, Parser)]
pub struct Cli {
#[clap(subcommand)]
pub subcommand: Option<Subcommand>,
#[clap(flatten)]
pub run: RunCmd,
}
impl SubstrateCli for Cli {
fn impl_name() -> String {
"Parity Zombienet/Undying".into()
}
fn impl_version() -> String {
env!("CARGO_PKG_VERSION").into()
}
fn description() -> String {
env!("CARGO_PKG_DESCRIPTION").into()
}
fn author() -> String {
env!("CARGO_PKG_AUTHORS").into()
}
fn support_url() -> String {
"https://github.com/paritytech/polkadot/issues/new".into()
}
fn copyright_start_year() -> i32 {
2022
}
fn executable_name() -> String {
"undying-collator".into()
}
fn load_spec(&self, id: &str) -> std::result::Result<Box<dyn sc_service::ChainSpec>, String> {
let id = if id.is_empty() { "rococo" } else { id };
Ok(match id {
"rococo-staging" =>
Box::new(polkadot_service::chain_spec::rococo_staging_testnet_config()?),
"rococo-local" =>
Box::new(polkadot_service::chain_spec::rococo_local_testnet_config()?),
"rococo" => Box::new(polkadot_service::chain_spec::rococo_config()?),
path => {
let path = std::path::PathBuf::from(path);
Box::new(polkadot_service::RococoChainSpec::from_json_file(path)?)
},
})
}
fn native_runtime_version(
_spec: &Box<dyn polkadot_service::ChainSpec>,
) -> &'static RuntimeVersion {
&polkadot_service::rococo_runtime::VERSION
}
}
@@ -0,0 +1,419 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Collator for the `Undying` test parachain.
use futures::channel::oneshot;
use futures_timer::Delay;
use parity_scale_codec::{Decode, Encode};
use polkadot_node_primitives::{
maybe_compress_pov, Collation, CollationResult, CollationSecondedSignal, CollatorFn,
MaybeCompressedPoV, PoV, Statement,
};
use polkadot_primitives::v1::{CollatorId, CollatorPair, Hash};
use sp_core::{traits::SpawnNamed, Pair};
use std::{
collections::HashMap,
sync::{
atomic::{AtomicU32, Ordering},
Arc, Mutex,
},
time::Duration,
};
use test_parachain_undying::{execute, hash_state, BlockData, GraveyardState, HeadData};
/// Default PoV size which also drives state size.
const DEFAULT_POV_SIZE: usize = 1000;
/// Default PVF time complexity - 1 signature per block.
const DEFAULT_PVF_COMPLEXITY: u32 = 1;
/// Calculates the head and state for the block with the given `number`.
fn calculate_head_and_state_for_number(
number: u64,
graveyard_size: usize,
pvf_complexity: u32,
) -> (HeadData, GraveyardState) {
let index = 0u64;
let mut graveyard = vec![0u8; graveyard_size * graveyard_size];
let zombies = 0;
let seal = [0u8; 32];
// Ensure a larger compressed PoV.
graveyard.iter_mut().enumerate().for_each(|(i, grave)| {
*grave = i as u8;
});
let mut state = GraveyardState { index, graveyard, zombies, seal };
let mut head =
HeadData { number: 0, parent_hash: Hash::default().into(), post_state: hash_state(&state) };
while head.number < number {
let block = BlockData { state, tombstones: 1_000, iterations: pvf_complexity };
let (new_head, new_state) =
execute(head.hash(), head.clone(), block).expect("Produces valid block");
head = new_head;
state = new_state;
}
(head, state)
}
/// The state of the undying parachain.
struct State {
// We need to keep these around until the including relay chain blocks are finalized.
// This is because disputes can trigger reverts up to last finalized block, so we
// want that state to collate on older relay chain heads.
head_to_state: HashMap<Arc<HeadData>, GraveyardState>,
number_to_head: HashMap<u64, Arc<HeadData>>,
/// Block number of the best block.
best_block: u64,
/// PVF time complexity.
pvf_complexity: u32,
/// Defines the state size (Vec<u8>). Our PoV includes the entire state so this value will
/// drive the PoV size.
/// Important note: block execution heavily clones this state, so something like 300.000 is
/// the max value here, otherwise we'll get OOM during wasm execution.
/// TODO: Implement a static state, and use `ballast` to inflate the PoV size. This way
/// we can just discard the `ballast` before processing the block.
graveyard_size: usize,
}
impl State {
/// Init the genesis state.
fn genesis(graveyard_size: usize, pvf_complexity: u32) -> Self {
let index = 0u64;
let mut graveyard = vec![0u8; graveyard_size * graveyard_size];
let zombies = 0;
let seal = [0u8; 32];
// Ensure a larger compressed PoV.
graveyard.iter_mut().enumerate().for_each(|(i, grave)| {
*grave = i as u8;
});
let state = GraveyardState { index, graveyard, zombies, seal };
let head_data =
HeadData { number: 0, parent_hash: Default::default(), post_state: hash_state(&state) };
let head_data = Arc::new(head_data);
Self {
head_to_state: vec![(head_data.clone(), state.clone())].into_iter().collect(),
number_to_head: vec![(0, head_data)].into_iter().collect(),
best_block: 0,
pvf_complexity,
graveyard_size,
}
}
/// Advance the state and produce a new block based on the given `parent_head`.
///
/// Returns the new [`BlockData`] and the new [`HeadData`].
fn advance(&mut self, parent_head: HeadData) -> (BlockData, HeadData) {
self.best_block = parent_head.number;
let state = if let Some(head_data) = self.number_to_head.get(&self.best_block) {
self.head_to_state.get(head_data).cloned().unwrap_or_else(|| {
calculate_head_and_state_for_number(
parent_head.number,
self.graveyard_size,
self.pvf_complexity,
)
.1
})
} else {
let (_, state) = calculate_head_and_state_for_number(
parent_head.number,
self.graveyard_size,
self.pvf_complexity,
);
state
};
// Start with prev state and transaction to execute (place 1000 tombstones).
let block = BlockData { state, tombstones: 1000, iterations: self.pvf_complexity };
let (new_head, new_state) =
execute(parent_head.hash(), parent_head, block.clone()).expect("Produces valid block");
let new_head_arc = Arc::new(new_head.clone());
self.head_to_state.insert(new_head_arc.clone(), new_state);
self.number_to_head.insert(new_head.number, new_head_arc);
(block, new_head)
}
}
/// The collator of the undying parachain.
pub struct Collator {
state: Arc<Mutex<State>>,
key: CollatorPair,
seconded_collations: Arc<AtomicU32>,
}
impl Default for Collator {
fn default() -> Self {
Self::new(DEFAULT_POV_SIZE, DEFAULT_PVF_COMPLEXITY)
}
}
impl Collator {
/// Create a new collator instance with the state initialized from genesis and `pov_size`
/// parameter. The same parameter needs to be passed when exporting the genesis state.
pub fn new(pov_size: usize, pvf_complexity: u32) -> Self {
let graveyard_size = ((pov_size / std::mem::size_of::<u8>()) as f64).sqrt().ceil() as usize;
log::info!(
"PoV target size: {} bytes. Graveyard size: ({} x {})",
pov_size,
graveyard_size,
graveyard_size
);
log::info!("PVF time complexity: {}", pvf_complexity);
Self {
state: Arc::new(Mutex::new(State::genesis(graveyard_size, pvf_complexity))),
key: CollatorPair::generate().0,
seconded_collations: Arc::new(AtomicU32::new(0)),
}
}
/// Get the SCALE encoded genesis head of the parachain.
pub fn genesis_head(&self) -> Vec<u8> {
self.state
.lock()
.unwrap()
.number_to_head
.get(&0)
.expect("Genesis header exists")
.encode()
}
/// Get the validation code of the undying parachain.
pub fn validation_code(&self) -> &[u8] {
test_parachain_undying::wasm_binary_unwrap()
}
/// Get the collator key.
pub fn collator_key(&self) -> CollatorPair {
self.key.clone()
}
/// Get the collator id.
pub fn collator_id(&self) -> CollatorId {
self.key.public()
}
/// Create the collation function.
///
/// This collation function can be plugged into the overseer to generate collations for the undying parachain.
pub fn create_collation_function(
&self,
spawner: impl SpawnNamed + Clone + 'static,
) -> CollatorFn {
use futures::FutureExt as _;
let state = self.state.clone();
let seconded_collations = self.seconded_collations.clone();
Box::new(move |relay_parent, validation_data| {
let parent = HeadData::decode(&mut &validation_data.parent_head.0[..])
.expect("Decodes parent head");
let (block_data, head_data) = state.lock().unwrap().advance(parent);
log::info!(
"created a new collation on relay-parent({}): {:?}",
relay_parent,
head_data,
);
// The pov is the actually the initial state and the transactions.
let pov = PoV { block_data: block_data.encode().into() };
let collation = Collation {
upward_messages: Vec::new(),
horizontal_messages: Vec::new(),
new_validation_code: None,
head_data: head_data.encode().into(),
proof_of_validity: MaybeCompressedPoV::Raw(pov.clone()),
processed_downward_messages: 0,
hrmp_watermark: validation_data.relay_parent_number,
};
log::info!("Raw PoV size for collation: {} bytes", pov.block_data.0.len(),);
let compressed_pov = maybe_compress_pov(pov);
log::info!(
"Compressed PoV size for collation: {} bytes",
compressed_pov.block_data.0.len(),
);
let (result_sender, recv) = oneshot::channel::<CollationSecondedSignal>();
let seconded_collations = seconded_collations.clone();
spawner.spawn(
"undying-collator-seconded",
None,
async move {
if let Ok(res) = recv.await {
if !matches!(
res.statement.payload(),
Statement::Seconded(s) if s.descriptor.pov_hash == compressed_pov.hash(),
) {
log::error!(
"Seconded statement should match our collation: {:?}",
res.statement.payload()
);
std::process::exit(-1);
}
seconded_collations.fetch_add(1, Ordering::Relaxed);
}
}
.boxed(),
);
async move { Some(CollationResult { collation, result_sender: Some(result_sender) }) }
.boxed()
})
}
/// Wait until `blocks` are built and enacted.
pub async fn wait_for_blocks(&self, blocks: u64) {
let start_block = self.state.lock().unwrap().best_block;
loop {
Delay::new(Duration::from_secs(1)).await;
let current_block = self.state.lock().unwrap().best_block;
if start_block + blocks <= current_block {
return
}
}
}
/// Wait until `seconded` collations of this collator are seconded by a parachain validator.
///
/// The internal counter isn't de-duplicating the collations when counting the number of seconded collations. This
/// means when one collation is seconded by X validators, we record X seconded messages.
pub async fn wait_for_seconded_collations(&self, seconded: u32) {
let seconded_collations = self.seconded_collations.clone();
loop {
Delay::new(Duration::from_secs(1)).await;
if seconded <= seconded_collations.load(Ordering::Relaxed) {
return
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use futures::executor::block_on;
use polkadot_parachain::primitives::{ValidationParams, ValidationResult};
use polkadot_primitives::v1::{Hash, PersistedValidationData};
#[test]
fn collator_works() {
let spawner = sp_core::testing::TaskExecutor::new();
let collator = Collator::new(1_000, 1);
let collation_function = collator.create_collation_function(spawner);
for i in 0..5 {
let parent_head =
collator.state.lock().unwrap().number_to_head.get(&i).unwrap().clone();
let validation_data = PersistedValidationData {
parent_head: parent_head.encode().into(),
..Default::default()
};
let collation =
block_on(collation_function(Default::default(), &validation_data)).unwrap();
validate_collation(&collator, (*parent_head).clone(), collation.collation);
}
}
fn validate_collation(collator: &Collator, parent_head: HeadData, collation: Collation) {
use polkadot_node_core_pvf::testing::validate_candidate;
let block_data = match collation.proof_of_validity {
MaybeCompressedPoV::Raw(pov) => pov.block_data,
MaybeCompressedPoV::Compressed(_) => panic!("Only works with uncompressed povs"),
};
let ret_buf = validate_candidate(
collator.validation_code(),
&ValidationParams {
parent_head: parent_head.encode().into(),
block_data,
relay_parent_number: 1,
relay_parent_storage_root: Hash::zero(),
}
.encode(),
)
.unwrap();
let ret = ValidationResult::decode(&mut &ret_buf[..]).unwrap();
let new_head = HeadData::decode(&mut &ret.head_data.0[..]).unwrap();
assert_eq!(
**collator
.state
.lock()
.unwrap()
.number_to_head
.get(&(parent_head.number + 1))
.unwrap(),
new_head
);
}
#[test]
fn advance_to_state_when_parent_head_is_missing() {
let collator = Collator::new(1_000, 1);
let graveyard_size = collator.state.lock().unwrap().graveyard_size;
let mut head = calculate_head_and_state_for_number(10, graveyard_size, 1).0;
for i in 1..10 {
head = collator.state.lock().unwrap().advance(head).1;
assert_eq!(10 + i, head.number);
}
let collator = Collator::new(1_000, 1);
let mut second_head = collator
.state
.lock()
.unwrap()
.number_to_head
.get(&0)
.cloned()
.unwrap()
.as_ref()
.clone();
for _ in 1..20 {
second_head = collator.state.lock().unwrap().advance(second_head.clone()).1;
}
assert_eq!(second_head, head);
}
}
@@ -0,0 +1,111 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Collator for the `Undying` test parachain.
use polkadot_cli::{Error, Result};
use polkadot_node_primitives::CollationGenerationConfig;
use polkadot_node_subsystem::messages::{CollationGenerationMessage, CollatorProtocolMessage};
use polkadot_primitives::v1::Id as ParaId;
use sc_cli::{Error as SubstrateCliError, Role, SubstrateCli};
use sp_core::hexdisplay::HexDisplay;
use test_parachain_undying_collator::Collator;
mod cli;
use cli::Cli;
fn main() -> Result<()> {
let cli = Cli::from_args();
match cli.subcommand {
Some(cli::Subcommand::ExportGenesisState(params)) => {
// `pov_size` and `pvf_complexity` need to match the ones that we start the collator
// with.
let collator = Collator::new(params.pov_size, params.pvf_complexity);
println!("0x{:?}", HexDisplay::from(&collator.genesis_head()));
Ok::<_, Error>(())
},
Some(cli::Subcommand::ExportGenesisWasm(_params)) => {
// We pass some dummy values for `pov_size` and `pvf_complexity` as these don't
// matter for `wasm` export.
println!("0x{:?}", HexDisplay::from(&Collator::default().validation_code()));
Ok(())
},
None => {
let runner = cli.create_runner(&cli.run.base).map_err(|e| {
SubstrateCliError::Application(
Box::new(e) as Box<(dyn 'static + Send + Sync + std::error::Error)>
)
})?;
runner.run_node_until_exit(|config| async move {
let role = config.role.clone();
match role {
Role::Light => Err("Light client not supported".into()),
_ => {
let collator = Collator::new(cli.run.pov_size, cli.run.pvf_complexity);
let full_node = polkadot_service::build_full(
config,
polkadot_service::IsCollator::Yes(collator.collator_key()),
None,
true,
None,
None,
false,
polkadot_service::RealOverseerGen,
)
.map_err(|e| e.to_string())?;
let mut overseer_handle = full_node
.overseer_handle
.expect("Overseer handle should be initialized for collators");
let genesis_head_hex =
format!("0x{:?}", HexDisplay::from(&collator.genesis_head()));
let validation_code_hex =
format!("0x{:?}", HexDisplay::from(&collator.validation_code()));
let para_id = ParaId::from(cli.run.parachain_id);
log::info!("Running `Undying` collator for parachain id: {}", para_id);
log::info!("Genesis state: {}", genesis_head_hex);
log::info!("Validation code: {}", validation_code_hex);
let config = CollationGenerationConfig {
key: collator.collator_key(),
collator: collator
.create_collation_function(full_node.task_manager.spawn_handle()),
para_id,
};
overseer_handle
.send_msg(CollationGenerationMessage::Initialize(config), "Collator")
.await;
overseer_handle
.send_msg(CollatorProtocolMessage::CollateOn(para_id), "Collator")
.await;
Ok(full_node.task_manager)
},
}
})
},
}?;
Ok(())
}
@@ -0,0 +1,86 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Integration test that ensures that we can build and include parachain
//! blocks of the `Undying` parachain.
const PUPPET_EXE: &str = env!("CARGO_BIN_EXE_undying_collator_puppet_worker");
// If this test is failing, make sure to run all tests with the `real-overseer` feature being enabled.
#[substrate_test_utils::test]
async fn collating_using_undying_collator() {
use polkadot_primitives::v1::Id as ParaId;
use sp_keyring::AccountKeyring::*;
let mut builder = sc_cli::LoggerBuilder::new("");
builder.with_colors(false);
builder.init().expect("Set up logger");
let para_id = ParaId::from(100);
let alice_config = polkadot_test_service::node_config(
|| {},
tokio::runtime::Handle::current(),
Alice,
Vec::new(),
true,
);
// start alice
let alice = polkadot_test_service::run_validator_node(alice_config, Some(PUPPET_EXE.into()));
let bob_config = polkadot_test_service::node_config(
|| {},
tokio::runtime::Handle::current(),
Bob,
vec![alice.addr.clone()],
true,
);
// start bob
let bob = polkadot_test_service::run_validator_node(bob_config, Some(PUPPET_EXE.into()));
let collator = test_parachain_undying_collator::Collator::new(1_000, 1);
// register parachain
alice
.register_parachain(para_id, collator.validation_code().to_vec(), collator.genesis_head())
.await
.unwrap();
// run the collator node
let mut charlie = polkadot_test_service::run_collator_node(
tokio::runtime::Handle::current(),
Charlie,
|| {},
vec![alice.addr.clone(), bob.addr.clone()],
collator.collator_key(),
);
charlie
.register_collator(
collator.collator_key(),
para_id,
collator.create_collation_function(charlie.task_manager.spawn_handle()),
)
.await;
// Wait until the parachain has 4 blocks produced.
collator.wait_for_blocks(4).await;
// Wait until the collator received `12` seconded statements for its collations.
collator.wait_for_seconded_collations(12).await;
}
@@ -0,0 +1,162 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Basic parachain that adds a number as part of its state.
#![no_std]
#![cfg_attr(
not(feature = "std"),
feature(core_intrinsics, lang_items, core_panic_info, alloc_error_handler)
)]
use parity_scale_codec::{Decode, Encode};
use sp_std::vec::Vec;
use tiny_keccak::{Hasher as _, Keccak};
#[cfg(not(feature = "std"))]
mod wasm_validation;
#[cfg(not(feature = "std"))]
#[global_allocator]
static ALLOC: dlmalloc::GlobalDlmalloc = dlmalloc::GlobalDlmalloc;
const LOG_TARGET: &str = "runtime::undying";
// Make the WASM binary available.
#[cfg(feature = "std")]
include!(concat!(env!("OUT_DIR"), "/wasm_binary.rs"));
fn keccak256(input: &[u8]) -> [u8; 32] {
let mut out = [0u8; 32];
let mut keccak256 = Keccak::v256();
keccak256.update(input);
keccak256.finalize(&mut out);
out
}
/// Wasm binary unwrapped. If built with `BUILD_DUMMY_WASM_BINARY`, the function panics.
#[cfg(feature = "std")]
pub fn wasm_binary_unwrap() -> &'static [u8] {
WASM_BINARY.expect(
"Development wasm binary is not available. Testing is only \
supported with the flag disabled.",
)
}
/// Head data for this parachain.
#[derive(Default, Clone, Hash, Eq, PartialEq, Encode, Decode, Debug)]
pub struct HeadData {
/// Block number
pub number: u64,
/// parent block keccak256
pub parent_hash: [u8; 32],
/// hash of post-execution state.
pub post_state: [u8; 32],
}
impl HeadData {
pub fn hash(&self) -> [u8; 32] {
keccak256(&self.encode())
}
}
/// Block data for this parachain.
#[derive(Default, Clone, Encode, Decode, Debug)]
pub struct GraveyardState {
/// The grave index of the last placed tombstone.
pub index: u64,
/// We use a matrix where each element represents a grave.
/// The unsigned integer tracks the number of tombstones raised on
/// each grave.
pub graveyard: Vec<u8>,
// TODO: Add zombies. All of the graves produce zombies at a regular interval
// defined in blocks. The number of zombies produced scales with the tombstones.
// This would allow us to have a configurable and reproducible PVF execution time.
// However, PVF preparation time will likely rely on prebuild wasm binaries.
pub zombies: u64,
// Grave seal.
pub seal: [u8; 32],
}
/// Block data for this parachain.
#[derive(Default, Clone, Encode, Decode, Debug)]
pub struct BlockData {
/// The state
pub state: GraveyardState,
/// The number of tombstones to erect per iteration. For each tombstone placed
/// a hash operation is performed as CPU burn.
pub tombstones: u64,
/// The number of iterations to perform.
pub iterations: u32,
}
pub fn hash_state(state: &GraveyardState) -> [u8; 32] {
keccak256(state.encode().as_slice())
}
/// Executes all graveyard transactions in the block.
pub fn execute_transaction(mut block_data: BlockData) -> GraveyardState {
let graveyard_size = block_data.state.graveyard.len();
for _ in 0..block_data.iterations {
for _ in 0..block_data.tombstones {
block_data.state.graveyard[block_data.state.index as usize] =
block_data.state.graveyard[block_data.state.index as usize].wrapping_add(1);
block_data.state.index =
((block_data.state.index.saturating_add(1)) as usize % graveyard_size) as u64;
}
// Chain hash the seals and burn CPU.
block_data.state.seal = hash_state(&block_data.state);
}
block_data.state
}
/// Start state mismatched with parent header's state hash.
#[derive(Debug)]
pub struct StateMismatch;
/// Execute a block body on top of given parent head, producing new parent head
/// and new state if valid.
pub fn execute(
parent_hash: [u8; 32],
parent_head: HeadData,
block_data: BlockData,
) -> Result<(HeadData, GraveyardState), StateMismatch> {
assert_eq!(parent_hash, parent_head.hash());
if hash_state(&block_data.state) != parent_head.post_state {
log::debug!(
target: LOG_TARGET,
"state has diff vs head: {:?} vs {:?}",
hash_state(&block_data.state),
parent_head.post_state,
);
return Err(StateMismatch)
}
// We need to clone the block data as the fn will mutate it's state.
let new_state = execute_transaction(block_data.clone());
Ok((
HeadData {
number: parent_head.number + 1,
parent_hash,
post_state: hash_state(&new_state),
},
new_state,
))
}
@@ -0,0 +1,45 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! WASM validation for the `Undying` parachain.
use crate::{BlockData, HeadData};
use parachain::primitives::{HeadData as GenericHeadData, ValidationResult};
use parity_scale_codec::{Decode, Encode};
#[no_mangle]
pub extern "C" fn validate_block(params: *const u8, len: usize) -> u64 {
let params = unsafe { parachain::load_params(params, len) };
let parent_head =
HeadData::decode(&mut &params.parent_head.0[..]).expect("invalid parent head format.");
let mut block_data =
BlockData::decode(&mut &params.block_data.0[..]).expect("invalid block data format.");
let parent_hash = crate::keccak256(&params.parent_head.0[..]);
let (new_head, _) =
crate::execute(parent_hash, parent_head, block_data).expect("Executes block");
parachain::write_result(&ValidationResult {
head_data: GenericHeadData(new_head.encode()),
new_validation_code: None,
upward_messages: sp_std::vec::Vec::new(),
horizontal_messages: sp_std::vec::Vec::new(),
processed_downward_messages: 0,
hrmp_watermark: params.relay_parent_number,
})
}
@@ -35,11 +35,13 @@ RUN apt-get update && \
# add adder-collator binary to docker image
COPY ./adder-collator /usr/local/bin
COPY ./undying-collator /usr/local/bin
USER adder-collator
# check if executable works in this container
RUN /usr/local/bin/adder-collator --version
RUN /usr/local/bin/undying-collator --version
EXPOSE 30333 9933 9944
VOLUME ["/adder-collator"]
@@ -0,0 +1,102 @@
Description: PVF preparation & execution time
Network: ./0001-parachains-pvf.toml
Creds: config
# Some sanity checks
alice: is up
bob: is up
charlie: is up
dave: is up
eve: is up
ferdie: is up
one: is up
two: is up
# Check authority status.
alice: reports node_roles is 4
bob: reports node_roles is 4
charlie: reports node_roles is 4
dave: reports node_roles is 4
eve: reports node_roles is 4
ferdie: reports node_roles is 4
one: reports node_roles is 4
two: reports node_roles is 4
# Ensure parachains are registered.
alice: parachain 2000 is registered within 60 seconds
bob: parachain 2001 is registered within 60 seconds
charlie: parachain 2002 is registered within 60 seconds
dave: parachain 2003 is registered within 60 seconds
ferdie: parachain 2004 is registered within 60 seconds
eve: parachain 2005 is registered within 60 seconds
one: parachain 2006 is registered within 60 seconds
two: parachain 2007 is registered within 60 seconds
# Check if network is fully connected.
alice: reports peers count is at least 15 within 15 seconds
bob: reports peers count is at least 15 within 15 seconds
charlie: reports peers count is at least 15 within 15 seconds
dave: reports peers count is at least 15 within 15 seconds
ferdie: reports peers count is at least 15 within 15 seconds
eve: reports peers count is at least 15 within 15 seconds
one: reports peers count is at least 15 within 15 seconds
two: reports peers count is at least 15 within 15 seconds
# Ensure parachains made progress.
alice: parachain 2000 block height is at least 10 within 300 seconds
alice: parachain 2001 block height is at least 10 within 300 seconds
alice: parachain 2002 block height is at least 10 within 300 seconds
alice: parachain 2003 block height is at least 10 within 300 seconds
alice: parachain 2004 block height is at least 10 within 300 seconds
alice: parachain 2005 block height is at least 10 within 300 seconds
alice: parachain 2006 block height is at least 10 within 300 seconds
alice: parachain 2007 block height is at least 10 within 300 seconds
# Check preparation time is under 10s.
# Check all buckets <= 10.
alice: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
bob: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
charlie: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
dave: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
ferdie: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
eve: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
one: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
two: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
# Check all buckets >= 20.
alice: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
bob: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
charlie: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
dave: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
ferdie: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
eve: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
one: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
two: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
# Check execution time.
# There are two different timeout conditions: BACKING_EXECUTION_TIMEOUT(2s) and
# APPROVAL_EXECUTION_TIMEOUT(6s). Currently these are not differentiated by metrics
# because the metrics are defined in `polkadot-node-core-pvf` which is a level below
# the relevant subsystems.
# That being said, we will take the simplifying assumption of testing only the
# 2s timeout.
# We do this check by ensuring all executions fall into bucket le="2" or lower.
# First, check if we have at least 1 sample, but we should have many more.
alice: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
bob: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
charlie: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
dave: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
ferdie: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
eve: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
one: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
two: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
# Check if we have no samples > 2s.
alice: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
bob: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
charlie: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
dave: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
ferdie: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
eve: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
one: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
two: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
@@ -0,0 +1,132 @@
[settings]
timeout = 1000
[relaychain]
default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}"
chain = "rococo-local"
chain_spec_command = "polkadot build-spec --chain rococo-local --disable-default-bootnode"
[[relaychain.nodes]]
name = "alice"
extra_args = [ "--alice", "-lparachain=debug,runtime=debug" ]
[[relaychain.nodes]]
name = "bob"
extra_args = [ "--bob", "-lparachain=debug,runtime=debug"]
[[relaychain.nodes]]
name = "charlie"
extra_args = [ "--charlie", "-lparachain=debug,runtime=debug" ]
[[relaychain.nodes]]
name = "dave"
extra_args = [ "--dave", "-lparachain=debug,runtime=debug"]
[[relaychain.nodes]]
name = "ferdie"
extra_args = [ "--ferdie", "-lparachain=debug,runtime=debug" ]
[[relaychain.nodes]]
name = "eve"
extra_args = [ "--eve", "-lparachain=debug,runtime=debug"]
[[relaychain.nodes]]
name = "one"
extra_args = [ "--one", "-lparachain=debug,runtime=debug" ]
[[relaychain.nodes]]
name = "two"
extra_args = [ "--two", "-lparachain=debug,runtime=debug"]
[[parachains]]
id = 2000
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=1"
[parachains.collator]
name = "collator01"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--pvf-complexity=1", "--parachain-id=2000"]
[[parachains]]
id = 2001
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=10"
[parachains.collator]
name = "collator02"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--parachain-id=2001", "--pvf-complexity=10"]
[[parachains]]
id = 2002
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=100"
[parachains.collator]
name = "collator03"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--parachain-id=2002", "--pvf-complexity=100"]
[[parachains]]
id = 2003
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=20000 --pvf-complexity=300"
[parachains.collator]
name = "collator04"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=20000", "--parachain-id=2003", "--pvf-complexity=300"]
[[parachains]]
id = 2004
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=300"
[parachains.collator]
name = "collator05"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--parachain-id=2004", "--pvf-complexity=300"]
[[parachains]]
id = 2005
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=20000 --pvf-complexity=400"
[parachains.collator]
name = "collator06"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=20000", "--pvf-complexity=400", "--parachain-id=2005"]
[[parachains]]
id = 2006
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=300"
[parachains.collator]
name = "collator07"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--pvf-complexity=300", "--parachain-id=2006"]
[[parachains]]
id = 2007
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=300"
[parachains.collator]
name = "collator08"
image = "{{COL_IMAGE}}"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--pvf-complexity=300", "--parachain-id=2007"]
[types.Header]
number = "u64"
parent_hash = "Hash"
post_state = "Hash"
@@ -0,0 +1,69 @@
Description: Disputes initiation, conclusion and lag
Network: ./0002-parachains-disputes.toml
Creds: config
alice: is up
bob: is up
charlie: is up
dave: is up
eve: is up
ferdie: is up
one: is up
two: is up
# Check authority status and peers.
alice: reports node_roles is 4
bob: reports node_roles is 4
charlie: reports node_roles is 4
dave: reports node_roles is 4
eve: reports node_roles is 4
ferdie: reports node_roles is 4
one: reports node_roles is 4
two: reports node_roles is 4
# Ensure parachains are registered.
alice: parachain 2000 is registered within 30 seconds
bob: parachain 2001 is registered within 30 seconds
charlie: parachain 2002 is registered within 30 seconds
dave: parachain 2003 is registered within 30 seconds
alice: reports peers count is at least 11 within 15 seconds
bob: reports peers count is at least 11 within 15 seconds
charlie: reports peers count is at least 11 within 15 seconds
dave: reports peers count is at least 11 within 15 seconds
ferdie: reports peers count is at least 1 within 15 seconds
eve: reports peers count is at least 11 within 15 seconds
one: reports peers count is at least 11 within 15 seconds
two: reports peers count is at least 11 within 15 seconds
# Ensure parachains made progress.
alice: parachain 2000 block height is at least 10 within 200 seconds
alice: parachain 2001 block height is at least 10 within 200 seconds
alice: parachain 2002 block height is at least 10 within 200 seconds
alice: parachain 2003 block height is at least 10 within 200 seconds
# Check if disputes are initiated and concluded.
# TODO: check if disputes are concluded faster than initiated.
eve: reports parachain_candidate_disputes_total is at least 10 within 15 seconds
eve: reports parachain_candidate_dispute_concluded{validity="valid"} is at least 10 within 15 seconds
eve: reports parachain_candidate_dispute_concluded{validity="invalid"} is 0 within 15 seconds
# Check lag - approval
alice: reports polkadot_parachain_approval_checking_finality_lag is 0
bob: reports polkadot_parachain_approval_checking_finality_lag is 0
charlie: reports polkadot_parachain_approval_checking_finality_lag is 0
dave: reports polkadot_parachain_approval_checking_finality_lag is 0
ferdie: reports polkadot_parachain_approval_checking_finality_lag is 0
eve: reports polkadot_parachain_approval_checking_finality_lag is 0
one: reports polkadot_parachain_approval_checking_finality_lag is 0
two: reports polkadot_parachain_approval_checking_finality_lag is 0
# Check lag - dispute conclusion
alice: reports polkadot_parachain_disputes_finality_lag is 0
bob: reports polkadot_parachain_disputes_finality_lag is 0
charlie: reports polkadot_parachain_disputes_finality_lag is 0
dave: reports polkadot_parachain_disputes_finality_lag is 0
ferdie: reports polkadot_parachain_disputes_finality_lag is 0
eve: reports polkadot_parachain_disputes_finality_lag is 0
one: reports polkadot_parachain_disputes_finality_lag is 0
two: reports polkadot_parachain_disputes_finality_lag is 0
@@ -0,0 +1,99 @@
[settings]
timeout = 1000
[relaychain.genesis.runtime.runtime_genesis_config.configuration.config]
max_validators_per_core = 2
needed_approvals = 2
[relaychain]
default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}"
chain = "rococo-local"
chain_spec_command = "polkadot build-spec --chain rococo-local --disable-default-bootnode"
default_command = "polkadot"
[[relaychain.nodes]]
image = "{{MALUS_IMAGE}}"
name = "alice"
command = "malus dispute-ancestor"
extra_args = [ "--alice", "-lparachain=debug" ]
[[relaychain.nodes]]
image = "{{MALUS_IMAGE}}"
name = "bob"
command = "malus dispute-ancestor"
extra_args = [ "--bob", "-lparachain=debug"]
[[relaychain.nodes]]
image = "{{MALUS_IMAGE}}"
name = "charlie"
command = "malus dispute-ancestor"
extra_args = [ "--charlie", "-lparachain=debug" ]
[[relaychain.nodes]]
name = "dave"
extra_args = [ "--dave", "-lparachain=debug"]
[[relaychain.nodes]]
name = "ferdie"
extra_args = [ "--ferdie", "-lparachain=debug" ]
[[relaychain.nodes]]
name = "eve"
extra_args = [ "--eve", "-lparachain=debug"]
[[relaychain.nodes]]
name = "one"
extra_args = [ "--one", "-lparachain=debug" ]
[[relaychain.nodes]]
name = "two"
extra_args = [ "--two", "-lparachain=debug"]
[[parachains]]
id = 2000
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=1"
[parachains.collator]
image = "{{COL_IMAGE}}"
name = "collator01"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--pvf-complexity=1", "--parachain-id=2000"]
[[parachains]]
id = 2001
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=2"
[parachains.collator]
image = "{{COL_IMAGE}}"
name = "collator02"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--parachain-id=2001", "--pvf-complexity=2"]
[[parachains]]
id = 2002
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=100000 --pvf-complexity=10"
[parachains.collator]
image = "{{COL_IMAGE}}"
name = "collator03"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=100000", "--parachain-id=2002", "--pvf-complexity=10"]
[[parachains]]
id = 2003
addToGenesis = true
genesis_state_generator = "undying-collator export-genesis-state --pov-size=20000 --pvf-complexity=1000"
[parachains.collator]
image = "{{COL_IMAGE}}"
name = "collator04"
command = "undying-collator"
args = ["-lparachain=debug", "--pov-size=20000", "--parachain-id=2003", "--pvf-complexity=1000"]
[types.Header]
number = "u64"
parent_hash = "Hash"
post_state = "Hash"
@@ -1,5 +1,6 @@
[settings]
timeout = 1000
bootnode = true
[relaychain]
default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}"