From ea5dbd047520adb63b6102ed0d7a1d48df6dc4fe Mon Sep 17 00:00:00 2001 From: Javier Viola Date: Sat, 20 Nov 2021 15:03:28 +0100 Subject: [PATCH] introduce malus + zombienet based integration tests (#4131) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test/malus: craft the first maliciously disputing actor * initial draft * Add Dockerfile and instructions how to use it to build malus image locally * Forgot one flag for the build cmd * we are not docker specific, we are happy to use any containerruntime * shuffle things around * add initial tera based integration test * chores * fixins * simple setup to start * other samples (WIP) * add Docker version with cargo-chef * update substarte and small change of orders in commands in the container file * metrics one * fmt * minor * fixin * fix metric names * -d * add open gauge * fmt * spellcheck * fix test * adjust to changed error messages * refactor, more malus impls * more malus changes * foo * minor cleanup * suggest garbage candidate * chore * fix suggest garabge malus * malus: back garbage candidate * cargo lock * re-introduce metrics * chore: cargo fmt * undoe 1.54.0 output, CI uses 1.53.0 rustc * update location of js types * Fix trybuild * add tag to image name also; this will be replaced in the prod version * Tests fixed * add some fix me * add dockerfile for ci * Add docker file for malus for ci * use variables in .toml file * add chnages for malus test * some fixes * some more fixes * Update .gitlab-ci.yml * add local build for polkadot and malus * some fixes * enable disputes feature in CI * ok, ok * rename: MsgFilter -> MessageInterceptor * remove TODO that would not have worked * intercept * refactor * fix README and containers * fix * chore: cargo fmt * avoid some more malus-$VARIANT references * fix argument order * chore: add about * Update sanity check in relay chain selection * fix order, add dispute-unavailable-block malus * fixup: avoid underflow issue * it's all u32 * fix conditional use * Revert "it's all u32" This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e. * Revert "fixup: avoid underflow issue" This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47. * Revert "Update sanity check in relay chain selection" This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d. * update the malus bin * Update node/malus/integrationtests/0003-dispute-unavailable-block.feature Co-authored-by: Andronik Ordian * add some FIXME reminders * update path to index.js * Update .gitlab-ci.yml * Update node/malus/integrationtests/0001-dispute-valid-block.toml * try 1: make malus test run * chore: cargo fmt * temporary fix * use subcommand syntax from latest gurke * cargo +nightly fmt * add collator to a a test * docs: add env vars to README * update ci to run dispute-valid-block test * needs the polkadot image * Fix path for nodejs container * post merge fix * download proper dir containg configs for malus test * update the malus ci job * rm a whitespace * temp build for malus * use correct build command for temp malus * remove subcommands for now * set max validators per core in the default HostConfig * tabs * update beefy * fixup * fixup II * make one variant compile * make other variants compile * revert changes to chain_spec * fmt * build malus image from polkadot-test-malus again * revert unrelated changes * try fixing build-malus job * Revert "remove subcommands for now" This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e. * try fixing build-malus job II * MVP working dispute-ancestor * renames * fix PVF execution on malus * fix test * fix typo * fmt * checkmate * try something * make it actually work * some tweaks to 01 feature test * fmt * sleep a bit more * complete wococoization * some tweaks to 01 feature test * typo fix * use correct metric names * fix * ffs * . * try some rearrangement * Attempt to wait till initial node bootstrap in test * Fix test syntax * Run malus tests with v2 script * Proper symlink created * simnet v14 * add zombienet tests * add zombie net test - draft * add more tests to dispute suite * add within to fix tests * replace test directory and start test migration * migrate all the tests * add timeout to tests * reduce debug * make easy to test in dev * set appropriated debug * use image from ci * fix config for test * set images from ci * fix config * add COLIMAGE env * tweek tests * reduce debug * typo * wip, migrate old test to zombie-net * adjunt test config for zombie-net * run mauls 0001 test only * clean and setup smoke-test in zombie-net * add extra time to assertinons * clean code to merge and improve README * add info to access logs * improved readme * merge master and resolve conflicts * Update zombienet_tests/README.md Co-authored-by: Bernhard Schuster * clean and consolidate zombienet name * change runner in gitlab * add comment explain why we use wococo * change tag for runner * remove unused tests * remove dup Dockerfile and update description * fmt * fix compilation post-merge * fmt * cut me Some slack Co-authored-by: Bernhard Schuster Co-authored-by: radupopa2010 Co-authored-by: Bastian Köcher Co-authored-by: Anton Gavrilov Co-authored-by: Andronik Ordian Co-authored-by: Lldenaurois --- polkadot/.gitlab-ci.yml | 141 +++++++++-- polkadot/Cargo.lock | 8 + polkadot/cli/src/command.rs | 4 +- polkadot/node/malus/Cargo.toml | 25 +- polkadot/node/malus/README.md | 58 ++++- .../malus/container/Containerfile-cargo-chef | 155 ++++++++++++ polkadot/node/malus/container/build.sh | 1 + .../container/malus-local-build.Containerfile | 66 +++++ .../polkadot-local-build.Containerfile | 66 +++++ .../0001-dispute-valid-block.feature | 29 +++ .../0001-dispute-valid-block.toml | 40 +++ .../node/malus/src/{lib.rs => interceptor.rs} | 3 - polkadot/node/malus/src/malus.rs | 126 ++++++++++ polkadot/node/malus/src/shared.rs | 49 ++++ polkadot/node/malus/src/variant-a.rs | 121 --------- .../src/variants/back_garbage_candidate.rs | 230 ++++++++++++++++++ .../src/variants/dispute_valid_candidates.rs | 121 +++++++++ polkadot/node/malus/src/variants/mod.rs | 26 ++ .../src/variants/suggest_garbage_candidate.rs | 172 +++++++++++++ polkadot/node/service/src/lib.rs | 6 +- polkadot/node/subsystem-util/src/lib.rs | 7 +- .../test/polkadot-simnet/node/src/main.rs | 4 +- polkadot/scripts/dockerfiles/malus.Dockerfile | 50 ++++ polkadot/scripts/gitlab/lingua.dic | 8 +- polkadot/simnet_tests/README.md | 83 ------- .../parachains/configs/adder.json | 7 - .../configs/simple_rococo_testnet.toml | 53 ---- .../parachains/test_suit_description.toml | 13 - .../parachains/tests/parachains.feature | 6 - polkadot/zombienet_tests/README.md | 16 ++ .../0001-parachains-smoke-test.feature | 9 + .../0001-parachains-smoke-test.toml | 29 +++ 32 files changed, 1401 insertions(+), 331 deletions(-) create mode 100644 polkadot/node/malus/container/Containerfile-cargo-chef create mode 100755 polkadot/node/malus/container/build.sh create mode 100644 polkadot/node/malus/container/malus-local-build.Containerfile create mode 100644 polkadot/node/malus/container/polkadot-local-build.Containerfile create mode 100644 polkadot/node/malus/integrationtests/0001-dispute-valid-block.feature create mode 100644 polkadot/node/malus/integrationtests/0001-dispute-valid-block.toml rename polkadot/node/malus/src/{lib.rs => interceptor.rs} (99%) create mode 100644 polkadot/node/malus/src/malus.rs create mode 100644 polkadot/node/malus/src/shared.rs delete mode 100644 polkadot/node/malus/src/variant-a.rs create mode 100644 polkadot/node/malus/src/variants/back_garbage_candidate.rs create mode 100644 polkadot/node/malus/src/variants/dispute_valid_candidates.rs create mode 100644 polkadot/node/malus/src/variants/mod.rs create mode 100644 polkadot/node/malus/src/variants/suggest_garbage_candidate.rs create mode 100644 polkadot/scripts/dockerfiles/malus.Dockerfile delete mode 100644 polkadot/simnet_tests/README.md delete mode 100644 polkadot/simnet_tests/parachains/configs/adder.json delete mode 100644 polkadot/simnet_tests/parachains/configs/simple_rococo_testnet.toml delete mode 100644 polkadot/simnet_tests/parachains/test_suit_description.toml delete mode 100644 polkadot/simnet_tests/parachains/tests/parachains.feature create mode 100644 polkadot/zombienet_tests/README.md create mode 100644 polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.feature create mode 100644 polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.toml diff --git a/polkadot/.gitlab-ci.yml b/polkadot/.gitlab-ci.yml index 6e1a7d6152..ff8a4ef34b 100644 --- a/polkadot/.gitlab-ci.yml +++ b/polkadot/.gitlab-ci.yml @@ -27,10 +27,10 @@ variables: CI_IMAGE: "paritytech/ci-linux:production" DOCKER_OS: "debian:stretch" ARCH: "x86_64" + ZOMBIENET_IMAGE: "docker.io/paritypr/zombienet" VAULT_SERVER_URL: "https://vault.parity-mgmt-vault.parity.io" VAULT_AUTH_PATH: "gitlab-parity-io-jwt" VAULT_AUTH_ROLE: "cicd_gitlab_parity_${CI_PROJECT_NAME}" - SIMNET_IMAGE: "europe-west3-docker.pkg.dev/parity-simnet/simnet-images/simnet:v14" PIPELINE_SCRIPTS_TAG: "v0.1" default: @@ -191,6 +191,7 @@ test-build-linux-stable: script: - ./scripts/gitlab/test_linux_stable.sh # we're using the bin built here, instead of having a parallel `build-linux-release` + # disputes feature is needed for zombie-net parachains malus test - time cargo build --release --verbose --bin polkadot --features "disputes" - sccache -s # pack artifacts @@ -259,6 +260,26 @@ build-adder-collator: - echo "adder-collator version = $(cat ./artifacts/VERSION) (EXTRATAG = $(cat ./artifacts/EXTRATAG))" - cp -r scripts/* ./artifacts +build-malus: + stage: test + <<: *collect-artifacts + <<: *docker-env + <<: *compiler-info + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" + - if: $CI_COMMIT_REF_NAME == "master" + - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs + script: + - time cargo build --release --verbose -p polkadot-test-malus --features disputes + - sccache -s + # pack artifacts + - mkdir -p ./artifacts + - mv ./target/release/malus ./artifacts/. + - echo -n "${CI_COMMIT_REF_NAME}" > ./artifacts/VERSION + - echo -n "${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}" > ./artifacts/EXTRATAG + - echo "polkadot-test-malus = $(cat ./artifacts/VERSION) (EXTRATAG = $(cat ./artifacts/EXTRATAG))" + - cp -r scripts/* ./artifacts + #### stage: build .check-dependent-project: &check-dependent-project @@ -393,13 +414,13 @@ publish-polkadot-image: - job: test-build-linux-stable artifacts: true after_script: - # pass artifacts to the simnet-tests job + # pass artifacts to the zombienet-tests job # https://docs.gitlab.com/ee/ci/multi_project_pipelines.html#with-variable-inheritance - echo "PARACHAINS_IMAGE_NAME=${IMAGE_NAME}" > ./artifacts/parachains.env - echo "PARACHAINS_IMAGE_TAG=$(cat ./artifacts/EXTRATAG)" >> ./artifacts/parachains.env artifacts: reports: - # this artifact is used in simnet-tests job + # this artifact is used in zombienet-tests job dotenv: ./artifacts/parachains.env publish-adder-collator-image: @@ -417,14 +438,40 @@ publish-adder-collator-image: artifacts: true after_script: - buildah logout --all - # pass artifacts to the simnet-tests job + # pass artifacts to the zombienet-tests job - echo "COLLATOR_IMAGE_NAME=${IMAGE_NAME}" > ./artifacts/collator.env - echo "COLLATOR_IMAGE_TAG=$(cat ./artifacts/EXTRATAG)" >> ./artifacts/collator.env artifacts: reports: - # this artifact is used in simnet-tests job + # this artifact is used in zombienet-tests job dotenv: ./artifacts/collator.env +publish-malus-image: + # service image for Simnet + stage: build + <<: *build-push-image + variables: + <<: *image-variables + # scripts/dockerfiles/malus.Dockerfile + DOCKERFILE: dockerfiles/malus.Dockerfile + IMAGE_NAME: docker.io/paritypr/malus + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" + - if: $CI_COMMIT_REF_NAME == "master" + - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs + needs: + - job: build-malus + artifacts: true + after_script: + - buildah logout "$IMAGE_NAME" + # pass artifacts to the zombienet-tests job + - echo "MALUS_IMAGE_NAME=${IMAGE_NAME}" > ./artifacts/malus.env + - echo "MALUS_IMAGE_TAG=$(cat ./artifacts/EXTRATAG)" >> ./artifacts/malus.env + artifacts: + reports: + # this artifact is used in zombienet-tests job + dotenv: ./artifacts/malus.env + update_polkadot_weights: &update-weights stage: build when: manual @@ -550,38 +597,80 @@ deploy-polkasync-kusama: allow_failure: true trigger: "parity/infrastructure/parity-testnet" -simnet-tests: +zombienet-tests-parachains-smoke-test: stage: deploy - image: "${SIMNET_IMAGE}" + image: "${ZOMBIENET_IMAGE}" <<: *kubernetes-env - <<: *rules-test-and-rococo - variables: - GH_DIR: "https://github.com/paritytech/polkadot/tree/master/simnet_tests" + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" + - if: $CI_COMMIT_REF_NAME == "master" + - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs + - if: $CI_COMMIT_REF_NAME == "rococo-v1" needs: - job: publish-polkadot-image + - job: publish-malus-image - job: publish-adder-collator-image - # `parachains.env` brings here `$PARACHAINS_IMAGE_NAME` and `$PARACHAINS_IMAGE_TAG` - # (`$EXTRATAG` here, # i.e. `2643-0.8.29-5f689e0a-6b24dc54`). - # `collator.env` brings here `$COLLATOR_IMAGE_NAME` and `$COLLATOR_IMAGE_TAG` - # For local tests with docker `$PARACHAINS_IMAGE_TAG` and `$COLLATOR_IMAGE_TAG` - # can be replaced with `master` tag. - # SIMNET_REF is a gitlab variable + variables: + GH_DIR: 'https://github.com/paritytech/polkadot/tree/bernhard-malus-fx-zombienet/zombienet_tests/parachains' + # FIXME: use the master after the merge of the malus pr + # GH_DIR: 'https://github.com/paritytech/polkadot/tree/master/zombienet_tests/parachains' + before_script: - - echo "Simnet Tests Config" - - echo "${SIMNET_IMAGE}" + - echo "Zombie-net Tests Config" + - echo "${ZOMBIENET_IMAGE_NAME}" - echo "${PARACHAINS_IMAGE_NAME} ${PARACHAINS_IMAGE_TAG}" - - echo "${COLLATOR_IMAGE_NAME} ${COLLATOR_IMAGE_TAG}" + - echo "${MALUS_IMAGE_NAME} ${MALUS_IMAGE_TAG}" - echo "${GH_DIR}" + - export DEBUG=zombie,zombie::network-node + - export ZOMBIENET_INTEGRATION_TEST_IMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG} + - export MALUS_IMAGE=${MALUS_IMAGE_NAME}:${MALUS_IMAGE_TAG} + - export COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG} + script: - - /home/nonroot/simnet/scripts/run-test-environment-manager-v2.sh + - /home/nonroot/zombie-net/scripts/run-test-env-manager.sh --github-remote-dir="${GH_DIR}" - --tag=smoketest - --image="PARACHAINSIMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}" - --image="SYNTHIMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}" - --image="COLIMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}" - --image="SCRIPTSIMAGE=${SIMNET_IMAGE}" + --test="0001-parachains-smoke-test.feature" allow_failure: true retry: 2 tags: - - polkadot-simnet + - zombienet-polkadot-integration-test + +zombienet-tests-malus-dispute-valid: + stage: deploy + image: "${ZOMBIENET_IMAGE}" + <<: *kubernetes-env + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" + - if: $CI_COMMIT_REF_NAME == "master" + - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs + - if: $CI_COMMIT_REF_NAME == "rococo-v1" + needs: + - job: publish-polkadot-image + - job: publish-malus-image + - job: publish-adder-collator-image + + variables: + GH_DIR: 'https://github.com/paritytech/polkadot/tree/bernhard-malus-fx-zombienet/node/malus/integrationtests' + # FIXME: use the master after the merge of the malus pr + # GH_DIR: 'https://github.com/paritytech/polkadot/tree/master/node/malus/integrationtests' + + before_script: + - echo "Zombie-net Tests Config" + - echo "${ZOMBIENET_IMAGE_NAME}" + - echo "${PARACHAINS_IMAGE_NAME} ${PARACHAINS_IMAGE_TAG}" + - echo "${MALUS_IMAGE_NAME} ${MALUS_IMAGE_TAG}" + - echo "${GH_DIR}" + - export DEBUG=zombie,zombie::network-node + - export ZOMBIENET_INTEGRATION_TEST_IMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG} + - export MALUS_IMAGE=${MALUS_IMAGE_NAME}:${MALUS_IMAGE_TAG} + - export COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG} + + script: + - /home/nonroot/zombie-net/scripts/run-test-env-manager.sh + --github-remote-dir="${GH_DIR}" + --test="0001-dispute-valid-block.feature" + allow_failure: true + retry: 2 + tags: + - zombienet-polkadot-integration-test diff --git a/polkadot/Cargo.lock b/polkadot/Cargo.lock index bba5e4d96c..b037101bfc 100644 --- a/polkadot/Cargo.lock +++ b/polkadot/Cargo.lock @@ -7127,15 +7127,23 @@ dependencies = [ "async-trait", "color-eyre", "futures 0.3.17", + "futures-timer 3.0.2", "parity-util-mem", "polkadot-cli", + "polkadot-node-core-backing", "polkadot-node-core-candidate-validation", + "polkadot-node-core-dispute-coordinator", "polkadot-node-core-pvf", + "polkadot-node-primitives", "polkadot-node-subsystem", "polkadot-node-subsystem-test-helpers", + "polkadot-node-subsystem-types", "polkadot-node-subsystem-util", + "polkadot-primitives", "sp-core", + "sp-keystore", "structopt", + "tracing", ] [[package]] diff --git a/polkadot/cli/src/command.rs b/polkadot/cli/src/command.rs index cf40efa34c..a862840cad 100644 --- a/polkadot/cli/src/command.rs +++ b/polkadot/cli/src/command.rs @@ -219,8 +219,8 @@ fn ensure_dev(spec: &Box) -> std::result::Result<(), Str /// accepts an alternative overseer generator, to adjust behavior /// for integration tests as needed. #[cfg(feature = "malus")] -pub fn run_node(cli: Cli, overseer_gen: impl service::OverseerGen) -> Result<()> { - run_node_inner(cli, overseer_gen) +pub fn run_node(run: Cli, overseer_gen: impl service::OverseerGen) -> Result<()> { + run_node_inner(run, overseer_gen) } fn run_node_inner(cli: Cli, overseer_gen: impl service::OverseerGen) -> Result<()> { diff --git a/polkadot/node/malus/Cargo.toml b/polkadot/node/malus/Cargo.toml index e90ff999ad..b572f23af4 100644 --- a/polkadot/node/malus/Cargo.toml +++ b/polkadot/node/malus/Cargo.toml @@ -1,11 +1,3 @@ -[lib] -name = "malus" -path = "src/lib.rs" - -[[bin]] -name = "malus-variant-a" -path = "src/variant-a.rs" - [package] name = "polkadot-test-malus" description = "Misbehaving nodes for local testnets, system and Simnet tests." @@ -16,17 +8,34 @@ edition = "2018" readme = "README.md" publish = false +[[bin]] +name = "malus" +path = "src/malus.rs" + [dependencies] polkadot-cli = { path = "../../cli", default-features = false, features = [ "cli", "malus" ] } polkadot-node-subsystem = { path = "../subsystem" } polkadot-node-subsystem-util = { path = "../subsystem-util" } +polkadot-node-subsystem-types = { path = "../subsystem-types" } +polkadot-node-core-dispute-coordinator = { path = "../core/dispute-coordinator" } polkadot-node-core-candidate-validation = { path = "../core/candidate-validation" } +polkadot-node-core-backing = { path = "../core/backing" } +polkadot-node-primitives = { path = "../primitives" } +polkadot-primitives = { path = "../../primitives" } polkadot-node-core-pvf = { path = "../core/pvf" } parity-util-mem = { version = "0.10.0", default-features = false, features = ["jemalloc-global"] } color-eyre = { version = "0.5.11", default-features = false } assert_matches = "1.5" structopt = "0.3.25" async-trait = "0.1.51" +sp-keystore = { git = "https://github.com/paritytech/substrate", branch = "master" } +futures = "0.3.16" +futures-timer = "3.0.2" +tracing = "0.1.26" + +[features] +default = [] # we do not enable disputes by default to avoid feature leak +disputes = ["polkadot-cli/disputes"] [dev-dependencies] polkadot-node-subsystem-test-helpers = { path = "../subsystem-test-helpers" } diff --git a/polkadot/node/malus/README.md b/polkadot/node/malus/README.md index a52e19bfef..4f71016d77 100644 --- a/polkadot/node/malus/README.md +++ b/polkadot/node/malus/README.md @@ -1,3 +1,59 @@ # malus -Create nemesis nodes with alternate, at best fault, at worst intentionally destructive behavior traits. \ No newline at end of file +Create nemesis nodes with alternate, at best faulty, at worst intentionally destructive behavior traits. + +The first argument determines the behavior strain. The currently supported are: + +* `suggest-garbage-candidate` +* `back-garbage-candidate` +* `dispute-ancestor` + +## Integration test cases + +To define integration tests create file +in the toml format as used with [zombienet][zombienet] +under `./integrationtests` describing the network to spawn and +also the `feature` file (with `.feature` extension ) using the format +defined in the [DSL doc](https://github.com/paritytech/zombienet/blob/main/docs/testing-dsl-definition.md). + +## Usage + +> Assumes you already gained permissiones, ping in element @javier:matrix.parity.io to get access. +> and you have cloned the [zombienet][zombienet] repo. + +To launch a test case in the development cluster use (e.g. for the ./node/malus/integrationtests/0001-dispute-valid-block.toml): + +```sh +# declare the containers pulled in by zombie-net test definitions +export MALUS_IMAGE=docker.io/paritypr/malus:4131-ccd09bbf +export ZOMBIENET_INTEGRATION_TEST_IMAGE=docker.io/paritypr/synth-wave:4131-0.9.12-ccd09bbf-29a1ac18 +export COL_IMAGE=docker.io/paritypr/colander:4131-ccd09bbf + +# login chore, once, with the values as provided in the above guide +gcloud auth login +gcloud config set project "parity-zombienet" +gcloud container clusters get-credentials "parity-zombienet" --zone "europe-west3-b" --project parity-zombienet + +# launching the actual test +cd zombienet +npm run build +node dist/cli.js test /node/malus/integrationtests/0001-dispute-valid-block.feature + +# Access logs (in google cloud storage) +gsutil ls gs://zombienet-logs/zombie-/logs/ +``` + +This will also teardown the namespace after completion. + +## Container Image Building Note + +In order to build the container image you need to have the latest changes from +polkadot and substrate master branches. + +```sh +pwd # run this from the current dir +podman build -t paritypr/malus:v1 -f Containerfile ../../.. +``` + +[zombienet]: https://github.com/paritytech/zombienet +[gke]: (https://github.com/paritytech/gurke/blob/main/docs/How-to-setup-access-to-gke-k8s-cluster.md) diff --git a/polkadot/node/malus/container/Containerfile-cargo-chef b/polkadot/node/malus/container/Containerfile-cargo-chef new file mode 100644 index 0000000000..65fd1d1ae2 --- /dev/null +++ b/polkadot/node/malus/container/Containerfile-cargo-chef @@ -0,0 +1,155 @@ +# +### Builder stage +# + +FROM rust as builder + +WORKDIR /usr/src/polkadot-malus +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + ca-certificates \ + clang \ + curl \ + cmake \ + libssl1.1 \ + libssl-dev \ + pkg-config + +RUN export PATH="$PATH:$HOME/.cargo/bin" && \ + rustup toolchain install nightly && \ + rustup target add wasm32-unknown-unknown --toolchain nightly && \ + rustup default stable + +COPY polkadot/ /usr/src/polkadot-malus/polkadot/ +COPY substrate/ /usr/src/polkadot-malus/substrate/ + +WORKDIR /usr/src/polkadot-malus/polkadot + +RUN cargo build -p polkadot-test-malus --release +RUN cp -v /usr/src/polkadot-malus/polkadot/target/release/malus /usr/local/bin + +# check if executable works in this container +RUN /usr/local/bin/malus $VARIANT --version + +# +### Runtime +# + +FROM debian:buster-slim as runtime +RUN apt-get update && \ + apt-get install -y curl tini + +COPY --from=builder /usr/src/polkadot-malus/polkadot/target/release/malus /usr/local/bin +# Non-root user for security purposes. +# +# UIDs below 10,000 are a security risk, as a container breakout could result +# in the container being ran as a more privileged user on the host kernel with +# the same UID. +# +# Static GID/UID is also useful for chown'ing files outside the container where +# such a user does not exist. +RUN groupadd --gid 10001 nonroot && \ + useradd --home-dir /home/nonroot \ + --create-home \ + --shell /bin/bash \ + --gid nonroot \ + --groups nonroot \ + --uid 10000 nonroot +WORKDIR /home/nonroot/polkadot-malus + +RUN chown -R nonroot. /home/nonroot + +# Use the non-root user to run our application +# Tell run test script that it runs in container +USER nonroot +# check if executable works in this container +RUN /usr/local/bin/malus --version +# Tini allows us to avoid several Docker edge cases, see https://github.com/krallin/tini. +ENTRYPOINT ["tini", "--", "/usr/local/bin/malus"] + + + + +FROM rust:1.54.0 as planner +WORKDIR /usr/src/polkadot-malus +# We only pay the installation cost once, +# it will be cached from the second build onwards +RUN cargo install cargo-chef +COPY polkadot/ /usr/src/polkadot-malus/polkadot/ +COPY substrate/ /usr/src/polkadot-malus/substrate/ +WORKDIR /usr/src/polkadot-malus/polkadot +RUN cargo chef prepare --recipe-path recipe.json + + +FROM rust:1.54.0 as cacher +WORKDIR /usr/src/polkadot-malus/polkadot +RUN cargo install cargo-chef +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + ca-certificates \ + clang \ + curl \ + cmake \ + libssl1.1 \ + libssl-dev \ + pkg-config +RUN export PATH="$PATH:$HOME/.cargo/bin" && \ + rustup toolchain install nightly && \ + rustup target add wasm32-unknown-unknown --toolchain nightly && \ + rustup default stable +COPY --from=planner /usr/src/polkadot-malus/polkadot/recipe.json recipe.json +RUN cargo chef cook --release --recipe-path recipe.json + + +FROM rust:1.54.0 as builder +WORKDIR /usr/src/polkadot-malus +COPY polkadot/ /usr/src/polkadot-malus/polkadot/ +COPY substrate/ /usr/src/polkadot-malus/substrate/ +# Copy over the cached dependencies +WORKDIR /usr/src/polkadot-malus/polkadot +COPY --from=cacher /usr/src/polkadot-malus/polkadot/target target +COPY --from=cacher $CARGO_HOME $CARGO_HOME +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + ca-certificates \ + clang \ + curl \ + cmake \ + libssl1.1 \ + libssl-dev \ + pkg-config +RUN export PATH="$PATH:$HOME/.cargo/bin" && \ + rustup toolchain install nightly && \ + rustup target add wasm32-unknown-unknown --toolchain nightly && \ + rustup default stable +RUN cargo build -p polkadot-test-malus --release + + +FROM debian:buster-slim as runtime +RUN apt-get update && \ + apt-get install -y curl tini +COPY --from=builder /usr/src/polkadot-malus/polkadot/target/release/malus /usr/local/bin +# Non-root user for security purposes. +# +# UIDs below 10,000 are a security risk, as a container breakout could result +# in the container being ran as a more privileged user on the host kernel with +# the same UID. +# +# Static GID/UID is also useful for chown'ing files outside the container where +# such a user does not exist. +RUN groupadd --gid 10001 nonroot && \ + useradd --home-dir /home/nonroot \ + --create-home \ + --shell /bin/bash \ + --gid nonroot \ + --groups nonroot \ + --uid 10000 nonroot +WORKDIR /home/nonroot/polkadot-malus +RUN chown -R nonroot. /home/nonroot +# Use the non-root user to run our application +# Tell run test script that it runs in container +USER nonroot +# check if executable works in this container +RUN /usr/local/bin/malus --version +# Tini allows us to avoid several Docker edge cases, see https://github.com/krallin/tini. +ENTRYPOINT ["/usr/local/bin/malus"] diff --git a/polkadot/node/malus/container/build.sh b/polkadot/node/malus/container/build.sh new file mode 100755 index 0000000000..a277ac5dda --- /dev/null +++ b/polkadot/node/malus/container/build.sh @@ -0,0 +1 @@ +podman build -t paritypr/malus:v1 -f Containerfile ../../../.. diff --git a/polkadot/node/malus/container/malus-local-build.Containerfile b/polkadot/node/malus/container/malus-local-build.Containerfile new file mode 100644 index 0000000000..ee91e98b23 --- /dev/null +++ b/polkadot/node/malus/container/malus-local-build.Containerfile @@ -0,0 +1,66 @@ +# +### Builder stage +# + +FROM rust as builder + +WORKDIR /usr/src/polkadot-malus +COPY polkadot/ /usr/src/polkadot-malus/polkadot/ +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + ca-certificates \ + clang \ + curl \ + cmake \ + libssl1.1 \ + libssl-dev \ + pkg-config + +RUN export PATH="$PATH:$HOME/.cargo/bin" && \ + rustup toolchain install nightly && \ + rustup target add wasm32-unknown-unknown --toolchain nightly && \ + rustup default stable + + +WORKDIR /usr/src/polkadot-malus/polkadot + +RUN cargo build -p polkadot-test-malus --release --verbose +RUN cp -v /usr/src/polkadot-malus/polkadot/target/release/malus /usr/local/bin + +# check if executable works in this container +RUN /usr/local/bin/malus --version + +# +### Runtime +# + +FROM debian:buster-slim as runtime +RUN apt-get update && \ + apt-get install -y curl tini + +COPY --from=builder /usr/src/polkadot-malus/polkadot/target/release/malus /usr/local/bin +# Non-root user for security purposes. +# +# UIDs below 10,000 are a security risk, as a container breakout could result +# in the container being ran as a more privileged user on the host kernel with +# the same UID. +# +# Static GID/UID is also useful for chown'ing files outside the container where +# such a user does not exist. +RUN groupadd --gid 10001 nonroot && \ + useradd --home-dir /home/nonroot \ + --create-home \ + --shell /bin/bash \ + --gid nonroot \ + --groups nonroot \ + --uid 10000 nonroot +WORKDIR /home/nonroot/polkadot-malus + +RUN chown -R nonroot. /home/nonroot + +# Use the non-root user to run our application +USER nonroot +# check if executable works in this container +RUN /usr/local/bin/malus --version +# Tini allows us to avoid several Docker edge cases, see https://github.com/krallin/tini. +ENTRYPOINT ["tini", "--", "/usr/local/bin/malus"] diff --git a/polkadot/node/malus/container/polkadot-local-build.Containerfile b/polkadot/node/malus/container/polkadot-local-build.Containerfile new file mode 100644 index 0000000000..7a7e26a03c --- /dev/null +++ b/polkadot/node/malus/container/polkadot-local-build.Containerfile @@ -0,0 +1,66 @@ +# +### Builder stage +# + +FROM rust as builder + +WORKDIR /usr/src/polkadot +COPY polkadot/ /usr/src/polkadot +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + ca-certificates \ + clang \ + curl \ + cmake \ + libssl1.1 \ + libssl-dev \ + pkg-config + +RUN export PATH="$PATH:$HOME/.cargo/bin" && \ + rustup toolchain install nightly && \ + rustup target add wasm32-unknown-unknown --toolchain nightly && \ + rustup default stable + + +WORKDIR /usr/src/polkadot + +RUN cargo build --release --bin polkadot --features disputes --verbose +RUN cp -v /usr/src/polkadot/target/release/polkadot /usr/local/bin + +# check if executable works in this container +RUN /usr/local/bin/polkadot --version + +# +### Runtime +# + +FROM debian:buster-slim as runtime +RUN apt-get update && \ + apt-get install -y curl tini + +COPY --from=builder /usr/src/polkadot/target/release/polkadot /usr/local/bin +# Non-root user for security purposes. +# +# UIDs below 10,000 are a security risk, as a container breakout could result +# in the container being ran as a more privileged user on the host kernel with +# the same UID. +# +# Static GID/UID is also useful for chown'ing files outside the container where +# such a user does not exist. +RUN groupadd --gid 10001 nonroot && \ + useradd --home-dir /home/nonroot \ + --create-home \ + --shell /bin/bash \ + --gid nonroot \ + --groups nonroot \ + --uid 10000 nonroot +WORKDIR /home/nonroot/polkadot + +RUN chown -R nonroot. /home/nonroot + +# Use the non-root user to run our application +USER nonroot +# check if executable works in this container +RUN /usr/local/bin/polkadot --version +# Tini allows us to avoid several Docker edge cases, see https://github.com/krallin/tini. +ENTRYPOINT ["tini", "--", "/usr/local/bin/polkadot"] diff --git a/polkadot/node/malus/integrationtests/0001-dispute-valid-block.feature b/polkadot/node/malus/integrationtests/0001-dispute-valid-block.feature new file mode 100644 index 0000000000..bb548cdc63 --- /dev/null +++ b/polkadot/node/malus/integrationtests/0001-dispute-valid-block.feature @@ -0,0 +1,29 @@ +Description: Disputes +Network: ./0001-dispute-valid-block.toml +Creds: config.gcloud + + +alice: is up +bob: is up +charlie: is up +david is up +alice: reports node_roles is 4 +bob: reports node_roles is 4 +alice: reports sub_libp2p_is_major_syncing is 0 +alice: reports block height is at least 2 within 15 seconds +alice: reports peers count is at least 2 +bob: reports block height is at least 2 +bob: reports peers count is at least 2 +charlie: reports block height is at least 2 +charlie: reports peers count is at least 2 +alice: reports parachain_candidate_disputes_total is at least 1 within 250 seconds +bob: reports parachain_candidate_disputes_total is at least 1 within 45 seconds +charlie: reports parachain_candidate_disputes_total is at least 1 within 45 seconds +alice: reports parachain_candidate_dispute_votes{validity="valid"} is at least 1 within 45 seconds +bob: reports parachain_candidate_dispute_votes{validity="valid"} is at least 2 within 45 seconds +charlie: reports parachain_candidate_dispute_votes{validity="valid"} is at least 2 within 45 seconds +alice: reports parachain_candidate_dispute_concluded{validity="valid"} is at least 1 within 45 seconds +alice: reports parachain_candidate_dispute_concluded{validity="invalid"} is 0 within 45 seconds +bob: reports parachain_candidate_dispute_concluded{validity="valid"} is at least 1 within 45 seconds +charlie: reports parachain_candidate_dispute_concluded{validity="valid"} is at least 1 within 45 seconds +charlie: reports parachain_candidate_dispute_concluded{validity="valid"} is at least 1 within 45 seconds diff --git a/polkadot/node/malus/integrationtests/0001-dispute-valid-block.toml b/polkadot/node/malus/integrationtests/0001-dispute-valid-block.toml new file mode 100644 index 0000000000..a1d711cd78 --- /dev/null +++ b/polkadot/node/malus/integrationtests/0001-dispute-valid-block.toml @@ -0,0 +1,40 @@ +[settings] +timeout = 1000 + +[relaychain] +default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}" +chain = "wococo-local" +chain_spec_command = "polkadot build-spec --chain wococo-local --disable-default-bootnode --raw > /cfg/wococo-local.json" +command = "polkadot" + + [[relaychain.nodes]] + name = "alice" + validator = true + extra_args = [ "--alice", "-lparachain=debug" ] + + [[relaychain.nodes]] + name = "bob" + validator = true + extra_args = [ "--bob", "-lparachain=debug" ] + + [[relaychain.nodes]] + name = "charlie" + validator = true + extra_args = [ "--charlie", "-lparachain=debug" ] + + [[relaychain.nodes]] + name = "david" + validator = true + command = "/usr/local/bin/malus dispute-ancestor" + extra_args = ["--dave", "-lparachain=debug"] + image = "{{MALUS_IMAGE}}" + autoConnectApi = false + +[[parachains]] +id = 100 + + [parachains.collator] + name = "collator01" + image = "{{COL_IMAGE}}" + command = "/usr/local/bin/adder-collator" + args = ["-lparachain=debug"] \ No newline at end of file diff --git a/polkadot/node/malus/src/lib.rs b/polkadot/node/malus/src/interceptor.rs similarity index 99% rename from polkadot/node/malus/src/lib.rs rename to polkadot/node/malus/src/interceptor.rs index abd4c6cfe7..5da91fd42b 100644 --- a/polkadot/node/malus/src/lib.rs +++ b/polkadot/node/malus/src/interceptor.rs @@ -24,9 +24,6 @@ use polkadot_node_subsystem::*; pub use polkadot_node_subsystem::{messages::AllMessages, overseer, FromOverseer}; use std::{future::Future, pin::Pin}; -#[cfg(test)] -mod tests; - /// Filter incoming and outgoing messages. pub trait MessageInterceptor: Send + Sync + Clone + 'static where diff --git a/polkadot/node/malus/src/malus.rs b/polkadot/node/malus/src/malus.rs new file mode 100644 index 0000000000..1f19ca7df0 --- /dev/null +++ b/polkadot/node/malus/src/malus.rs @@ -0,0 +1,126 @@ +// Copyright 2021 Parity Technologies (UK) Ltd. +// This file is part of Polkadot. + +// Polkadot is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Polkadot is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see . + +//! A malus or nemesis node launch code. + +use color_eyre::eyre; +use polkadot_cli::{Cli, RunCmd}; +use structopt::StructOpt; + +pub(crate) mod interceptor; +pub(crate) mod shared; + +mod variants; + +use variants::*; + +/// Define the different variants of behavior. +#[derive(Debug, StructOpt)] +#[structopt(about = "Malus - the nemesis of polkadot.")] +#[structopt(rename_all = "kebab-case")] +enum NemesisVariant { + /// Suggest a candidate with an invalid proof of validity. + SuggestGarbageCandidate(RunCmd), + /// Back a candidate with a specifically crafted proof of validity. + BackGarbageCandidate(RunCmd), + /// Delayed disputing of ancestors that are perfectly fine. + DisputeAncestor(RunCmd), + + #[allow(missing_docs)] + #[structopt(name = "prepare-worker", setting = structopt::clap::AppSettings::Hidden)] + PvfPrepareWorker(polkadot_cli::ValidationWorkerCommand), + + #[allow(missing_docs)] + #[structopt(name = "execute-worker", setting = structopt::clap::AppSettings::Hidden)] + PvfExecuteWorker(polkadot_cli::ValidationWorkerCommand), +} + +#[derive(Debug, StructOpt)] +#[allow(missing_docs)] +struct MalusCli { + #[structopt(subcommand)] + pub variant: NemesisVariant, +} + +fn run_cmd(run: RunCmd) -> Cli { + Cli { subcommand: None, run } +} + +impl MalusCli { + /// Launch a malus node. + fn launch(self) -> eyre::Result<()> { + match self.variant { + NemesisVariant::BackGarbageCandidate(cmd) => + polkadot_cli::run_node(run_cmd(cmd), BackGarbageCandidate)?, + NemesisVariant::SuggestGarbageCandidate(cmd) => + polkadot_cli::run_node(run_cmd(cmd), SuggestGarbageCandidate)?, + NemesisVariant::DisputeAncestor(cmd) => + polkadot_cli::run_node(run_cmd(cmd), DisputeValidCandidates)?, + NemesisVariant::PvfPrepareWorker(cmd) => { + #[cfg(target_os = "android")] + { + return Err("PVF preparation workers are not supported under this platform") + .into() + } + + #[cfg(not(target_os = "android"))] + { + polkadot_node_core_pvf::prepare_worker_entrypoint(&cmd.socket_path); + } + }, + NemesisVariant::PvfExecuteWorker(cmd) => { + #[cfg(target_os = "android")] + { + return Err("PVF execution workers are not supported under this platform").into() + } + + #[cfg(not(target_os = "android"))] + { + polkadot_node_core_pvf::execute_worker_entrypoint(&cmd.socket_path); + } + }, + } + Ok(()) + } +} + +fn main() -> eyre::Result<()> { + color_eyre::install()?; + let cli = MalusCli::from_args(); + cli.launch()?; + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn subcommand_works() { + let cli = MalusCli::from_iter_safe(IntoIterator::into_iter([ + "malus", + "dispute-ancestor", + "--bob", + ])) + .unwrap(); + assert_matches::assert_matches!(cli, MalusCli { + variant: NemesisVariant::DisputeAncestor(run), + .. + } => { + assert!(run.base.bob); + }); + } +} diff --git a/polkadot/node/malus/src/shared.rs b/polkadot/node/malus/src/shared.rs new file mode 100644 index 0000000000..3c1d55d0d8 --- /dev/null +++ b/polkadot/node/malus/src/shared.rs @@ -0,0 +1,49 @@ +// Copyright 2021 Parity Technologies (UK) Ltd. +// This file is part of Polkadot. + +// Polkadot is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Polkadot is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see . + +use futures::prelude::*; +use polkadot_node_primitives::SpawnNamed; + +pub const MALUS: &str = "MALUS😈😈😈"; + +#[allow(unused)] +pub(crate) const MALICIOUS_POV: &[u8] = "😈😈pov_looks_valid_to_me😈😈".as_bytes(); + +/// Launch a service task for each item in the provided queue. +#[allow(unused)] +pub(crate) fn launch_processing_task(spawner: &S, queue: Q, action: F) +where + F: Fn(X) -> U + Send + 'static, + U: Future + Send + 'static, + Q: Stream + Send + 'static, + X: Send, + S: 'static + SpawnNamed + Clone + Unpin, +{ + let spawner2: S = spawner.clone(); + spawner.spawn( + "nemesis-queue-processor", + Some("malus"), + Box::pin(async move { + let spawner3 = spawner2.clone(); + queue + .for_each(move |input| { + spawner3.spawn("nemesis-task", Some("malus"), Box::pin(action(input))); + async move { () } + }) + .await; + }), + ); +} diff --git a/polkadot/node/malus/src/variant-a.rs b/polkadot/node/malus/src/variant-a.rs deleted file mode 100644 index ed395a9d88..0000000000 --- a/polkadot/node/malus/src/variant-a.rs +++ /dev/null @@ -1,121 +0,0 @@ -// Copyright 2017-2020 Parity Technologies (UK) Ltd. -// This file is part of Polkadot. - -// Polkadot is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// Polkadot is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with Polkadot. If not, see . - -//! A malicious overseer. -//! -//! An example on how to use the `OverseerGen` pattern to -//! instantiate a modified subsystem implementation -//! for usage with `simnet`/Gurke. - -#![allow(missing_docs)] - -use color_eyre::eyre; -use polkadot_cli::{ - prepared_overseer_builder, - service::{ - AuthorityDiscoveryApi, AuxStore, BabeApi, Block, Error, HeaderBackend, OverseerGen, - OverseerGenArgs, ParachainHost, ProvideRuntimeApi, SpawnNamed, - }, - Cli, -}; - -// Import extra types relevant to the particular -// subsystem. -use polkadot_node_core_candidate_validation::CandidateValidationSubsystem; -use polkadot_node_subsystem::{ - messages::{AllMessages, CandidateValidationMessage}, - overseer::{self, Overseer, OverseerConnector, OverseerHandle}, - FromOverseer, -}; - -use malus::*; - -// Filter wrapping related types. -use std::sync::{ - atomic::{AtomicUsize, Ordering}, - Arc, -}; - -use structopt::StructOpt; - -/// Silly example, just drop every second outgoing message. -#[derive(Clone, Default, Debug)] -struct Skippy(Arc); - -impl MessageInterceptor for Skippy -where - Sender: overseer::SubsystemSender - + overseer::SubsystemSender - + Clone - + 'static, -{ - type Message = CandidateValidationMessage; - - fn intercept_incoming( - &self, - _sender: &mut Sender, - msg: FromOverseer, - ) -> Option> { - if self.0.fetch_add(1, Ordering::Relaxed) % 2 == 0 { - Some(msg) - } else { - None - } - } - fn intercept_outgoing(&self, msg: AllMessages) -> Option { - Some(msg) - } -} - -/// Generates an overseer that exposes bad behavior. -struct BehaveMaleficient; - -impl OverseerGen for BehaveMaleficient { - fn generate<'a, Spawner, RuntimeClient>( - &self, - connector: OverseerConnector, - args: OverseerGenArgs<'a, Spawner, RuntimeClient>, - ) -> Result<(Overseer>, OverseerHandle), Error> - where - RuntimeClient: 'static + ProvideRuntimeApi + HeaderBackend + AuxStore, - RuntimeClient::Api: ParachainHost + BabeApi + AuthorityDiscoveryApi, - Spawner: 'static + SpawnNamed + Clone + Unpin, - { - let candidate_validation_config = args.candidate_validation_config.clone(); - - prepared_overseer_builder(args)? - .replace_candidate_validation(|orig: CandidateValidationSubsystem| { - InterceptedSubsystem::new( - CandidateValidationSubsystem::with_config( - candidate_validation_config, - orig.metrics, - orig.pvf_metrics, - ), - Skippy::default(), - ) - }) - .build_with_connector(connector) - .map_err(|e| e.into()) - } -} - -fn main() -> eyre::Result<()> { - color_eyre::install()?; - let cli = Cli::from_args(); - assert_matches::assert_matches!(cli.subcommand, None); - polkadot_cli::run_node(cli, BehaveMaleficient)?; - Ok(()) -} diff --git a/polkadot/node/malus/src/variants/back_garbage_candidate.rs b/polkadot/node/malus/src/variants/back_garbage_candidate.rs new file mode 100644 index 0000000000..06938f2a4f --- /dev/null +++ b/polkadot/node/malus/src/variants/back_garbage_candidate.rs @@ -0,0 +1,230 @@ +// Copyright 2021 Parity Technologies (UK) Ltd. +// This file is part of Polkadot. + +// Polkadot is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Polkadot is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see . + +//! A malicious overseer backing a particular candidate with a +//! malicious proof of validity that is received. + +#![allow(missing_docs)] + +use polkadot_cli::{ + prepared_overseer_builder, + service::{ + AuthorityDiscoveryApi, AuxStore, BabeApi, Block, Error, HeaderBackend, Overseer, + OverseerConnector, OverseerGen, OverseerGenArgs, OverseerHandle, ParachainHost, + ProvideRuntimeApi, SpawnNamed, + }, +}; + +// Import extra types relevant to the particular +// subsystem. +use polkadot_node_core_candidate_validation::CandidateValidationSubsystem; +use polkadot_node_subsystem::messages::{ + AvailabilityRecoveryMessage, CandidateValidationMessage, ValidationFailed, +}; +use polkadot_node_subsystem_util as util; + +// Filter wrapping related types. +use crate::{interceptor::*, shared::*}; +use polkadot_node_primitives::{PoV, ValidationResult}; + +use polkadot_primitives::v1::{ + CandidateCommitments, CandidateDescriptor, CandidateReceipt, PersistedValidationData, + ValidationCode, +}; + +use futures::channel::oneshot; +use std::{ + collections::HashMap, + sync::{Arc, Mutex}, +}; + +#[derive(Clone, Debug)] +struct BribedPassageInner { + spawner: Spawner, + cache: HashMap, +} + +#[derive(Clone, Debug)] +struct BribedPassage { + inner: Arc>>, +} + +impl BribedPassage +where + Spawner: SpawnNamed, +{ + fn let_pass( + persisted_validation_data: PersistedValidationData, + validation_code: Option, + _candidate_descriptor: CandidateDescriptor, + _pov: Arc, + response_sender: oneshot::Sender>, + ) { + let candidate_commitmentments = CandidateCommitments { + head_data: persisted_validation_data.parent_head.clone(), + new_validation_code: validation_code, + ..Default::default() + }; + + response_sender + .send(Ok(ValidationResult::Valid(candidate_commitmentments, persisted_validation_data))) + .unwrap(); + } +} + +impl MessageInterceptor for BribedPassage +where + Sender: overseer::SubsystemSender + + overseer::SubsystemSender + + Clone + + Send + + 'static, + Spawner: SpawnNamed + Send + Clone + 'static, +{ + type Message = CandidateValidationMessage; + + fn intercept_incoming( + &self, + sender: &mut Sender, + msg: FromOverseer, + ) -> Option> { + match msg { + FromOverseer::Communication { + msg: + CandidateValidationMessage::ValidateFromExhaustive( + persisted_validation_data, + validation_code, + candidate_descriptor, + pov, + _duration, + response_sender, + ), + } if pov.block_data.0.as_slice() == MALICIOUS_POV => { + Self::let_pass( + persisted_validation_data, + Some(validation_code), + candidate_descriptor, + pov, + response_sender, + ); + None + }, + FromOverseer::Communication { + msg: + CandidateValidationMessage::ValidateFromChainState( + candidate_descriptor, + pov, + _duration, + response_sender, + ), + } if pov.block_data.0.as_slice() == MALICIOUS_POV => { + if let Some(candidate_receipt) = + self.inner.lock().unwrap().cache.get(&candidate_descriptor).cloned() + { + let mut subsystem_sender = sender.clone(); + let spawner = self.inner.lock().unwrap().spawner.clone(); + spawner.spawn( + "malus-back-garbage-adhoc", + Some("malus"), + Box::pin(async move { + let relay_parent = candidate_descriptor.relay_parent; + let session_index = util::request_session_index_for_child( + relay_parent, + &mut subsystem_sender, + ) + .await; + let session_index = session_index.await.unwrap().unwrap(); + + let (a_tx, a_rx) = oneshot::channel(); + + subsystem_sender + .send_message(AllMessages::from( + AvailabilityRecoveryMessage::RecoverAvailableData( + candidate_receipt, + session_index, + None, + a_tx, + ), + )) + .await; + + if let Ok(Ok(availability_data)) = a_rx.await { + Self::let_pass( + availability_data.validation_data, + None, + candidate_descriptor, + pov, + response_sender, + ); + } else { + tracing::info!( + target = MALUS, + "Could not get availability data, can't back" + ); + } + }), + ); + } else { + tracing::info!(target = MALUS, "No CandidateReceipt available to work with"); + } + None + }, + msg => Some(msg), + } + } + + fn intercept_outgoing(&self, msg: AllMessages) -> Option { + Some(msg) + } +} + +/// Generates an overseer that exposes bad behavior. +pub(crate) struct BackGarbageCandidate; + +impl OverseerGen for BackGarbageCandidate { + fn generate<'a, Spawner, RuntimeClient>( + &self, + connector: OverseerConnector, + args: OverseerGenArgs<'a, Spawner, RuntimeClient>, + ) -> Result<(Overseer>, OverseerHandle), Error> + where + RuntimeClient: 'static + ProvideRuntimeApi + HeaderBackend + AuxStore, + RuntimeClient::Api: ParachainHost + BabeApi + AuthorityDiscoveryApi, + Spawner: 'static + SpawnNamed + Clone + Unpin, + { + let candidate_validation_config = args.candidate_validation_config.clone(); + let spawner = args.spawner.clone(); + + prepared_overseer_builder(args)? + .replace_candidate_validation(|cv| { + InterceptedSubsystem::new( + CandidateValidationSubsystem::with_config( + candidate_validation_config, + cv.metrics, + cv.pvf_metrics, + ), + BribedPassage:: { + inner: Arc::new(Mutex::new(BribedPassageInner { + spawner, + cache: Default::default(), + })), + }, + ) + }) + .build_with_connector(connector) + .map_err(|e| e.into()) + } +} diff --git a/polkadot/node/malus/src/variants/dispute_valid_candidates.rs b/polkadot/node/malus/src/variants/dispute_valid_candidates.rs new file mode 100644 index 0000000000..aa5626c44b --- /dev/null +++ b/polkadot/node/malus/src/variants/dispute_valid_candidates.rs @@ -0,0 +1,121 @@ +// Copyright 2021 Parity Technologies (UK) Ltd. +// This file is part of Polkadot. + +// Polkadot is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Polkadot is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see . + +//! A malicious node that replaces approvals with invalid disputes +//! against valid candidates. +//! +//! Attention: For usage with `zombienet` only! + +#![allow(missing_docs)] + +use polkadot_cli::{ + prepared_overseer_builder, + service::{ + AuthorityDiscoveryApi, AuxStore, BabeApi, Block, Error, HeaderBackend, Overseer, + OverseerConnector, OverseerGen, OverseerGenArgs, OverseerHandle, ParachainHost, + ProvideRuntimeApi, SpawnNamed, + }, +}; + +// Filter wrapping related types. +use crate::interceptor::*; + +// Import extra types relevant to the particular +// subsystem. +use polkadot_node_core_backing::CandidateBackingSubsystem; +use polkadot_node_subsystem::messages::{ + ApprovalDistributionMessage, CandidateBackingMessage, DisputeCoordinatorMessage, +}; +use sp_keystore::SyncCryptoStorePtr; + +use std::sync::Arc; + +/// Replace outgoing approval messages with disputes. +#[derive(Clone, Debug)] +struct ReplaceApprovalsWithDisputes; + +impl MessageInterceptor for ReplaceApprovalsWithDisputes +where + Sender: overseer::SubsystemSender + Clone + Send + 'static, +{ + type Message = CandidateBackingMessage; + + fn intercept_incoming( + &self, + _sender: &mut Sender, + msg: FromOverseer, + ) -> Option> { + Some(msg) + } + + fn intercept_outgoing(&self, msg: AllMessages) -> Option { + match msg { + AllMessages::ApprovalDistribution(ApprovalDistributionMessage::DistributeApproval( + _, + )) => { + // drop the message on the floor + None + }, + AllMessages::DisputeCoordinator(DisputeCoordinatorMessage::ImportStatements { + candidate_hash, + candidate_receipt, + session, + .. + }) => { + // this would also dispute candidates we were not assigned to approve + Some(AllMessages::DisputeCoordinator( + DisputeCoordinatorMessage::IssueLocalStatement( + session, + candidate_hash, + candidate_receipt, + false, + ), + )) + }, + msg => Some(msg), + } + } +} + +/// Generates an overseer that disputes instead of approving valid candidates. +pub(crate) struct DisputeValidCandidates; + +impl OverseerGen for DisputeValidCandidates { + fn generate<'a, Spawner, RuntimeClient>( + &self, + connector: OverseerConnector, + args: OverseerGenArgs<'a, Spawner, RuntimeClient>, + ) -> Result<(Overseer>, OverseerHandle), Error> + where + RuntimeClient: 'static + ProvideRuntimeApi + HeaderBackend + AuxStore, + RuntimeClient::Api: ParachainHost + BabeApi + AuthorityDiscoveryApi, + Spawner: 'static + SpawnNamed + Clone + Unpin, + { + let spawner = args.spawner.clone(); + let crypto_store_ptr = args.keystore.clone() as SyncCryptoStorePtr; + let filter = ReplaceApprovalsWithDisputes; + + prepared_overseer_builder(args)? + .replace_candidate_backing(move |cb| { + InterceptedSubsystem::new( + CandidateBackingSubsystem::new(spawner, crypto_store_ptr, cb.params.metrics), + filter, + ) + }) + .build_with_connector(connector) + .map_err(|e| e.into()) + } +} diff --git a/polkadot/node/malus/src/variants/mod.rs b/polkadot/node/malus/src/variants/mod.rs new file mode 100644 index 0000000000..aab3203f5b --- /dev/null +++ b/polkadot/node/malus/src/variants/mod.rs @@ -0,0 +1,26 @@ +// Copyright 2021 Parity Technologies (UK) Ltd. +// This file is part of Polkadot. + +// Polkadot is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Polkadot is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see . + +//! Collection of behavior variants. + +mod back_garbage_candidate; +mod dispute_valid_candidates; +mod suggest_garbage_candidate; + +pub(crate) use self::{ + back_garbage_candidate::BackGarbageCandidate, dispute_valid_candidates::DisputeValidCandidates, + suggest_garbage_candidate::SuggestGarbageCandidate, +}; diff --git a/polkadot/node/malus/src/variants/suggest_garbage_candidate.rs b/polkadot/node/malus/src/variants/suggest_garbage_candidate.rs new file mode 100644 index 0000000000..f4e6c0eae5 --- /dev/null +++ b/polkadot/node/malus/src/variants/suggest_garbage_candidate.rs @@ -0,0 +1,172 @@ +// Copyright 2021 Parity Technologies (UK) Ltd. +// This file is part of Polkadot. + +// Polkadot is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. + +// Polkadot is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License +// along with Polkadot. If not, see . + +//! A malicious overseer proposing a garbage block. +//! +//! Supposed to be used with regular nodes or in conjunction +//! with [`malus-back-garbage-candidate.rs`](./malus-back-garbage-candidate.rs) +//! to simulate a coordinated attack. + +#![allow(missing_docs)] + +use polkadot_cli::{ + prepared_overseer_builder, + service::{ + AuthorityDiscoveryApi, AuxStore, BabeApi, Block, Error, HeaderBackend, Overseer, + OverseerConnector, OverseerGen, OverseerGenArgs, OverseerHandle, ParachainHost, + ProvideRuntimeApi, SpawnNamed, + }, +}; + +// Import extra types relevant to the particular +// subsystem. +use polkadot_node_core_backing::CandidateBackingSubsystem; +use polkadot_node_primitives::Statement; +use polkadot_node_subsystem::{ + messages::{CandidateBackingMessage, StatementDistributionMessage}, + overseer::{self, SubsystemSender}, +}; +use polkadot_node_subsystem_util as util; +// Filter wrapping related types. +use crate::interceptor::*; +use polkadot_primitives::v1::{ + CandidateCommitments, CandidateReceipt, CommittedCandidateReceipt, CompactStatement, Hash, + Signed, +}; +use sp_keystore::SyncCryptoStorePtr; +use util::metered; + +use std::sync::Arc; + +use crate::shared::*; + +/// Replaces the seconded PoV data +/// of outgoing messages by some garbage data. +#[derive(Clone)] +struct ReplacePoVBytes +where + Sender: Send, +{ + keystore: SyncCryptoStorePtr, + queue: metered::UnboundedMeteredSender<(Sender, Hash, CandidateReceipt)>, +} + +impl MessageInterceptor for ReplacePoVBytes +where + Sender: overseer::SubsystemSender + Clone + Send + 'static, +{ + type Message = CandidateBackingMessage; + + fn intercept_incoming( + &self, + sender: &mut Sender, + msg: FromOverseer, + ) -> Option> { + match msg { + FromOverseer::Communication { + msg: CandidateBackingMessage::Second(hash, candidate_receipt, _pov), + } => { + self.queue + .unbounded_send((sender.clone(), hash, candidate_receipt.clone())) + .unwrap(); + + None + }, + other => Some(other), + } + } + + fn intercept_outgoing(&self, msg: AllMessages) -> Option { + Some(msg) + } +} + +/// Generates an overseer that exposes bad behavior. +pub(crate) struct SuggestGarbageCandidate; + +impl OverseerGen for SuggestGarbageCandidate { + fn generate<'a, Spawner, RuntimeClient>( + &self, + connector: OverseerConnector, + args: OverseerGenArgs<'a, Spawner, RuntimeClient>, + ) -> Result<(Overseer>, OverseerHandle), Error> + where + RuntimeClient: 'static + ProvideRuntimeApi + HeaderBackend + AuxStore, + RuntimeClient::Api: ParachainHost + BabeApi + AuthorityDiscoveryApi, + Spawner: 'static + SpawnNamed + Clone + Unpin, + { + let spawner = args.spawner.clone(); + let (sink, source) = metered::unbounded(); + let keystore = args.keystore.clone() as SyncCryptoStorePtr; + + let filter = ReplacePoVBytes { keystore: keystore.clone(), queue: sink }; + + let keystore2 = keystore.clone(); + let spawner2 = spawner.clone(); + + let result = prepared_overseer_builder(args)? + .replace_candidate_backing(move |cb| { + InterceptedSubsystem::new( + CandidateBackingSubsystem::new(spawner2, keystore2, cb.params.metrics), + filter, + ) + }) + .build_with_connector(connector) + .map_err(|e| e.into()); + + launch_processing_task( + &spawner, + source, + move |(mut subsystem_sender, hash, candidate_receipt): (_, Hash, CandidateReceipt)| { + let keystore = keystore.clone(); + async move { + tracing::info!( + target = MALUS, + "Replacing seconded candidate pov with something else" + ); + + let committed_candidate_receipt = CommittedCandidateReceipt { + descriptor: candidate_receipt.descriptor.clone(), + commitments: CandidateCommitments::default(), + }; + + let statement = Statement::Seconded(committed_candidate_receipt); + + if let Ok(validator) = + util::Validator::new(hash, keystore.clone(), &mut subsystem_sender).await + { + let signed_statement: Signed = validator + .sign(keystore, statement) + .await + .expect("Signing works. qed") + .expect("Something must come out of this. qed"); + + subsystem_sender + .send_message(StatementDistributionMessage::Share( + hash, + signed_statement, + )) + .await; + } else { + tracing::info!("We are not a validator. Not siging anything."); + } + } + }, + ); + + result + } +} diff --git a/polkadot/node/service/src/lib.rs b/polkadot/node/service/src/lib.rs index 57079947bb..bc7b1f0d29 100644 --- a/polkadot/node/service/src/lib.rs +++ b/polkadot/node/service/src/lib.rs @@ -948,7 +948,11 @@ where chain_selection_config, dispute_coordinator_config, }, - )?; + ) + .map_err(|e| { + tracing::error!("Failed to init overseer: {}", e); + e + })?; let handle = Handle::new(overseer_handle.clone()); { diff --git a/polkadot/node/subsystem-util/src/lib.rs b/polkadot/node/subsystem-util/src/lib.rs index 4b400d8f4c..85201bce2e 100644 --- a/polkadot/node/subsystem-util/src/lib.rs +++ b/polkadot/node/subsystem-util/src/lib.rs @@ -630,13 +630,13 @@ where } /// Parameters to a job subsystem. -struct JobSubsystemParams { +pub struct JobSubsystemParams { /// A spawner for sub-tasks. spawner: Spawner, /// Arguments to each job. run_args: RunArgs, /// Metrics for the subsystem. - metrics: Metrics, + pub metrics: Metrics, } /// A subsystem which wraps jobs. @@ -648,7 +648,8 @@ struct JobSubsystemParams { /// include a hash, then they're forwarded to the appropriate individual job. /// - On outgoing messages from the jobs, it forwards them to the overseer. pub struct JobSubsystem { - params: JobSubsystemParams, + #[allow(missing_docs)] + pub params: JobSubsystemParams, _marker: std::marker::PhantomData, } diff --git a/polkadot/node/test/polkadot-simnet/node/src/main.rs b/polkadot/node/test/polkadot-simnet/node/src/main.rs index 3a52182cde..7f85073442 100644 --- a/polkadot/node/test/polkadot-simnet/node/src/main.rs +++ b/polkadot/node/test/polkadot-simnet/node/src/main.rs @@ -15,10 +15,10 @@ // along with Polkadot. If not, see . //! Binary used for Simnet nodes, supports all runtimes, although only polkadot is implemented currently. -//! This binary accepts all the CLI args the polkadot binary does, Only difference is it uses +//! This binary accepts all the CLI args the polkadot binary does, with the only difference that it uses //! manual-seal™ and babe for block authorship, it has a no-op verifier, so all blocks received over the network //! are imported and executed straight away. Block authorship/Finalization maybe done by calling the -//! `engine_createBlock` & `engine_FinalizeBlock` rpc methods respectively. +//! rpc methods `engine_createBlock` and `engine_FinalizeBlock` respectively. use std::error::Error; diff --git a/polkadot/scripts/dockerfiles/malus.Dockerfile b/polkadot/scripts/dockerfiles/malus.Dockerfile new file mode 100644 index 0000000000..c595bc6331 --- /dev/null +++ b/polkadot/scripts/dockerfiles/malus.Dockerfile @@ -0,0 +1,50 @@ +FROM debian:bullseye-slim + +# metadata +ARG VCS_REF +ARG BUILD_DATE +ARG IMAGE_NAME + +LABEL io.parity.image.authors="devops-team@parity.io" \ + io.parity.image.vendor="Parity Technologies" \ + io.parity.image.title="${IMAGE_NAME}" \ + io.parity.image.description="Malus - the nemesis of polkadot" \ + io.parity.image.source="https://github.com/paritytech/polkadot/blob/${VCS_REF}/scripts/dockerfiles/malus.Dockerfile" \ + io.parity.image.revision="${VCS_REF}" \ + io.parity.image.created="${BUILD_DATE}" \ + io.parity.image.documentation="https://github.com/paritytech/polkadot/" + +# show backtraces +ENV RUST_BACKTRACE 1 + +# install tools and dependencies +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + ca-certificates \ + curl \ + libssl1.1 \ + tini && \ +# apt cleanup + apt-get autoremove -y && \ + apt-get clean && \ + find /var/lib/apt/lists/ -type f -not -name lock -delete; \ +# add user + groupadd --gid 10000 nonroot && \ + useradd --home-dir /home/nonroot \ + --create-home \ + --shell /bin/bash \ + --gid nonroot \ + --groups nonroot \ + --uid 10000 nonroot + + +# add adder-collator binary to docker image +COPY ./malus /usr/local/bin + +USER nonroot + +# check if executable works in this container +RUN /usr/local/bin/malus --version + +# Tini allows us to avoid several Docker edge cases, see https://github.com/krallin/tini. +ENTRYPOINT ["tini", "--", "/bin/bash"] diff --git a/polkadot/scripts/gitlab/lingua.dic b/polkadot/scripts/gitlab/lingua.dic index 71f269e314..baf66d97c2 100644 --- a/polkadot/scripts/gitlab/lingua.dic +++ b/polkadot/scripts/gitlab/lingua.dic @@ -85,11 +85,13 @@ fungibility gameable getter/MS GiB/S +GKE GNUNet GPL/M GPLv3/M Grafana/MS Gurke/MS +gurke/MS Handler/MS HMP/SM HRMP @@ -135,7 +137,7 @@ lookahead/MS lookup/MS LRU mainnet/MS -malus +malus/MS MB/M Mbit merkle/MS @@ -259,8 +261,10 @@ teleport/RG teleportation/SM teleporter/SM teleporters -teleports +template/GSM testnet/MS +tera/M +teleports timeframe timestamp/MS tradeoff diff --git a/polkadot/simnet_tests/README.md b/polkadot/simnet_tests/README.md deleted file mode 100644 index 3bd7ae8ee0..0000000000 --- a/polkadot/simnet_tests/README.md +++ /dev/null @@ -1,83 +0,0 @@ -# Simulation tests, or end-to-end tests - -_The content of this directory is meant to be used by Parity's private CI/CD -infrastructure with private tools. At the moment those tools are still early -stage of development and we don't know if / when they will available for -public use._ - -## Contents of this directory - -This directory contains different test suits, everyone one of them contains the set of test cases. -Every test suits is defined by its definition file test_suit_description.toml. More information about -structure of test suits and test cases may be found in [SimNet repository](https://gitlab.parity.io/parity/simnet/-/tree/master/ci_helper). - -Every test case deploys a test network, using toml config file, and runs the test, -using a test scenario, written in [Cucumber](https://cucumber.io/). - -These test suits are run by Polkadot CI in different pipelines, for every commit in PR, for commit into master etc. -It's the responsibility of the test's developer to provide the correct tag for their test, in order to let CI know, when -this test case should be run. For the baseline the existing tags from the existing tests may be used. If these tags are not -sufficient, the new tag may be created. But CI team should be aware of this tag and condition, when this test case should be run. - -In order to run a test case locally, you need to install -[Gurke](https://github.com/paritytech/gurke) -Once you have access to a kubernetes cluster (meaning you can do `kubectl get pods`) -you can use Gurke in order to deploy a chain and run the test (see gurke's manual for the commands). -Kubernetes cluster can be local, spawned with -[kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) -or an instance living in the -[cloud](https://github.com/paritytech/gurke/blob/main/docs/How-to-setup-access-to-gke-k8s-cluster.md) - -## How to add new test cases -New test case may be added either into the existing test suit or with creation of the new test suit. -In any case it's better to create the test and run it locally first, using Gurke (see above). -- In order to add the test case into the existing test suit, the new element (test case) should be added into [[test-cases]] array in test_suit_description.toml of this test suit. The example: - -``` -# The existing test case -[[test-cases]] -tags = ["all", "smoketest"] -chain-config = "configs/default_local_testnet.toml" -scenarios = ["tests/001-smoketest.feature"] - -# The new test case -[[test-cases]] -tags = ["all", "load"] -allowed-to-fail = true -chain-config = "configs/default_local_testnet.toml" -scenarios = ["tests/002-loadtest.feature"] -``` -(See note about test case's tags above). - -- In order to create a new test suit for the test case, new folder with test suit description file (test_suit_description.toml) should be created. The exact name is mandatory, CI traverses all subfolfders of simnet_tests directory and looks for this file, in order to build the list of test suits. In this description file the general information about the test suit and array of the test cases should be specified. The example of test_suit_description.toml file with some verbose comments: -``` -name = "Name of the test suit" -description = "General information about the suit" -# It is the path to the setup script, that may be needed for the test suit. -# You can perform in this script the actions, that are required for the chain deployment, but have to be performed before -# spawing the chain. Like pre-generation of seeds. This script is run by CI before spawing the chain in the cluster -setup-script="setup_script.sh" -# If the config of the test case requires some custom Docker images, the names for these images should be listed in this section -# CI has to provide all these images in the format image_name = some_value in order to run test cases from this suit -# Obviosuly CI should be aware, if the new custom image is added. -required-images = [ - "SYNTHIMAGE", - "COLIMAGE", - "SCRIPTSIMAGE", - "PARACHAINSIMAGE" -] - -# Array of the test cases -# Every elements should be started with [[test-cases]] -[[test-cases]] -# See tags information above -tags = ["all", "smoketest"] -# The config, that will be used in order to spawn the chain -chain-config = "configs/simple_rococo_testnet.toml" -# The array of the scenarios, that will be run on the deployed chain -scenarios = ["tests/parachains.feature"] -``` - -### [Here is link to barcamp presentation of Simnet](https://www.crowdcast.io/e/ph49xu01) - -### [Here is link to the Simnet repo, hosted on private gitlab](https://gitlab.parity.io/parity/simnet/-/tree/master) diff --git a/polkadot/simnet_tests/parachains/configs/adder.json b/polkadot/simnet_tests/parachains/configs/adder.json deleted file mode 100644 index bd2af7df23..0000000000 --- a/polkadot/simnet_tests/parachains/configs/adder.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "Header":{ - "number":"u64", - "parent_hash":"Hash", - "post_state":"Hash" - } -} diff --git a/polkadot/simnet_tests/parachains/configs/simple_rococo_testnet.toml b/polkadot/simnet_tests/parachains/configs/simple_rococo_testnet.toml deleted file mode 100644 index c27768f471..0000000000 --- a/polkadot/simnet_tests/parachains/configs/simple_rococo_testnet.toml +++ /dev/null @@ -1,53 +0,0 @@ -[settings.setup] -timeout = 300 - -[settings.defaults] -image = "{{get_env(name="SYNTHIMAGE") | safe }}" -command = "polkadot" -chain-spec = "rococo-local.json" -chain-name = "rococo-local" -timeout = 300 - -[init_nodes.chainspec] -image = "{{get_env(name="SYNTHIMAGE") | safe }}" -command = "/usr/local/bin/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > /cfg/rococo-local.json" -fetch-files = [ "/cfg/rococo-local.json" ] -timeout = 300 - -[init_nodes.parachain-specs] -image = "{{get_env(name="COLIMAGE") | safe }}" -command = """ -/usr/local/bin/adder-collator export-genesis-state > /cfg/genesis-state && -/usr/local/bin/adder-collator export-genesis-wasm > /cfg/genesis-wasm -""" -fetch-files = [ "/cfg/genesis-wasm", "/cfg/genesis-state" ] -timeout = 300 - -[nodes.alice] -validator = true -image = "{{get_env(name="PARACHAINSIMAGE") | safe }}" -command = "polkadot" -extra-args = ["--alice"] - -[nodes.bob] -validator = true -image = "{{get_env(name="PARACHAINSIMAGE") | safe }}" -command = "polkadot" -extra-args = ["--bob"] - -[nodes.collator01] -image = "{{get_env(name="COLIMAGE") | safe }}" -command-with-args = "/usr/local/bin/adder-collator --chain /cfg/rococo-local.json --port 30333 --no-mdns --bootnodes /dns/bootnode/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp" - -[nodes.nodejs] -image = "{{get_env(name="SCRIPTSIMAGE") | safe }}" -command-with-args = """ -cd simnet_scripts; -npm run build; -node /usr/local/bin/simnet_scripts/dist/index.js register_parachain /cfg/genesis-wasm /cfg/genesis-state 100 true ws://bootnode:9944; -tail -f /dev/null -""" -copy-files = [ - "genesis-state", - "genesis-wasm", -] diff --git a/polkadot/simnet_tests/parachains/test_suit_description.toml b/polkadot/simnet_tests/parachains/test_suit_description.toml deleted file mode 100644 index 2597ab5747..0000000000 --- a/polkadot/simnet_tests/parachains/test_suit_description.toml +++ /dev/null @@ -1,13 +0,0 @@ -name = "Parachains test" -description = "Smoke test for parachains, that deploy a simple collator and register parachain" -required-images = [ - "SYNTHIMAGE", - "COLIMAGE", - "SCRIPTSIMAGE", - "PARACHAINSIMAGE" -] - -[[test-cases]] -tags = ["all", "smoketest"] -chain-config = "configs/simple_rococo_testnet.toml" -scenarios = ["tests/parachains.feature"] diff --git a/polkadot/simnet_tests/parachains/tests/parachains.feature b/polkadot/simnet_tests/parachains/tests/parachains.feature deleted file mode 100644 index 73ebb744e6..0000000000 --- a/polkadot/simnet_tests/parachains/tests/parachains.feature +++ /dev/null @@ -1,6 +0,0 @@ -Feature: ParaTesting - - Scenario: spawn parachains network and check parachains - Given a test network - Then sleep 200 seconds - Then launch 'node' with parameters '/usr/local/bin/simnet_scripts/dist/index.js test_parachain /usr/local/bin/simnet_scripts/type_defs/adder.json ws://localhost:11222 100 10' diff --git a/polkadot/zombienet_tests/README.md b/polkadot/zombienet_tests/README.md new file mode 100644 index 0000000000..2c732a6801 --- /dev/null +++ b/polkadot/zombienet_tests/README.md @@ -0,0 +1,16 @@ +# Zombienet tests + +_The content of this directory is meant to be used by Parity's private CI/CD infrastructure with private tools. At the moment those tools are still early stage of development and we don't know if / when they will available for public use._ + +## Contents of this directory + +`parachains` + At the moment this directory only have one test related to parachains: `/parachains-smoke-test`, that check the parachain registration and the block height. + +## Resources (private) + +* [zombienet repo](https://github.com/paritytech/zombienet) + +## Questions / permissions + +Ping in element [Javier](@javier:matrix.parity.io) to ask questions or grant permission to run the test from your local setup. diff --git a/polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.feature b/polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.feature new file mode 100644 index 0000000000..f134204708 --- /dev/null +++ b/polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.feature @@ -0,0 +1,9 @@ +Description: Smoke Test +Network: ./0001-parachains-smoke-test.toml +Creds: config.gcloud + + +alice: is up +bob: is up +alice: parachain 100 is registered within 225 seconds +alice: parachain 100 block height is at least 10 within 200 seconds \ No newline at end of file diff --git a/polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.toml b/polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.toml new file mode 100644 index 0000000000..2b1a9fb0fa --- /dev/null +++ b/polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.toml @@ -0,0 +1,29 @@ +[settings] +timeout = 1000 + +[relaychain] +default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}" +chain = "rococo-local" +chain_spec_command = "polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > /cfg/rococo-local.json" +command = "polkadot" + + [[relaychain.nodes]] + name = "alice" + extra_args = [ "--alice" ] + + [[relaychain.nodes]] + name = "bob" + extra_args = [ "--bob" ] + +[[parachains]] +id = 100 + + [parachains.collator] + name = "collator01" + image = "{{COL_IMAGE}}" + command = "/usr/local/bin/adder-collator" + +[types.Header] +number = "u64" +parent_hash = "Hash" +post_state = "Hash"