introduce malus + zombienet based integration tests (#4131)

* test/malus: craft the first maliciously disputing actor * initial draft * Add Dockerfile and instructions how to use it to build malus image locally * Forgot one flag for the build cmd * we are not docker specific, we are happy to use any containerruntime * shuffle things around * add initial tera based integration test * chores * fixins * simple setup to start * other samples (WIP) * add Docker version with cargo-chef * update substarte and small change of orders in commands in the container file * metrics one * fmt * minor * fixin * fix metric names * -d * add open gauge * fmt * spellcheck * fix test * adjust to changed error messages * refactor, more malus impls * more malus changes * foo * minor cleanup * suggest garbage candidate * chore * fix suggest garabge malus * malus: back garbage candidate * cargo lock * re-introduce metrics * chore: cargo fmt * undoe 1.54.0 output, CI uses 1.53.0 rustc * update location of js types * Fix trybuild * add tag to image name also; this will be replaced in the prod version * Tests fixed * add some fix me * add dockerfile for ci * Add docker file for malus for ci * use variables in .toml file * add chnages for malus test * some fixes * some more fixes * Update .gitlab-ci.yml * add local build for polkadot and malus * some fixes * enable disputes feature in CI * ok, ok * rename: MsgFilter -> MessageInterceptor * remove TODO that would not have worked * intercept * refactor * fix README and containers * fix * chore: cargo fmt * avoid some more malus-$VARIANT references * fix argument order * chore: add about * Update sanity check in relay chain selection * fix order, add dispute-unavailable-block malus * fixup: avoid underflow issue * it's all u32 * fix conditional use * Revert "it's all u32" This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e. * Revert "fixup: avoid underflow issue" This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47. * Revert "Update sanity check in relay chain selection" This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d. * update the malus bin * Update node/malus/integrationtests/0003-dispute-unavailable-block.feature Co-authored-by: Andronik Ordian <write@reusable.software> * add some FIXME reminders * update path to index.js * Update .gitlab-ci.yml * Update node/malus/integrationtests/0001-dispute-valid-block.toml * try 1: make malus test run * chore: cargo fmt * temporary fix * use subcommand syntax from latest gurke * cargo +nightly fmt * add collator to a a test * docs: add env vars to README * update ci to run dispute-valid-block test * needs the polkadot image * Fix path for nodejs container * post merge fix * download proper dir containg configs for malus test * update the malus ci job * rm a whitespace * temp build for malus * use correct build command for temp malus * remove subcommands for now * set max validators per core in the default HostConfig * tabs * update beefy * fixup * fixup II * make one variant compile * make other variants compile * revert changes to chain_spec * fmt * build malus image from polkadot-test-malus again * revert unrelated changes * try fixing build-malus job * Revert "remove subcommands for now" This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e. * try fixing build-malus job II * MVP working dispute-ancestor * renames * fix PVF execution on malus * fix test * fix typo * fmt * checkmate * try something * make it actually work * some tweaks to 01 feature test * fmt * sleep a bit more * complete wococoization * some tweaks to 01 feature test * typo fix * use correct metric names * fix * ffs * . * try some rearrangement * Attempt to wait till initial node bootstrap in test * Fix test syntax * Run malus tests with v2 script * Proper symlink created * simnet v14 * add zombienet tests * add zombie net test - draft * add more tests to dispute suite * add within to fix tests * replace test directory and start test migration * migrate all the tests * add timeout to tests * reduce debug * make easy to test in dev * set appropriated debug * use image from ci * fix config for test * set images from ci * fix config * add COLIMAGE env * tweek tests * reduce debug * typo * wip, migrate old test to zombie-net * adjunt test config for zombie-net * run mauls 0001 test only * clean and setup smoke-test in zombie-net * add extra time to assertinons * clean code to merge and improve README * add info to access logs * improved readme * merge master and resolve conflicts * Update zombienet_tests/README.md Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * clean and consolidate zombienet name * change runner in gitlab * add comment explain why we use wococo * change tag for runner * remove unused tests * remove dup Dockerfile and update description * fmt * fix compilation post-merge * fmt * cut me Some slack Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> Co-authored-by: radupopa2010 <radupopa2010@yahoo.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com> Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2026-05-30 17:31:03 +00:00 · 2021-11-20 15:03:28 +01:00
parent e35ebee0f1
commit ea5dbd0475
32 changed files with 1401 additions and 331 deletions
@@ -27,10 +27,10 @@ variables:
  CI_IMAGE:                        "paritytech/ci-linux:production"
  DOCKER_OS:                       "debian:stretch"
  ARCH:                            "x86_64"
+  ZOMBIENET_IMAGE:                 "docker.io/paritypr/zombienet"
  VAULT_SERVER_URL:                "https://vault.parity-mgmt-vault.parity.io"
  VAULT_AUTH_PATH:                 "gitlab-parity-io-jwt"
  VAULT_AUTH_ROLE:                 "cicd_gitlab_parity_${CI_PROJECT_NAME}"
-  SIMNET_IMAGE:                    "europe-west3-docker.pkg.dev/parity-simnet/simnet-images/simnet:v14"
  PIPELINE_SCRIPTS_TAG:            "v0.1"

 default:
@@ -191,6 +191,7 @@ test-build-linux-stable:
  script:
    - ./scripts/gitlab/test_linux_stable.sh
    # we're using the bin built here, instead of having a parallel `build-linux-release`
+    # disputes feature is needed for zombie-net parachains malus test
    - time cargo build --release --verbose --bin polkadot --features "disputes"
    - sccache -s
    # pack artifacts
@@ -259,6 +260,26 @@ build-adder-collator:
    - echo "adder-collator version = $(cat ./artifacts/VERSION) (EXTRATAG = $(cat ./artifacts/EXTRATAG))"
    - cp -r scripts/* ./artifacts

+build-malus:
+  stage:                           test
+  <<:                              *collect-artifacts
+  <<:                              *docker-env
+  <<:                              *compiler-info
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_COMMIT_REF_NAME == "master"
+    - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/                         # PRs
+  script:
+    - time cargo build --release --verbose -p polkadot-test-malus --features disputes
+    - sccache -s
+    # pack artifacts
+    - mkdir -p ./artifacts
+    - mv ./target/release/malus ./artifacts/.
+    - echo -n "${CI_COMMIT_REF_NAME}" > ./artifacts/VERSION
+    - echo -n "${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}" > ./artifacts/EXTRATAG
+    - echo "polkadot-test-malus = $(cat ./artifacts/VERSION) (EXTRATAG = $(cat ./artifacts/EXTRATAG))"
+    - cp -r scripts/* ./artifacts
+
 #### stage:                        build

 .check-dependent-project:          &check-dependent-project
@@ -393,13 +414,13 @@ publish-polkadot-image:
    - job:                         test-build-linux-stable
      artifacts:                   true
  after_script:
-    # pass artifacts to the simnet-tests job
+    # pass artifacts to the zombienet-tests job
    # https://docs.gitlab.com/ee/ci/multi_project_pipelines.html#with-variable-inheritance
    - echo "PARACHAINS_IMAGE_NAME=${IMAGE_NAME}" > ./artifacts/parachains.env
    - echo "PARACHAINS_IMAGE_TAG=$(cat ./artifacts/EXTRATAG)" >> ./artifacts/parachains.env
  artifacts:
    reports:
-      # this artifact is used in simnet-tests job
+      # this artifact is used in zombienet-tests job
      dotenv: ./artifacts/parachains.env

 publish-adder-collator-image:
@@ -417,14 +438,40 @@ publish-adder-collator-image:
      artifacts:                   true
  after_script:
    - buildah logout --all
-    # pass artifacts to the simnet-tests job
+    # pass artifacts to the zombienet-tests job
    - echo "COLLATOR_IMAGE_NAME=${IMAGE_NAME}" > ./artifacts/collator.env
    - echo "COLLATOR_IMAGE_TAG=$(cat ./artifacts/EXTRATAG)" >> ./artifacts/collator.env
  artifacts:
    reports:
-      # this artifact is used in simnet-tests job
+      # this artifact is used in zombienet-tests job
      dotenv: ./artifacts/collator.env

+publish-malus-image:
+  # service image for Simnet
+  stage:                           build
+  <<:                              *build-push-image
+  variables:
+    <<:                            *image-variables
+    # scripts/dockerfiles/malus.Dockerfile
+    DOCKERFILE:                    dockerfiles/malus.Dockerfile
+    IMAGE_NAME:                    docker.io/paritypr/malus
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_COMMIT_REF_NAME == "master"
+    - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/                         # PRs
+  needs:
+    - job:                         build-malus
+      artifacts:                   true
+  after_script:
+    - buildah logout "$IMAGE_NAME"
+    # pass artifacts to the zombienet-tests job
+    - echo "MALUS_IMAGE_NAME=${IMAGE_NAME}" > ./artifacts/malus.env
+    - echo "MALUS_IMAGE_TAG=$(cat ./artifacts/EXTRATAG)" >> ./artifacts/malus.env
+  artifacts:
+    reports:
+      # this artifact is used in zombienet-tests job
+      dotenv: ./artifacts/malus.env
+
 update_polkadot_weights:           &update-weights
  stage:                           build
  when:                            manual
@@ -550,38 +597,80 @@ deploy-polkasync-kusama:
  allow_failure:                   true
  trigger:                         "parity/infrastructure/parity-testnet"

-simnet-tests:
+zombienet-tests-parachains-smoke-test:
  stage:                           deploy
-  image:                           "${SIMNET_IMAGE}"
+  image:                           "${ZOMBIENET_IMAGE}"
  <<:                              *kubernetes-env
-  <<:                              *rules-test-and-rococo
-  variables:
-    GH_DIR:                        "https://github.com/paritytech/polkadot/tree/master/simnet_tests"
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_COMMIT_REF_NAME == "master"
+    - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/                         # PRs
+    - if: $CI_COMMIT_REF_NAME == "rococo-v1"
  needs:
    - job:                         publish-polkadot-image
+    - job:                         publish-malus-image
    - job:                         publish-adder-collator-image
-  # `parachains.env` brings here `$PARACHAINS_IMAGE_NAME` and `$PARACHAINS_IMAGE_TAG`
-  # (`$EXTRATAG` here, # i.e. `2643-0.8.29-5f689e0a-6b24dc54`).
-  # `collator.env` brings here `$COLLATOR_IMAGE_NAME` and  `$COLLATOR_IMAGE_TAG`
-  # For local tests with docker `$PARACHAINS_IMAGE_TAG` and `$COLLATOR_IMAGE_TAG`
-  # can be replaced with `master` tag.

-  # SIMNET_REF is a gitlab variable
+  variables:
+    GH_DIR:                        'https://github.com/paritytech/polkadot/tree/bernhard-malus-fx-zombienet/zombienet_tests/parachains'
+    # FIXME: use the master after the merge of the malus pr
+    # GH_DIR:                        'https://github.com/paritytech/polkadot/tree/master/zombienet_tests/parachains'
+
  before_script:
-    - echo "Simnet Tests Config"
-    - echo "${SIMNET_IMAGE}"
+    - echo "Zombie-net Tests Config"
+    - echo "${ZOMBIENET_IMAGE_NAME}"
    - echo "${PARACHAINS_IMAGE_NAME} ${PARACHAINS_IMAGE_TAG}"
-    - echo "${COLLATOR_IMAGE_NAME} ${COLLATOR_IMAGE_TAG}"
+    - echo "${MALUS_IMAGE_NAME} ${MALUS_IMAGE_TAG}"
    - echo "${GH_DIR}"
+    - export DEBUG=zombie,zombie::network-node
+    - export ZOMBIENET_INTEGRATION_TEST_IMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}
+    - export MALUS_IMAGE=${MALUS_IMAGE_NAME}:${MALUS_IMAGE_TAG}
+    - export COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}
+
  script:
-    - /home/nonroot/simnet/scripts/run-test-environment-manager-v2.sh
+    - /home/nonroot/zombie-net/scripts/run-test-env-manager.sh
        --github-remote-dir="${GH_DIR}"
-        --tag=smoketest
-        --image="PARACHAINSIMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}"
-        --image="SYNTHIMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}"
-        --image="COLIMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}"
-        --image="SCRIPTSIMAGE=${SIMNET_IMAGE}"
+        --test="0001-parachains-smoke-test.feature"
  allow_failure:                   true
  retry: 2
  tags:
-    - polkadot-simnet
+    - zombienet-polkadot-integration-test
+
+zombienet-tests-malus-dispute-valid:
+  stage:                           deploy
+  image:                           "${ZOMBIENET_IMAGE}"
+  <<:                              *kubernetes-env
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_COMMIT_REF_NAME == "master"
+    - if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/                         # PRs
+    - if: $CI_COMMIT_REF_NAME == "rococo-v1"
+  needs:
+    - job:                         publish-polkadot-image
+    - job:                         publish-malus-image
+    - job:                         publish-adder-collator-image
+
+  variables:
+    GH_DIR:                        'https://github.com/paritytech/polkadot/tree/bernhard-malus-fx-zombienet/node/malus/integrationtests'
+    # FIXME: use the master after the merge of the malus pr
+    # GH_DIR:                        'https://github.com/paritytech/polkadot/tree/master/node/malus/integrationtests'
+
+  before_script:
+    - echo "Zombie-net Tests Config"
+    - echo "${ZOMBIENET_IMAGE_NAME}"
+    - echo "${PARACHAINS_IMAGE_NAME} ${PARACHAINS_IMAGE_TAG}"
+    - echo "${MALUS_IMAGE_NAME} ${MALUS_IMAGE_TAG}"
+    - echo "${GH_DIR}"
+    - export DEBUG=zombie,zombie::network-node
+    - export ZOMBIENET_INTEGRATION_TEST_IMAGE=${PARACHAINS_IMAGE_NAME}:${PARACHAINS_IMAGE_TAG}
+    - export MALUS_IMAGE=${MALUS_IMAGE_NAME}:${MALUS_IMAGE_TAG}
+    - export COL_IMAGE=${COLLATOR_IMAGE_NAME}:${COLLATOR_IMAGE_TAG}
+
+  script:
+    - /home/nonroot/zombie-net/scripts/run-test-env-manager.sh
+        --github-remote-dir="${GH_DIR}"
+        --test="0001-dispute-valid-block.feature"
+  allow_failure:                   true
+  retry: 2
+  tags:
+    - zombienet-polkadot-integration-test