introduce malus + zombienet based integration tests (#4131)

* test/malus: craft the first maliciously disputing actor

* initial draft

* Add Dockerfile and instructions how to use it to build malus image locally

* Forgot one flag for the build cmd

* we are not docker specific, we are happy to use any containerruntime

* shuffle things around

* add initial tera based integration test

* chores

* fixins

* simple setup to start

* other samples (WIP)

* add Docker version with cargo-chef

* update substarte and small change of orders in commands in the container file

* metrics one

* fmt

* minor

* fixin

* fix metric names

* -d

* add open gauge

* fmt

* spellcheck

* fix test

* adjust to changed error messages

* refactor, more malus impls

* more malus changes

* foo

* minor cleanup

* suggest garbage candidate

* chore

* fix suggest garabge malus

* malus: back garbage candidate

* cargo lock

* re-introduce metrics

* chore: cargo fmt

* undoe 1.54.0 output, CI uses 1.53.0 rustc

* update location of js types

* Fix trybuild

* add tag to image name also; this will be replaced in the prod version

* Tests fixed

* add some fix me

* add dockerfile for ci

* Add docker file for malus for ci

* use variables in .toml file

* add chnages for malus test

* some fixes

* some more fixes

* Update .gitlab-ci.yml

* add local build for polkadot and malus

* some fixes

* enable disputes feature in CI

* ok, ok

* rename: MsgFilter -> MessageInterceptor

* remove TODO that would not have worked

* intercept

* refactor

* fix README and containers

* fix

* chore: cargo fmt

* avoid some more malus-$VARIANT references

* fix argument order

* chore: add about

* Update sanity check in relay chain selection

* fix order, add dispute-unavailable-block malus

* fixup: avoid underflow issue

* it's all u32

* fix conditional use

* Revert "it's all u32"

This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e.

* Revert "fixup: avoid underflow issue"

This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47.

* Revert "Update sanity check in relay chain selection"

This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d.

* update the malus bin

* Update node/malus/integrationtests/0003-dispute-unavailable-block.feature

Co-authored-by: Andronik Ordian <write@reusable.software>

* add some FIXME reminders

* update path to index.js

* Update .gitlab-ci.yml

* Update node/malus/integrationtests/0001-dispute-valid-block.toml

* try 1: make malus test run

* chore: cargo fmt

* temporary fix

* use subcommand syntax from latest gurke

* cargo +nightly fmt

* add collator to a a test

* docs: add env vars to README

* update ci to run dispute-valid-block test

* needs the polkadot image

* Fix path for nodejs container

* post merge fix

* download proper dir  containg configs for malus test

* update the malus ci job

* rm a whitespace

* temp build for malus

* use correct build command for temp malus

* remove subcommands for now

* set max validators per core in the default HostConfig

* tabs

* update beefy

* fixup

* fixup II

* make one variant compile

* make other variants compile

* revert changes to chain_spec

* fmt

* build malus image from polkadot-test-malus again

* revert unrelated changes

* try fixing build-malus job

* Revert "remove subcommands for now"

This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e.

* try fixing build-malus job II

* MVP working dispute-ancestor

* renames

* fix PVF execution on malus

* fix test

* fix typo

* fmt

* checkmate

* try something

* make it actually work

* some tweaks to 01 feature test

* fmt

* sleep a bit more

* complete wococoization

* some tweaks to 01 feature test

* typo fix

* use correct metric names

* fix

* ffs

* .

* try some rearrangement

* Attempt to wait till initial node bootstrap in test

* Fix test syntax

* Run malus tests with v2 script

* Proper symlink created

* simnet v14

* add zombienet tests

* add zombie net test - draft

* add more tests to dispute suite

* add within to fix tests

* replace test directory and start test migration

* migrate all the tests

* add timeout to tests

* reduce debug

* make easy to test in dev

* set appropriated debug

* use image from ci

* fix config for test

* set images from ci

* fix config

* add COLIMAGE env

* tweek tests

* reduce debug

* typo

* wip, migrate old test to zombie-net

* adjunt test config for zombie-net

* run mauls 0001 test only

* clean and setup smoke-test in zombie-net

* add extra time to assertinons

* clean code to merge and improve README

* add info to access logs

* improved readme

* merge master and resolve conflicts

* Update zombienet_tests/README.md

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* clean and consolidate zombienet name

* change runner in gitlab

* add comment explain why we use wococo

* change tag for runner

* remove unused tests

* remove dup Dockerfile and update description

* fmt

* fix compilation post-merge

* fmt

* cut me Some slack

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>

polkadot/zombienet_tests/README.md

@@ -0,0 +1,16 @@
+# Zombienet tests
+
+_The content of this directory is meant to be used by Parity's private CI/CD infrastructure with private tools. At the moment those tools are still early stage of development and we don't know if / when they will available for public use._
+
+## Contents of this directory
+
+`parachains`
+    At the moment this directory only have one test related to parachains: `/parachains-smoke-test`, that check the parachain registration and the block height.
+
+## Resources (private)
+
+* [zombienet repo](https://github.com/paritytech/zombienet)
+
+## Questions / permissions
+
+Ping in element [Javier](@javier:matrix.parity.io) to ask questions or grant permission to run the test from your local setup.

polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.feature

@@ -0,0 +1,9 @@
+Description: Smoke Test
+Network: ./0001-parachains-smoke-test.toml
+Creds: config.gcloud
+
+
+alice: is up
+bob: is up
+alice: parachain 100 is registered within 225 seconds
+alice: parachain 100 block height is at least 10 within 200 seconds

polkadot/zombienet_tests/parachains/0001-parachains-smoke-test.toml

@@ -0,0 +1,29 @@
+[settings]
+timeout = 1000
+
+[relaychain]
+default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}"
+chain = "rococo-local"
+chain_spec_command = "polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > /cfg/rococo-local.json"
+command = "polkadot"
+
+  [[relaychain.nodes]]
+  name = "alice"
+  extra_args = [ "--alice" ]
+
+  [[relaychain.nodes]]
+  name = "bob"
+  extra_args = [ "--bob" ]
+
+[[parachains]]
+id = 100
+
+  [parachains.collator]
+  name = "collator01"
+  image = "{{COL_IMAGE}}"
+  command = "/usr/local/bin/adder-collator"
+
+[types.Header]
+number = "u64"
+parent_hash = "Hash"
+post_state = "Hash"