* [WIP] PVF: Split out worker binaries
* Address compilation problems and re-design a bit
* Reorganize once more, fix tests
* Reformat with new nightly to make `cargo fmt` test happy
* Address `clippy` warnings
* Add temporary trace to debug zombienet tests
* Fix zombienet node upgrade test
* Fix malus and its CI
* Fix building worker binaries with malus
* More fixes for malus
* Remove unneeded cli subcommands
* Support placing auxiliary binaries to `/usr/libexec`
* Fix spelling
* Spelling
Co-authored-by: Marcin S. <marcin@realemail.net>
* Implement review comments (mostly nits)
* Fix worker node version flag
* Rework getting the worker paths
* Address a couple of review comments
* Minor restructuring
* Fix CI error
* Add tests for worker binaries detection
* Improve tests; try to fix CI
* Move workers module into separate file
* Try to fix failing test and workers not printing latest version
- Tests were not finding the worker binaries
- Workers were not being rebuilt when the version changed
- Made some errors easier to read
* Make a bunch of fixes
* Rebuild nodes on version change
* Fix more issues
* Fix tests
* Pass node version from node into dependencies to avoid recompiles
- [X] get version in CLI
- [X] pass it in to service
- [X] pass version along to PVF
- [X] remove rerun from service
- [X] add rerun to CLI
- [X] don’t rerun pvf/worker’s (these should be built by nodes which have rerun enabled)
* Some more improvements for smoother tests
- [X] Fix tests
- [X] Make puppet workers pass None for version and remove rerun
- [X] Make test collators self-contained
* Add back rerun to PVF workers
* Move worker binaries into files in cli crate
As a final optimization I've separated out each worker binary from its own crate
into the CLI crate. Before, the worker bin shared a crate with the worker lib,
so when the binaries got recompiled so did the libs and everything transitively
depending on the libs. This commit fixes this regression that was causing
recompiles after every commit.
* Fix bug (was passing worker version for node version)
* Move workers out of cli into root src/bin/ dir
- [X] Pass in node version from top-level (polkadot)
- [X] Add build.rs with rerun-git-head to root dir
* Add some sanity checks for workers to dockerfiles
* Update malus
+ [X] Make it self-contained
+ [X] Undo multiple binary changes
* Try to fix clippy errors
* Address `cargo run` issue
- [X] Add default-run for polkadot
- [X] Add note about installation to error
* Update readme (installation instructions)
* Allow disabling external workers for local/testing setups
+ [X] cli flag to enable single-binary mode
+ [X] Add message to error
* Revert unnecessary Cargo.lock changes
* Remove unnecessary build scripts from collators
* Add back missing malus commands (should fix failing ZN job)
* Some minor fixes
* Update Cargo.lock
* Fix some build errors
* Undo self-contained binaries; cli flag to disable version check
+ [X] Remove --dont-run-external-workers
+ [X] Add --disable-worker-version-check
+ [X] Remove PVF subcommands
+ [X] Redo malus changes
* Try to fix failing job and add some docs for local tests
---------
Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: parity-processbot <>
* test that successfully verify self-generated proof
* verify proof result on all unpaused nodes
* tighten interface: reuse connection
* generate the proof on arbitrary node
* s/stub-name/validator
seems this doesn't pass some regex
* rename script
* add basic mmr-leaves test
* documentation formatting
* check lower bound on mmr leaves
* test beefy rpc: finalized heads match
* test mmr api: stateless proofs
* set lower bound on number of leaves
* change leaves in proof generation
* remove TODOs
* cleanup: consistently ignore zndsl name arg
* refactor: simplify returns
Co-authored-by: Serban Iorga <serban@parity.io>
* lax finalized head test
Co-authored-by: Adrian Catangiu <adrian@parity.io>
* fixup! refactor: simplify returns
* refactor out getApis
Co-authored-by: Serban Iorga <serban@parity.io>
* split out paused validator from group
* refactor: don't ignore node arg
don't randomize selection of proof generating / finalized head
retrieving node, but use the node arg for this.
* only check min block height, not relative
Co-authored-by: Adrian Catangiu <adrian@parity.io>
* verify finalized heads share same canonical chain
picks the node with the highest finalized head, gets its blockchain
headers since genesis, and verifies that all other nodes' finalized
heads are in said blockchain.
* fixup! split out paused validator from group
Co-authored-by: Serban Iorga <serban@parity.io>
---------
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
* test/malus: craft the first maliciously disputing actor
* initial draft
* Add Dockerfile and instructions how to use it to build malus image locally
* Forgot one flag for the build cmd
* we are not docker specific, we are happy to use any containerruntime
* shuffle things around
* add initial tera based integration test
* chores
* fixins
* simple setup to start
* other samples (WIP)
* add Docker version with cargo-chef
* update substarte and small change of orders in commands in the container file
* metrics one
* fmt
* minor
* fixin
* fix metric names
* -d
* add open gauge
* fmt
* spellcheck
* fix test
* adjust to changed error messages
* refactor, more malus impls
* more malus changes
* foo
* minor cleanup
* suggest garbage candidate
* chore
* fix suggest garabge malus
* malus: back garbage candidate
* cargo lock
* re-introduce metrics
* chore: cargo fmt
* undoe 1.54.0 output, CI uses 1.53.0 rustc
* update location of js types
* Fix trybuild
* add tag to image name also; this will be replaced in the prod version
* Tests fixed
* add some fix me
* add dockerfile for ci
* Add docker file for malus for ci
* use variables in .toml file
* add chnages for malus test
* some fixes
* some more fixes
* Update .gitlab-ci.yml
* add local build for polkadot and malus
* some fixes
* enable disputes feature in CI
* ok, ok
* rename: MsgFilter -> MessageInterceptor
* remove TODO that would not have worked
* intercept
* refactor
* fix README and containers
* fix
* chore: cargo fmt
* avoid some more malus-$VARIANT references
* fix argument order
* chore: add about
* Update sanity check in relay chain selection
* fix order, add dispute-unavailable-block malus
* fixup: avoid underflow issue
* it's all u32
* fix conditional use
* Revert "it's all u32"
This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e.
* Revert "fixup: avoid underflow issue"
This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47.
* Revert "Update sanity check in relay chain selection"
This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d.
* update the malus bin
* Update node/malus/integrationtests/0003-dispute-unavailable-block.feature
Co-authored-by: Andronik Ordian <write@reusable.software>
* add some FIXME reminders
* update path to index.js
* Update .gitlab-ci.yml
* Update node/malus/integrationtests/0001-dispute-valid-block.toml
* try 1: make malus test run
* chore: cargo fmt
* temporary fix
* use subcommand syntax from latest gurke
* cargo +nightly fmt
* add collator to a a test
* docs: add env vars to README
* update ci to run dispute-valid-block test
* needs the polkadot image
* Fix path for nodejs container
* post merge fix
* download proper dir containg configs for malus test
* update the malus ci job
* rm a whitespace
* temp build for malus
* use correct build command for temp malus
* remove subcommands for now
* set max validators per core in the default HostConfig
* tabs
* update beefy
* fixup
* fixup II
* make one variant compile
* make other variants compile
* revert changes to chain_spec
* fmt
* build malus image from polkadot-test-malus again
* revert unrelated changes
* try fixing build-malus job
* Revert "remove subcommands for now"
This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e.
* try fixing build-malus job II
* MVP working dispute-ancestor
* renames
* fix PVF execution on malus
* fix test
* fix typo
* fmt
* checkmate
* try something
* make it actually work
* some tweaks to 01 feature test
* fmt
* sleep a bit more
* complete wococoization
* some tweaks to 01 feature test
* typo fix
* use correct metric names
* fix
* ffs
* .
* try some rearrangement
* Attempt to wait till initial node bootstrap in test
* Fix test syntax
* Run malus tests with v2 script
* Proper symlink created
* simnet v14
* add zombienet tests
* add zombie net test - draft
* add more tests to dispute suite
* add within to fix tests
* replace test directory and start test migration
* migrate all the tests
* add timeout to tests
* reduce debug
* make easy to test in dev
* set appropriated debug
* use image from ci
* fix config for test
* set images from ci
* fix config
* add COLIMAGE env
* tweek tests
* reduce debug
* typo
* wip, migrate old test to zombie-net
* adjunt test config for zombie-net
* run mauls 0001 test only
* clean and setup smoke-test in zombie-net
* add extra time to assertinons
* clean code to merge and improve README
* add info to access logs
* improved readme
* merge master and resolve conflicts
* Update zombienet_tests/README.md
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* clean and consolidate zombienet name
* change runner in gitlab
* add comment explain why we use wococo
* change tag for runner
* remove unused tests
* remove dup Dockerfile and update description
* fmt
* fix compilation post-merge
* fmt
* cut me Some slack
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
Chevdor