Commit Graph

184 Commits

Author SHA1 Message Date
Bastian Köcher c40c4bcbe9 Companion CI: Make sure to pass the update crates properly (#4928) 2022-02-16 12:02:04 +01:00
Javier Viola b2b723a3db remove upgrade test (#4873) 2022-02-08 16:57:26 +00:00
Alexander Samusev 8ee165c2c1 [ci] Fix scheduled pipeline (#4855)
* [ci] Fix scheduled pipeline

* empty commit for pipeline rerun
2022-02-07 11:48:56 +01:00
Alexander Samusev 12f3e943b1 [ci] revert #4779 (#4843) 2022-02-03 16:24:45 +01:00
radupopa2010 f69ca55712 feat(cicd): expire artifacts after 7 days (#4833)
We received several alerts about disk space in gitlab server.
Even if we have 1TiB allocated, disk space is consumed very fast.
We need to address this problem by expiring artifacts after 7 days.
2022-02-02 13:56:41 +01:00
Alexander Samusev d9ab8ff576 [ci] Fix docker credentials (#4826) 2022-02-01 16:11:55 +01:00
Alexander Samusev 89c86d6300 [ci] remove publish-s3-release (#4779) 2022-01-25 12:06:04 +01:00
Alexander Samusev 6b7e40cd6c [ci] pipeline chores (#4775)
* [ci] WIP: pipeline chores

* fix vault

* fix secrets
2022-01-24 18:18:53 +01:00
Javier Viola 883b490cae [Zombienet] add dummy upgrade test (#4660) 2022-01-20 10:02:50 +01:00
Bastian Köcher 550e81b63f Check that try-runtime compiles in CI (#4736)
It also moves some simple scripts directly into the .gitlab.yaml file.
2022-01-18 16:17:07 +01:00
Denis Pisarev 3ce6f9212c CI: remove rococo from CI (#4666)
* CI: remove rococo from CI

* CI: typo

* CI: another typo

Co-authored-by: CI system <>
2022-01-10 18:28:06 +01:00
Andronik c805500dfb zombinet: bump collator image (#4668) 2022-01-07 09:04:20 +01:00
Alexander Samusev 041543882b [ci] .gitlab-ci.yml chores (#4663)
* rename malus.Dockerfile

* ci chores

* fix pipeline
2022-01-06 12:25:24 +01:00
Andronik a44349f652 custom cargo profiles for production (#4627)
* add custom profiles and specify MSVR as 1.57

* fix host-perf-check command

* use testnet profile for CI images

* do not do lto for testnet profile

* fix artifact path

* test with testnet profile to reuse build artifacts

* Revert "fix host-perf-check command"

This reverts commit f1d15492204b8251685a97636cbb5a5f394f21da.

* bump zombienet version

Co-authored-by: Javier Viola <javier@parity.io>
2021-12-29 14:10:40 +00:00
Andronik Ordian d077ac2c5e zombienet: fixed adder-collator image for the smoke test (#4600)
* zombinet: fixed adder-collator image for the smoke test

* update COL_IMAGE

* bump zombienet version

* try a different version

Co-authored-by: Javier Viola <javier@parity.io>
2021-12-27 13:29:37 +01:00
João Paulo Silva de Souza b3e36538cd make check-dependent-* only be executed in PRs (#4588)
Make check-dependent-* jobs only be executed in PRs instead of both PRs and
master.

Reason 1: The companion is not merged at the same time as the parent PR
([1](https://github.com/paritytech/parity-processbot/issues/347#issuecomment-994729950)),
therefore the pipeline will fail on master since the companion PR is not yet
merged in the other repository. This scenario is demonstrated by the pipeline
of
https://github.com/paritytech/substrate/commit/82cc3746450ae9722a249f4ddf83b8de59ba6e0d.

Reason 2: The job can still fail on master due to a new commit on the companion
PR's repository which was merged after `bot merge` happened, as demonstrated by
the following scheme:

1. Parent PR is merged
2. Companion PR is updated and set to merge in the future
3. In the meantime a new commit is merged into the companion PR repository's
  master branch
4. The `check-dependent-*` job runs on master but, due to the new commit, it
  fails for unrelated reasons

While "Reason 2" can be used as an argument against this PR, in that it would
be useful to know if the integration is failing on master, "Reason 1" should be
taken care of due to this inherent flaw of the current companion build system
design.
2021-12-27 10:21:39 +01:00
Javier Viola 78c3284ce7 [Zombienet] fix test creds (#4562)
* remove custom kube file in creds

* typo in image name echo
2021-12-18 20:56:02 +01:00
Javier Viola 686d82a437 use v1.0.2 of zombienet (#4553) 2021-12-18 14:12:54 +01:00
Bernhard Schuster 2457c26a08 enable disputes for known chains, except for polkadot (#4464)
* enable disputes, for all known chains but polkadot

* chore: fmt

* don't propagate disputes either

* review

* remove disputes feature

* remove superfluous line

* Update node/service/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* fixup

* allow being a dummy

* rialto

* add an enum, to make things work better

* overseer

* fix test

* comments

* move condition out

* excess arg

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-12-17 17:50:48 +00:00
Chris Sosnin 8b1514862d Dispute coordinator: look for included candidates in non-finalized chain (#4508)
* Fetch ancestors of the activated leaf

* Replace param type

* Increase step size

* Request block numbers for ancestors

* Store activated leaves in lru cache

* Fix doc-comment

* Rework LRU usage

* Typos & formatting

* Handle errors better

* Introduce a size limit for the ancestry

* Return fatal error when fetching finalized block

* Update tests

* Add tests for ordering provider

* Better naming

* fix zombienet test, new version

* zombienet add debug

* debug zombienet

Co-authored-by: Javier Viola <javier@parity.io>
2021-12-17 12:07:28 +01:00
Javier Viola 2c0d7730d7 Support new version of zombienet (#4528)
* test new version of zombienet

* use default chain spec command

* use commit ref to get test files

* change test spec for zombienet new versionzombienet_tests/parachains/0001-parachains-smoke-test.toml

* changes for new version of zombienet

* use paritytech zombienet version
2021-12-16 18:23:21 +01:00
Pierre Besson 6f2f9fe381 rename polkasync gitlab job to deploy-parity-testnet (#4461) 2021-12-16 12:43:58 +01:00
Pierre Krieger 77d70ac0d3 Companion PR for removing Prometheus metrics prefix (#3623)
* Companion PR for removing Prometheus metrics prefix

* Was missing some metrics

* Fix missing renames

* Fix test

* Fixes

* Update test

* Update Substrate

* Second time

* remove prefix from intergration test for zombienet

* update zombienet image

* Update Substrate

Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
2021-12-10 15:24:54 +01:00
Chevdor b081149939 Fix path of the polkadot sha256 (#4458) 2021-12-03 16:59:44 +01:00
Bastian Köcher cb8cb7a12c Fix cumulus companion CI job (#4451)
* Fix cumulus companion CI job

* Pass by env variable

* Switch to the latest release

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>
2021-12-03 14:14:50 +01:00
Alexander Samusev 2744faf6f0 [ci] Add ssh token for publishing gh-pages (#4347)
* [ci] Added ssh token for publishing gh-pages

* returned github_token

* fix ssh commands for publish-rustdoc

* fix ssh command for publish-docs

* Fix publish-rustdoc, remove clear /tmp
2021-11-25 10:18:38 +01:00
Javier Viola 5cca7e8d9c use correct gh_dir env for zombienet (#4346) 2021-11-22 12:26:13 +00:00
Javier Viola ea5dbd0475 introduce malus + zombienet based integration tests (#4131)
* test/malus: craft the first maliciously disputing actor

* initial draft

* Add Dockerfile and instructions how to use it to build malus image locally

* Forgot one flag for the build cmd

* we are not docker specific, we are happy to use any containerruntime

* shuffle things around

* add initial tera based integration test

* chores

* fixins

* simple setup to start

* other samples (WIP)

* add Docker version with cargo-chef

* update substarte and small change of orders in commands in the container file

* metrics one

* fmt

* minor

* fixin

* fix metric names

* -d

* add open gauge

* fmt

* spellcheck

* fix test

* adjust to changed error messages

* refactor, more malus impls

* more malus changes

* foo

* minor cleanup

* suggest garbage candidate

* chore

* fix suggest garabge malus

* malus: back garbage candidate

* cargo lock

* re-introduce metrics

* chore: cargo fmt

* undoe 1.54.0 output, CI uses 1.53.0 rustc

* update location of js types

* Fix trybuild

* add tag to image name also; this will be replaced in the prod version

* Tests fixed

* add some fix me

* add dockerfile for ci

* Add docker file for malus for ci

* use variables in .toml file

* add chnages for malus test

* some fixes

* some more fixes

* Update .gitlab-ci.yml

* add local build for polkadot and malus

* some fixes

* enable disputes feature in CI

* ok, ok

* rename: MsgFilter -> MessageInterceptor

* remove TODO that would not have worked

* intercept

* refactor

* fix README and containers

* fix

* chore: cargo fmt

* avoid some more malus-$VARIANT references

* fix argument order

* chore: add about

* Update sanity check in relay chain selection

* fix order, add dispute-unavailable-block malus

* fixup: avoid underflow issue

* it's all u32

* fix conditional use

* Revert "it's all u32"

This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e.

* Revert "fixup: avoid underflow issue"

This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47.

* Revert "Update sanity check in relay chain selection"

This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d.

* update the malus bin

* Update node/malus/integrationtests/0003-dispute-unavailable-block.feature

Co-authored-by: Andronik Ordian <write@reusable.software>

* add some FIXME reminders

* update path to index.js

* Update .gitlab-ci.yml

* Update node/malus/integrationtests/0001-dispute-valid-block.toml

* try 1: make malus test run

* chore: cargo fmt

* temporary fix

* use subcommand syntax from latest gurke

* cargo +nightly fmt

* add collator to a a test

* docs: add env vars to README

* update ci to run dispute-valid-block test

* needs the polkadot image

* Fix path for nodejs container

* post merge fix

* download proper dir  containg configs for malus test

* update the malus ci job

* rm a whitespace

* temp build for malus

* use correct build command for temp malus

* remove subcommands for now

* set max validators per core in the default HostConfig

* tabs

* update beefy

* fixup

* fixup II

* make one variant compile

* make other variants compile

* revert changes to chain_spec

* fmt

* build malus image from polkadot-test-malus again

* revert unrelated changes

* try fixing build-malus job

* Revert "remove subcommands for now"

This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e.

* try fixing build-malus job II

* MVP working dispute-ancestor

* renames

* fix PVF execution on malus

* fix test

* fix typo

* fmt

* checkmate

* try something

* make it actually work

* some tweaks to 01 feature test

* fmt

* sleep a bit more

* complete wococoization

* some tweaks to 01 feature test

* typo fix

* use correct metric names

* fix

* ffs

* .

* try some rearrangement

* Attempt to wait till initial node bootstrap in test

* Fix test syntax

* Run malus tests with v2 script

* Proper symlink created

* simnet v14

* add zombienet tests

* add zombie net test - draft

* add more tests to dispute suite

* add within to fix tests

* replace test directory and start test migration

* migrate all the tests

* add timeout to tests

* reduce debug

* make easy to test in dev

* set appropriated debug

* use image from ci

* fix config for test

* set images from ci

* fix config

* add COLIMAGE env

* tweek tests

* reduce debug

* typo

* wip, migrate old test to zombie-net

* adjunt test config for zombie-net

* run mauls 0001 test only

* clean and setup smoke-test in zombie-net

* add extra time to assertinons

* clean code to merge and improve README

* add info to access logs

* improved readme

* merge master and resolve conflicts

* Update zombienet_tests/README.md

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* clean and consolidate zombienet name

* change runner in gitlab

* add comment explain why we use wococo

* change tag for runner

* remove unused tests

* remove dup Dockerfile and update description

* fmt

* fix compilation post-merge

* fmt

* cut me Some slack

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2021-11-20 14:03:28 +00:00
Bernhard Schuster 2595d8c23c chore: fmt should print version (#4274)
Because it's just too aggravating having to fetch the container every time to
find out the version of cargo +nightly fmt.
2021-11-12 17:31:28 +00:00
Denis Pisarev 9d5724f926 CI: chore (#3957)
* CI: chore

* CI: lsof and jq are installed

* CI: generate docs with deps

* CI: remove check width, fmt checks it

* CI: simplify rules

* CI: make CI image variable

* CI: more chore

* rm check_line_width.sh

* CI: fix spellcheck; fix check-transaction

* CI: return installing jq

* lint: remove dupes
2021-11-11 17:38:05 +01:00
Anton Gavrilov bc61fa86f0 Change path for the tests to master (#4223) 2021-11-04 18:41:59 +00:00
radupopa2010 05bfc53878 Update SimNet test to the new format (#4051)
* simnet-v10 update simnet image version

* update config

* update path to js scripts

* use current branch to download config for tests

* use current branch to download config for tests

* temp use simnet-v10

* update also the toml file

* Rework test according to CIHelper format

* Call cihelper script for running tests

* Switch to v12

* Version 13

* simnet v14

* Readme extended

Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2021-11-03 11:34:26 +01:00
Andronik Ordian 9d4efa340e enable disputes feature in CI (#4168) 2021-10-28 08:40:13 -05:00
João Paulo Silva de Souza ce10434170 remove logging from the check-dependent-* job (#4120)
such command belongs to the script rather than the job
2021-10-21 16:03:54 +02:00
João Paulo Silva de Souza f2afedb91b remove hardcoded pipeline scripts tag (#4109)
the tag will be moved to Gitlab CI/CD variables
2021-10-19 19:53:42 +02:00
João Paulo Silva de Souza 8dbfea68c8 use pipeline-scripts for dependent projects script (#4017) 2021-10-14 14:25:26 +02:00
Sergejs Kostjucenko f31eac6ec6 Remove bridges from checks (#4040)
* Remove bridges from check_line_width

* Remove bridges from spellcheck
2021-10-08 14:43:17 +03:00
Bernhard Schuster 462bbc94b4 chore: ci list files that spellcheck finds (#3992)
* chore: ci list files that spellcheck finds

Avoid -r flag for false positives

* avoid master

* improve
2021-10-04 12:36:54 +00:00
radupopa2010 df184c8088 add polkadot-simnet runner (#3985) 2021-10-01 17:48:25 +03:00
Alexander Samusev f29b6db6e4 Docker files chore (#3880)
* Renamed staking-miner Dockerfiles

* change readme stacking-miner

* mv docker/ to scripts/docker/polkadot

* renamed dockerfiles in scripts/docker

* renamed Dockerfiles

* Update scripts/docker/staking-miner/staking_miner_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* renamed release.Dockerfile

* renamed staking-miner dockerfiles

* add label polkadot_builder.Dockerfile

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot_injected_release.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/collator_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* move gpg server as variable to GHA config

* simplified polkadot_builder.Dockerfile

* gitlab-ci modify cp command in test-build-linux-stable

* fix tests

* renamed scripts/docker to scripts/dockerfiles

* fix spellcheck

* fix spellcheck

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
2021-09-28 17:47:52 +02:00
Bastian Köcher 48b19bb319 Substrate Companion for rust 1.54 (#3807)
* Substrate Companion for rust 1.54

* CI: run with a staging CI image

* Revert "CI: run with a staging CI image"

This reverts commit f3b51da0a4c0ffa43b576af5482a2be256948870.

* CI: debug, again

* compilation fixes

* tabify

* update Substrate

* Update .gitlab-ci.yml

* CI: add ubuntu-based image with npm

Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: parity-processbot <>
2021-09-24 15:46:45 +02:00
Jaco 6f002ac734 Re-enable metadata compare CI check (#3895) 2021-09-20 17:20:48 +02:00
Andrew Jones 4c7539cab5 Companion for #8615: enrich metadata with type information (#3336)
* Use beefy branch with scale-info

* Add patches

* Sprinkle some TypeInfo derives

* Add some TypeInfo deriv

* Cargo.lock

* Derive TypeInfo and skip type params for Xcm types

* Cargo.lock

* Fix up scale_info bounds attributes

* Fix up dependencies

* Use my own beefy-primitives branch

* Bump BEEFY

* Update patches

* Add some scale-info dependencies and TypeInfo derives

* More TypeInfo decoration

* Update scale-info

* Some TypeInfos and remove more Event pallet::metadata

* Moar TypeInfos

* TypeInfos galore, fix up metadata runtime API

* TypeInfo

* TypeInfos, update other runtime metadata APIs

* Fix up Kusama, comment out some `usize` QueueSize parameter types

* Remove local diener patches

* Cargo.lock

* Cargo.lock

* Update to scale-info crates.io release

* Update primitive-types branch

* Update pallet-beefy to use custom branch

* Update other parity-common deps

* Update parity-common patches

* bump a bunch of deps in parity-common

* Remove parity-common patches

* Bump finality-grandpa version

* Cargo.lock

* Update scale-info to 0.9.1

* Add recursion_limit for runtime-parachains

* Add some scale_info attributes

* Cargo.lock

* Revert finality-grandpa bump

* Cargo.lock, scale-info update

* cargo update

* Make sure using patched version of finality-grandpa

* Use patched scale-info

* Update to scale-info 0.10.0

* Update finality-grandpa

* Cargo.lock

* Update beefy deps

* Update beefy deps again

* Add scale-info dependency

* Remove deprecated pallet::metadata attributes.

* Add some missing scale-info deps and derives

* Use some variant struct call syntax

* Add missing TypeInfo impl

* Add some more TypeInfo impls

* Convert some call enum struct variant constructors

* More scale-info deps and derives

* Call enum struct variants

* TypeInfo derives

* Call enum variant structs

* scale-info deps and derives

* Call enum variant struct constructors

* Use beefy-primitives scale-info feature

* Use grandpa-bridge-gadget master branch

* Remove finality-grandpa patch

* Add missing scale_info dependency and derive

* Fix up some call variant constructors

* Add missing scale_info dependency

* Fix some test errors

* More TypeInfo derives

* More call variant structs

* Call variant structs in tests

* Cargo.lock

* Fmt

* Fix more call struct variants

* Another call struct variant

* add scale-info/std features explicitly

* More call struct variants

* Add missing scale-info dependency

* Fmt

* review: activate scale-info/std where missing

* Remove some duplicate std feature activation

* review: add scale_info bounds() attr

* More call variant structs

* Remove recursion limit

* Update beefy-primitives

* Update beefy-primitives

* Fix simnet call variant struct errors

* Fmt

* cargo update -p beefy-primitives

* Add some missing TypeInfo derives

* Fix some call variants

* Fix some call variant underscores

* Cargo.lock

* Cargo.lock

* Add missing TypeInfo derive

* Add some more missing TypeInfo derives

* Even more missing TypeInfo derives

* Add TypeInfo derives to new xcm types

* Fmt

* Cargo.lock

* Add missing TypeInfo impls

* Cargo.lock

* More missing TypeInfos

* Fixes

* Cargo.lock

* Cargo.lock

* Add TypeInfo impls to xcm v2

* Update to scale-info 1.0

* Update finality-grandpa 0.14.4, patch for now

* Update beefy

* Remove patched finality-grandpa

* Add TypeInfo impl to Outcome

* Fixes

* Call variant struct

* Call variant struct

* Fix test

* Add TypeInfo impl

* Cargo.lock

* Cargo.lock

* Cargo.lock

* git checkout master Cargo.lock

* update Substrate

* Add missing scale-info features for beefy-primitives

* Fmt

* Remove check for now

* Update beefy-primitives, removes scale-info feature

* Update beefy-primitives again

Co-authored-by: adoerr <0xad@gmx.net>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <info@kchr.de>
2021-09-15 15:38:45 -05:00
Andronik Ordian 33d7f9dd7b make polkadot-runtime optional feature (#3820)
* make polkadot-runtime optional feature

* sprinkle some cfg statements

* ok

* ok, ok

* add CI check

* set -e

* chmod +x

* fixes

* fmt

* nicer compile errors

* Update outdated comments
2021-09-13 10:28:33 +00:00
Sergejs Kostjucenko c918d57fd4 Add vault secrets to puplish-rustdoc job (#3816) 2021-09-09 10:48:20 +03:00
Sergejs Kostjucenko 0ff3015a8a Change pipeline to use Vault (#3722)
* Change pipeline to use Vault
2021-09-08 21:39:09 +03:00
Sergejs Kostjucenko d918d64c48 Fix buildah issue 3500 (#3805) 2021-09-07 14:33:51 +03:00
Martin Pugh e014e53f4a remove check_web_wasm job (#3654) 2021-08-17 20:02:34 +02:00
radupopa2010 fdb1653783 Simnet v8 move tests (#3647)
* Update to simnet v8

* rm duplicate

* add tags also

* revertme: add simnet image version for testing CI

* add info regarding the image names and values

* fix lint error

* one more try to fix the lint

* hardcode simnet version and revertme

* hardcode simnet

* more hardcodding

* I wonder how may silly mistakes I can make

* simnet image name needs to be hardcoded

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* remove revertme changes

* rm unused script

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2021-08-17 16:49:52 +02:00
Denis Pisarev 4e89b6adde CI: return needs (#3625) 2021-08-11 21:55:14 +02:00