* more improvements for the crate publishing pipeline
* move default definitions to the publish-crates script
* add script to check the crate publishing pipeline at the start
* fix yaml references
* move more variables to .crates-publishing-pipeline
* separate .crates-publishing-pipeline from .crates-publishing-variables
* clean up redundant and unused code
* Update UI tests for 1.66
* Fix `test_enum` assertion for Rust 1.66
* Fix another `test_enum` assertion for Rust 1.66
* Fix another `test_enum` assertion for Rust 1.66
* Fix another `test_enum` assertion for Rust 1.66
* avoid unintentionally canceling the scheduled crate publishing job
because publish-crates and publish-crates-manual share the resource group
"crates-publishing", any instance of publish-crates-manual cancels a running
instance of publish-crates, as demonstrated by
https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179
a workaround for that unintended interaction is to avoid creating instances of
publish-crates-manual and instead require pipelines to be triggered manually by
checking $CI_JOB_MANUAL == "true"
* check manual pipelines by $CI_PIPELINE_SOURCE instead of $CI_JOB_MANUAL
* make crate-publishing pipelines uninterruptible
* use conditional includes to work around interruptible limitations
* organize comments
* remove interruptible from common pipeline
* wip: check include
* wip: check include
* fix include
* fix include
* fix include
* fix yaml
* fix yaml
* remove shared common-pipeline
* wip: retry common-pipeline
* move .default-template to .gitlab-ci.yml
* fix the pipeline
add comments
* fix default-pipeline.yml
* revert publish-crates-manual to when: manual
* move "needs:" back to publish-crates
* avoid manual repetition
* improve previous commit
* try to avoid manual repetition
* fix indentation
* minor adjustments
* move defaults to top of .gitlab-ci.yml
* fix positioning on default in the diff
* comments
* indentation
* Apply suggestions from code review
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
* zombienet: warp-sync integration test added
* spelling
* Readme corrected
* dir name updated
* Check second phase of warp sync
* zombienet pipeline enable + naive test network
* zombienet stage added
* paritypr/substrate-debug image added for zombienet testing
* debugs added
* debugs added
* buildah problem fixed
* rollback
* runner tag
* test name corrected
* dir renamed (regex problem)
* common code clean up
* common code clean up
* fix
* warp sync test improvements
* full sha used
as short is too short (https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2051228#L38)
* disable tracing for nodes
* COMMON_USER -> DOCKERIO_USER
* refs reworked
* paritypr/substrate image used
* DOCKERIO -> DOCKER
* generate-ws-db toml cleanup
* improvements
* fix
* raw chain spec used
* zombienet v1.3.18 used
* zombienet: warp sync test enabled
* chain-spec path corrected
* log parsing improved
Checking if log does not container error or verification failed messages
* warp sync test: removed validators
* fix
* review remarks applied
* dir test name changed: 0000_block_building -> 0000-block-building
* transaction finalized test added
* transaction finalized test: error handling improved
* trigger CI job
* trigger CI job
* trigger CI job
* trigger CI job
* Explicitly touch `version.rs` to invalidate the related cache
* zombienet add logs as artifacts
* Revert "Explicitly touch `version.rs` to invalidate the related cache"
This reverts commit 9d00ccfe897a280581156c281961a32665dba6d5.
* file naming changed
Co-authored-by: parity-processbot <>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
* implement crate publishing from CI
* fix indentation
* use resource_group for job exclusivity
ensure that at most one instance of the publish-crates job is running at any given time to prevent race conditions
* correct publish = false
* Remove YAML anchors as GitLab's `extends:` doesn't need it
* Temporarily force cache upload for the new jobs
* Revert `RUSTY_CACHIER_FORCE_UPLOAD`
* pin libp2p-tcp=0.37.0 for sc-telemetry
* Revert "pin libp2p-tcp=0.37.0 for sc-telemetry"
This reverts commit 29146bfad6c31e8cf0e2f17ad92a71bb81a373af.
* always collect generated crates
* increase timeout for publish-crates-template
* Force upload the new job cache again
* Revert "Force upload the new job cache again"
This reverts commit 5a5feee1b2c51fdef768b25a76be4c3949ec1c99.
* reformat
* improve timeout explanation
* s/usual/average
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
* CI: Explicitly unset RUSTC_WRAPPER=sccache environment variable
* Try with `rusty-cachier` disabled
* Re-enable `rusty-cachier` and try with the staging image
* Bring back `production` image
* Sort crates before splitting them into groups (+ some improvements) (#12755)
* sort crates before splitting them into groups
this is useful so that crates always get routed to a specific group for a given version of the source code, which means that jobs for each batch can be reliably retried individually
* more verbose output
* misc improvements
* put uniq after sort
uniq filters by adjacent lines
* shellcheck
* rm useless backlashes
* handle edge case of no crates detected
* Revert "Sort crates before splitting them into groups (+ some improvements) (#12755)"
This reverts commit fde839183a12a2bd51efc7143ebcddeed81ea6fa.
Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
* check all crates individually
It's relevant to check workspace crates individually because otherwise their compilation problems
due to feature misconfigurations won't be caught, as exemplified by
https://github.com/paritytech/substrate/issues/12705
* adapt to lack of multiple macos runners
https://github.com/paritytech/substrate/pull/12709#discussion_r1022868752
* fix cancel-pipeline-cargo-check-each-crate-macos
* fix cargo-check-each-crate-macos again
* time command execution
* fix YAML anchors
* add explanation for rounding division
* ensure the minimum of one crate per group
* collect artifacts for pipeline stopper
* revert hardcoded crates_per_group
* re-add crates_per_group=1
* Update UI test output for rust 1.62.1
* switch ci to staging image to check that everything works
* fix artifacts node-bench-regression-guard
* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning
* Fix more clippy lints
* Make clippy happy by relying on auto-deref were possible
* Add tracking issue to the comments
* pin ci image
Co-authored-by: alvicsam <alvicsam@gmail.com>
* Introduce `rusty-cachier`
* Return LF at the end of file
* Use `entrypoint` to `unshare(1)` into a new mount namespace
* Use `rusty-cachier`-provided absolute path for `CARGO_TARGET_DIR` everywhere
* Debug single `build-rustdoc` job
* CI: debug
* CI: debug
* CI: debug
* `unshare(1)` is no longer needed
* CI: remove debug
* Revert "Debug single `build-rustdoc` job"
* Formatiing
* Update scripts/ci/gitlab/pipeline/build.yml
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* The 2022-02-10 nightly can't build some deps
There's a sig that returns `impl IntoIterator<Item = (&'static str, OsString)>` and an Option is being retured but it's incorrectly not allowing a `?` in the method.
* removing duplicate test
* [ci] add job cargo-check-nixos
* add dummy variables nix check
* fix check-dependent jobs
* fix check-dependent-project template
Co-authored-by: alvicsam <alvicsam@gmail.com>
This tries to fix the CI if there is no polkadot companion. Currently we don't update Polkadot
master in Cumulus, which means we may use some old commit that isn't compiling with the latest
Substrate master anymore. This can happen if there was a pr that had a companion in Polkadot, but no
companion was required for Cumulus. Then Cumulus will still point to some old Polkadot commit that
isn't compiling anymore with the latest Substrate commit. So, we need to tell the script to use the
latest master of Polkadot. If there is a companion for Polkadot, it would simply override the extra
dependency patch later on.
* Prepare for rust stable 1.59
Besides preparing the UI tests this also adds a new script update-rust-stable.sh script for
simplifying the update of a rust stable version. This script will run all UI tests for the new
rust stable version and updating the expected output.
* Ensure we run the UI tests in CI
* use staging ci image
* More test updates
* Unignore test (#11097)
* empty commit for pipeline rerun
* empty commit for pipeline rerun
* Try to make clippy happy
* More clippy fixes
* FMT
* ci image production
Co-authored-by: alvicsam <alvicsam@gmail.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Javier Viola