* Pin ci-linux image for rust 1.69
* Update ui tests for rust 1.69
* Address new rust 1.69 clippy lints
* `derive_hash_xor_eq` has been renamed to `derived_hash_with_manual_eq`
* The new `extra-unused-type-parameters` complains about a bunch of
callsites where extraneous type parameters are used for consistency
with other functions.
* Restore `check-crates-publishing-pipeline` job
This job was errorneously deleted in lieu of the
`check-crates-publishing` job.
This reverts commit 202836cb47.
* Don't run `check-crate-publishing` job on prs.
Originally attempted in #14044. It's been broken for a while and we're
not sure yet how to move forward in the longer term.
* ci: remove the cargo-deny job
It's been broken and disabled for quite a while, and per internal
discussion there doesn't seem to be any interest in fixing it and
actually using the job output for anything.
* ci: add new cargo-deny-licenses job
It'll run on all PRs and fail if external dependencies with unsuitable
licenses are detected.
* Use a new approach to make automatic-crate-publishing uninterruptible
It's not neccessary to dynamically change the `interruptible` setting
for *all* jobs as the old approach does; gitlab already considers a
pipeline uninterruptible as soon as a single uninterruptible job has
started (cf https://docs.gitlab.com/ee/ci/yaml/#interruptible).
IMO this approach is more readable, as it avoids dynamically loading the
.defaults section from different files based on import conditions; the
logic is now shorter and entirely contained in the main .gitlab-ci.yml.
* Make triggered multi-project pipelines uninterruptible
This check is meant to catch pipelines triggered by the scripts ci-linux
staging tests. The correct CI_PIPELINE_SOURCE for multi-project
pipelines such as this is "pipeline"; "parent_pipeline" is only set for
child pipelines triggered /within the same repo/.
cf https://docs.gitlab.com/ee/ci/jobs/job_control.html#common-if-clauses-for-rules
* timestamp gitlab ci job outputs
Based on previous work by @alvicsam in #13047.
* inline timestamp script
Some of our jobs don't check out the substrate repo.
* include .timestamp in pipelines overriding the default before_script
Still not including it in the zombienet jobs, they have their own
timestamping anyway.
* move timestamp.yml to shared pipeline repo
https://gitlab.parity.io/parity/infrastructure/ci_cd/shared
* bump zombienet version to v1.3.35
* remove checks, since now is checked by zn
* Update .gitlab-ci.yml
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* [ci] Deduplicate variables: sections in pipeline specs
The prettier yaml parser doesn't like these.
* [ci] provide git clean filter to format pipeline specs
* [ci] Reformat pipeline specs with prettier
* more improvements for the crate publishing pipeline
* move default definitions to the publish-crates script
* add script to check the crate publishing pipeline at the start
* fix yaml references
* move more variables to .crates-publishing-pipeline
* separate .crates-publishing-pipeline from .crates-publishing-variables
* clean up redundant and unused code
* Update UI tests for 1.66
* Fix `test_enum` assertion for Rust 1.66
* Fix another `test_enum` assertion for Rust 1.66
* Fix another `test_enum` assertion for Rust 1.66
* Fix another `test_enum` assertion for Rust 1.66
* avoid unintentionally canceling the scheduled crate publishing job
because publish-crates and publish-crates-manual share the resource group
"crates-publishing", any instance of publish-crates-manual cancels a running
instance of publish-crates, as demonstrated by
https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179
a workaround for that unintended interaction is to avoid creating instances of
publish-crates-manual and instead require pipelines to be triggered manually by
checking $CI_JOB_MANUAL == "true"
* check manual pipelines by $CI_PIPELINE_SOURCE instead of $CI_JOB_MANUAL
* make crate-publishing pipelines uninterruptible
* use conditional includes to work around interruptible limitations
* organize comments
* remove interruptible from common pipeline
* wip: check include
* wip: check include
* fix include
* fix include
* fix include
* fix yaml
* fix yaml
* remove shared common-pipeline
* wip: retry common-pipeline
* move .default-template to .gitlab-ci.yml
* fix the pipeline
add comments
* fix default-pipeline.yml
* revert publish-crates-manual to when: manual
* move "needs:" back to publish-crates
* avoid manual repetition
* improve previous commit
* try to avoid manual repetition
* fix indentation
* minor adjustments
* move defaults to top of .gitlab-ci.yml
* fix positioning on default in the diff
* comments
* indentation
* Apply suggestions from code review
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
* zombienet: warp-sync integration test added
* spelling
* Readme corrected
* dir name updated
* Check second phase of warp sync
* zombienet pipeline enable + naive test network
* zombienet stage added
* paritypr/substrate-debug image added for zombienet testing
* debugs added
* debugs added
* buildah problem fixed
* rollback
* runner tag
* test name corrected
* dir renamed (regex problem)
* common code clean up
* common code clean up
* fix
* warp sync test improvements
* full sha used
as short is too short (https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2051228#L38)
* disable tracing for nodes
* COMMON_USER -> DOCKERIO_USER
* refs reworked
* paritypr/substrate image used
* DOCKERIO -> DOCKER
* generate-ws-db toml cleanup
* improvements
* fix
* raw chain spec used
* zombienet v1.3.18 used
* zombienet: warp sync test enabled
* chain-spec path corrected
* log parsing improved
Checking if log does not container error or verification failed messages
* warp sync test: removed validators
* fix
* review remarks applied
* dir test name changed: 0000_block_building -> 0000-block-building
* transaction finalized test added
* transaction finalized test: error handling improved
* trigger CI job
* trigger CI job
* trigger CI job
* trigger CI job
* Explicitly touch `version.rs` to invalidate the related cache
* zombienet add logs as artifacts
* Revert "Explicitly touch `version.rs` to invalidate the related cache"
This reverts commit 9d00ccfe897a280581156c281961a32665dba6d5.
* file naming changed
Co-authored-by: parity-processbot <>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
* implement crate publishing from CI
* fix indentation
* use resource_group for job exclusivity
ensure that at most one instance of the publish-crates job is running at any given time to prevent race conditions
* correct publish = false
* Remove YAML anchors as GitLab's `extends:` doesn't need it
* Temporarily force cache upload for the new jobs
* Revert `RUSTY_CACHIER_FORCE_UPLOAD`
* pin libp2p-tcp=0.37.0 for sc-telemetry
* Revert "pin libp2p-tcp=0.37.0 for sc-telemetry"
This reverts commit 29146bfad6c31e8cf0e2f17ad92a71bb81a373af.
* always collect generated crates
* increase timeout for publish-crates-template
* Force upload the new job cache again
* Revert "Force upload the new job cache again"
This reverts commit 5a5feee1b2c51fdef768b25a76be4c3949ec1c99.
* reformat
* improve timeout explanation
* s/usual/average
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
* CI: Explicitly unset RUSTC_WRAPPER=sccache environment variable
* Try with `rusty-cachier` disabled
* Re-enable `rusty-cachier` and try with the staging image
* Bring back `production` image
* Sort crates before splitting them into groups (+ some improvements) (#12755)
* sort crates before splitting them into groups
this is useful so that crates always get routed to a specific group for a given version of the source code, which means that jobs for each batch can be reliably retried individually
* more verbose output
* misc improvements
* put uniq after sort
uniq filters by adjacent lines
* shellcheck
* rm useless backlashes
* handle edge case of no crates detected
* Revert "Sort crates before splitting them into groups (+ some improvements) (#12755)"
This reverts commit fde839183a12a2bd51efc7143ebcddeed81ea6fa.
Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
* check all crates individually
It's relevant to check workspace crates individually because otherwise their compilation problems
due to feature misconfigurations won't be caught, as exemplified by
https://github.com/paritytech/substrate/issues/12705
* adapt to lack of multiple macos runners
https://github.com/paritytech/substrate/pull/12709#discussion_r1022868752
* fix cancel-pipeline-cargo-check-each-crate-macos
* fix cargo-check-each-crate-macos again
* time command execution
* fix YAML anchors
* add explanation for rounding division
* ensure the minimum of one crate per group
* collect artifacts for pipeline stopper
* revert hardcoded crates_per_group
* re-add crates_per_group=1
Mira Ressel