Commit Graph

88 Commits

Author SHA1 Message Date
Vlad f1b34ccd5a Fix cargo-check-benches job (#12301) 2022-09-20 12:15:48 +03:00
Vlad bba345f8ed Use --locked for Cargo in CI everywhere (#12270) 2022-09-15 14:21:17 +03:00
Vlad 66e9f4925a Don't run cargo-check-benches for master based downstream staging pipelines (#12233) 2022-09-12 20:48:10 +03:00
Bastian Köcher 5647e71947 construct_runtime: Fix generation of types behind features (#12229)
* construct_runtime: Fix generation of types behind features

With the recent addition of supporting features in `construct_runtime!` there was a bug overseen.
The `AllPalletsWithSystem` etc type declarations would be declared twice when a certain was enabled.
The problem was that in the macro we didn't feature gate the types that should be declared when
there is no feature enabled. This pull request now takes care of feature gating this type behind
`all(#( not(feature) ))`. So, these types will only be enabled if no of the configured features is enabled.

* Fix tests

* FMT
2022-09-10 21:50:59 +02:00
Vlad 6c5d4668d0 Fully remove rusty-cachier from the node-bench-regression-guard job (#11904)
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
2022-09-08 15:35:44 +03:00
Bastian Köcher 9d75f3e3e7 Fix benchmarks and adds CI to test them (#12068)
* Fix benchmarks and adds CI to test them

Instead of waiting for benchmarks failing in Polkadot CI, we also can just test them in Substrate :P

* Do not overflow
2022-08-19 13:07:14 +02:00
Alexander Samusev db2d66e59b [ci] Improve cancel-pipeline job (#12008)
* [WIP][ci] Improve cancel-pipeline job

* fix job name

* test that fail works

* debug cancel-pipeline

* remove artifacts-false from cancel-pipeline jobs

* split cancel pipeline jobs

* fail test-linux-stable 2/3

* fail test-linux-stable 3/3

* fail cargo-check-benches 1/2

* fail cargo-check-benches 2/2

* fail test-linux-stable-int

* fail cargo-check-subkey

* fail check-tracing

* fail check-tracing

* fix pipeline
2022-08-12 11:03:27 +02:00
Nikos Kontakis 103f770e75 Rename node-runtime to node-kitchensink-runtime (#11930)
* Rename node=runtime to kithensink-runtime

* Undo md formatting
2022-08-02 15:25:52 +00:00
Alexander Samusev 2e8249dc17 [ci] chores: remove cargo install nextest (#11961) 2022-08-02 15:01:15 +02:00
Mak 135c584358 Integrate automatic update of substrate-node-template (#11931)
* Integrate automatic update of substrate-node-template

* Update scripts/ci/gitlab/pipeline/publish.yml

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
2022-07-29 12:16:30 +00:00
Alexander Samusev 151c5d3fd9 [ci] remove cargo-check-nixos job (#11873)
* [ci] remove cargo-check-nixos job

* remove shell.nix
2022-07-26 11:37:54 +02:00
Alexander Samusev 3753d70e45 [ci] fix node-bench-regression-guard job (#11901)
* [Do not merge] [ci] debug node-bench-regression-guard

* debug

* fix artifacts path

* add debug job

* debug

* debug

* fix job, return pipeline
2022-07-25 11:15:47 +02:00
Alexander Samusev c3812f409c [ci] Generate rustdocs without dependencies (#11885) 2022-07-21 15:35:18 +02:00
Alexander Samusev ed346d97c5 [ci] improvments to make pipeline faster (#11829)
* [DO NOT MERGE] test-linux-stable parallel on 3 ci nodes

* add debug message

* adjust rusty-cachier

* empty commit

* move test-linux-stable to test.yml

* make cargo-check-benches and test-wasmer-sandbox parallel

* fix comment

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* if to case

* use case instead if in cargo-check-benches

* format

* add comments to output

* add comment

* add quotes

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

Co-authored-by: parity-processbot <>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2022-07-15 10:44:46 +02:00
Vlad b357ecd467 Explain why rusty-cachier is put first (#11740) 2022-06-23 17:54:15 +03:00
Vlad 2d9f187364 Put rusty-cachier before PR merge into master for cargo-check-benches job (#11737) 2022-06-22 19:00:00 +03:00
João Paulo Silva de Souza 94a7e278fa check-dependent-cumulus should only be executed for PRs (#11693)
the script executed by check-dependent-cumulus only works for PRs, as shown in https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1630771#L87, which comes from https://github.com/paritytech/pipeline-scripts/blob/3ad10ddc0d985ef5326974a1143229c6429befab/check_dependent_project.sh#L443
2022-06-20 12:18:06 +00:00
Oliver Tale-Yazdi eb4be02155 Expose Benchmarking Component Ranges (#11545)
* Add component ranges to benchmarking

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Adding component ranges to templates

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/system/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* tweak script to reduce diff

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_identity --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/identity/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-06-03 20:43:54 +00:00
Alexander Samusev 98f6d8e96c [ci] use cargo nextest instead cargo test in test-linux-stable (#11576)
* [DO NOT MERGE] Experimenting with nextest

* enable jobs

* enable stages

* add comment

* create test-frame-support job
2022-06-02 21:18:59 +02:00
João Paulo Silva de Souza 3977de095b reactivate check-dependent-cumulus (#11506)
* re-enable check-dependent-cumulus

* temporary: use handle-extra-dependencies

* temporary: trim CI

* CI: include build stage

* CI: include test stage

* CI: include test stage

* Revert "temporary: trim CI"

This reverts commit dcf4ae8d842bc445a065c7ccdc3b6a603034faa4.

* CI: fix weird revert

* Revert "temporary: use handle-extra-dependencies"

This reverts commit bc0dc0f21f10284a23f66fdd8509ca6df89f2586.

* CI undebug

Co-authored-by: TriplEight <denis.pisarev@parity.io>
2022-06-02 16:15:32 +02:00
Vlad a084dd684f CI: fix build-subkey-macos build job (#11573)
* CI: fix `build-subkey-macos` build job

* CI: use full path for the `CARGO_TARGET_DIR` default value
2022-06-02 11:48:12 +03:00
Vlad 7d8477b48d rusty-cachier pipeline impovements and fixes (#11572) 2022-06-01 19:48:53 +03:00
Bastian Köcher bb6bbab687 Fix all warnings when building for wasm (#11569)
* Fix all warnings when building for wasm

Besides that it also enables warnings as errors for wasm builds in the CI.

* FMT

* Make clippy happy
2022-06-01 13:03:21 +00:00
Vlad a4fdcb9a06 Don't limit test-linux-stable-int job output (#11560) 2022-06-01 14:28:24 +03:00
Vlad c91c1c793e Introduce rusty-cachier (#11462)
* Introduce `rusty-cachier`

* Return LF at the end of file

* Use `entrypoint` to `unshare(1)` into a new mount namespace

* Use `rusty-cachier`-provided absolute path for `CARGO_TARGET_DIR` everywhere

* Debug single `build-rustdoc` job

* CI: debug

* CI: debug

* CI: debug

* `unshare(1)` is no longer needed

* CI: remove debug

* Revert "Debug single `build-rustdoc` job"

* Formatiing

* Update scripts/ci/gitlab/pipeline/build.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2022-05-31 17:10:04 +02:00
Sergejs Kostjucenko 6198a5fb06 add rule to the ci job (#11511) 2022-05-24 13:10:13 +03:00
Shawn Tabrizi d9f33dc7e9 Create Script to Run All Benchmarks (#11493)
* Create run_all_benchmarks.sh

* Update run_all_benchmarks.sh

* Update run_all_benchmarks.sh

* Review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update scripts/run_all_benchmarks.sh

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* typo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add default for $1

* Typo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update run_all_benchmarks.sh

* new weights on benchmarking machine

* prefer `--chain=dev`

* fix compile

* fix command

* fmt

* dont use square brackets

* Extend doc

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove +nightly

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error file an run execute everything optimistically

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-05-23 16:56:42 +00:00
João Paulo Silva de Souza 9dd9a98eff disable check-dependent-cumulus (#11450) 2022-05-18 11:35:15 +00:00
Alexander Samusev ce77cb1735 [ci] Adjust job order in pipeline test stage with Gitlab DAG (#11442)
* [Do Not Merge] Test gitlab DAG in pipeline

* add jobs for pipeline cancel

* add check-tracing to cancel-pipeline
2022-05-18 13:14:25 +02:00
Alexander Samusev 52d9bf24cc [ci] Divide .gitlab-ci.yml into several files (#11333)
* rebase

* update Cargo.lock

* revert Cargo.lock

* fix Cargo.lock from 11384
2022-05-11 11:01:02 +02:00
Shawn Tabrizi 7d5202341b Basic TOML Lint (#11348)
* basic lint

* lint ordering
2022-05-04 13:38:54 +00:00
Sergejs Kostjucenko 46891e849f Move scripts used in CI to the new location (#11008)
Move scripts used in CI to the new location - **./scripts/ci/**

* Move github scripts

* Move more files

* Move ci scripts and fix dependencies

* Update docs/node-template-release.md

Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>

* Remove Cargo.lock

* Apply suggestions from code review

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Make more paths uniform

Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2022-03-14 10:42:34 +02:00
Benjamin Kampmann 327e4ad4ac do not check unleash on every PR, only master and tags (#5054)
* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <pugh@s3kr.it>
2020-02-25 20:02:57 +01:00
s3krit 1e3e6a75f9 Add gitlab job for publishing draft releases (#5009)
Idea is much the same as we currently do on polkadot - When a new tag is pushed (that fits our pattern for tags, e.g. v1.2.3), find a list of labelled changes and generate a changelog based on that. Create a draft release on Github and post about it on Matrix.
2020-02-21 13:08:45 +01:00
Benjamin Kampmann 60e5011c72 Reorganising the repository - external renames and moves (#4074)
* Adding first rough ouline of the repository structure

* Remove old CI stuff

* add title

* formatting fixes

* move node-exits job's script to scripts dir

* Move docs into subdir

* move to bin

* move maintainence scripts, configs and helpers into its own dir

* add .local to ignore

* move core->client

* start up 'test' area

* move test client

* move test runtime

* make test move compile

* Add dependencies rule enforcement.

* Fix indexing.

* Update docs to reflect latest changes

* Moving /srml->/paint

* update docs

* move client/sr-* -> primitives/

* clean old readme

* remove old broken code in rhd

* update lock

* Step 1.

* starting to untangle client

* Fix after merge.

* start splitting out client interfaces

* move children and blockchain interfaces

* Move trie and state-machine to primitives.

* Fix WASM builds.

* fixing broken imports

* more interface moves

* move backend and light to interfaces

* move CallExecutor

* move cli off client

* moving around more interfaces

* re-add consensus crates into the mix

* fix subkey path

* relieve client from executor

* starting to pull out client from grandpa

* move is_decendent_of out of client

* grandpa still depends on client directly

* lemme tests pass

* rename srml->paint

* Make it compile.

* rename interfaces->client-api

* Move keyring to primitives.

* fixup libp2p dep

* fix broken use

* allow dependency enforcement to fail

* move fork-tree

* Moving wasm-builder

* make env

* move build-script-utils

* fixup broken crate depdencies and names

* fix imports for authority discovery

* fix typo

* update cargo.lock

* fixing imports

* Fix paths and add missing crates

* re-add missing crates
2019-11-14 21:51:17 +01:00
Yuanchao Sun 9515d34aee Also replace paths in [build-dependencies] (#4039)
* Also replace paths in [build-dependencies]

* Update scripts/node-template-release/src/main.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Slightly reduce memory usage

* Update scripts/node-template-release/src/main.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
2019-11-07 11:48:30 +01:00
Ashley 934d7aac1c Remove sr-arithmetic/fuzzer from workspace to fix windows builds (#3872)
* Remove sr-arithmetic/fuzzer from workspace to fix windows builds

* Remove sr-arithmetic/fuzzer from check_runtime.sh
2019-10-22 13:24:58 +02:00
Andrew Dirksen 470b62366f Change manual dependency on wasm-gc executable to an automatic cargo dependency. (#3854)
Improves user experience.
2019-10-19 10:38:19 +02:00
André Silva 6e5fe1a557 deps: update clap and structopt (#3809) 2019-10-12 18:31:49 +01:00
Bastian Köcher 875fa73728 Fix build-only-wasm script (#3536) 2019-09-03 14:13:32 +01:00
Bastian Köcher 4ff97bd856 Make it easier to just rebuild the WASM files (#3510)
* Adds `WASM_TARGET_DIRECTORY` env variable to `wasm-builder`

* Create the `wasm-builder-runner`'s in a common workspace

* Make `wasm-builder` trigger less rebuilds

* Version up

* Adds script for building only the WASM files

* Apply suggestions from code review

Co-Authored-By: André Silva <andre.beat@gmail.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
2019-08-29 09:06:51 +02:00
Bastian Köcher d81df14391 Make node-template-release create a workspace (#3484) 2019-08-27 07:43:10 +02:00
ddorgan 6815ce59fe Fix artifact location required for flaming fir deployments (#3268) 2019-07-31 17:49:04 +01:00
TriplEight d98f3c6023 WIP: Subkey docker image (#3183)
* WIP to release subkey im docker

* debug run with publishing

* debug build

* debug build 2

* easier sync

* fix the template

* fix typo
2019-07-26 08:40:53 +08:00
Max Inden ab761f7a09 scripts/sentry-node: Ensure release mode and reserved connection for A (#3077)
Ensure all nodes in the sentry setup run in release mode. In addition
configure sentry-a as a reserved node for validator-a.
2019-07-10 10:15:23 +01:00
Max Inden 6639ab339a Introduce flag to enable sentry nodes to participate in grandpa gossip (#3018)
Given the following situation: A validator 'A' is not supposed to be
connected to the public internet to protect it from e.g. a DoS attack.
Instead it connects to a sentry node 'sentry-A' which connects to the
public internet. Validator 'B' can reach validator 'A' via sentry node
'sentry-A' and vice versa.

A sentry node needs to participate in the grandpa gossip without
becoming a validator itself in order to forward these message to its
validator. This commit adds a new command line flag (`--grandpa-voter`)
forcing a node to participate in the grandpa voting process even though
no `--key` was specified. Due to the fact that it does not have a key,
it does not become a validator in the network.

In order to simulate the above situation this commit also adds a Docker
Compose file (`scripts/sentry-node/docker-compose.yml`) with further
documentation.
2019-07-09 18:35:37 +02:00
Bastian Köcher ae80c05232 Fix node-template-release and make cargo doc faster (#3019) 2019-07-04 16:26:03 +02:00
Bastian Köcher fe08221479 Build WASM binaries as part of cargo build (#2868)
* Introduce `wasm-builder` and `wasm-builder-runner` to retire `build.sh`

Make use of `wasm-builder` in `test-runtime`.

* Add build script and remove the wasm project

* Port `node-runtime` to new wasm-builder

* Make `substrate-executor` tests work with `wasm-builder`

* Move `node-template` to `wasm-builder`

* Remove `build.sh` :)

* Remove the last include_bytes

* Adds the missing build.rs files

* Remove `build.sh` from CI

* Debug CI

* Make it work in CI

* CI attempt 3

* Make `substrate-runtime-test` compile on stable

* Ahhh, some missed `include_bytes!`

* AHH

* Add suggestions

* Improve search for `Cargo.lock` and don't panic if it is not found

* Searching from manifest path was no good idea

* Make the `wasm-builder` source better configurable

* Expose the bloaty wasm binary as well

* Make sure to rerun WASM recompilation on changes in dependencies

* Introduce new `WASM_BUILD_TYPE` env and make sure to call `build.rs` on
changes to env variables

* Remove `build.sh` from READMEs

* Rename the projects

* Fixes CI

* Update lock file

* Fixes merge-conflict

* Apply suggestions from code review

Co-Authored-By: TriplEight <denis.pisarev@parity.io>

* Try to make windows happy

* Replace all back slashes in paths with slashes

* Apply suggestions from code review

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Use cargo from `CARGO` env variable

* Fix compilation

* Use `rustup` for running the nightly build

* Make individual projects skipable

* Fix compilation

* Fixes compilation

* Build all WASM projects in one workspace

* Replace more back slashes!

* Remove `inlcude_bytes!`

* Adds some documentation

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* More review comments

* Update `Cargo.lock`

* Set license

* Apply suggestions from code review

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* More review comments + adds `TRIGGER_WASM_BUILD` env

* Fix doc tests

* Increase version + update README

* Switch crates.io version of `wasm-builder`

* Update README

* Switch to released version of `wasm-builder-runner`
2019-07-04 11:34:06 +02:00
David Craven b2622b611e Don't label PR's with gotissues. (#2993) 2019-07-02 12:42:58 +02:00
Lio李欧 66ae3a57ff Passthru build.sh/script.sh command line arguments (#2837) 2019-06-11 16:00:27 +02:00