Commit Graph

59 Commits

Author SHA1 Message Date
Alexander Samusev 9a4e66c8a9 Add --workspace to clippy (#14772)
* [DNM] Debug clippy

* add --workspace
2023-08-15 17:21:53 +02:00
Piotr Mikołajczyk b24b66c991 Cross-contract calling: simple debugger (#14678)
* Provide basic breakpoints

* Rename to Observer

* Rename feature. Single trait. Borrow-checker

* : frame_system::Config

* Confused type name

* Minor bugs

* pub trait

* No unnecessary cloning

* Make node compile with all features

* Move everything debug-related to a single module

* Add docs and implementation for ()

* fmt

* Make it feature-gated or for tests

* Prepare testing kit

* Testcase

* Fmt

* Use feature in dev-deps

* ?

* feature propagation

* AAAA

* lol, that doesn't make much sense to me

* Turn on

* clippy

* Remove self dep

* fmt, feature-gating test

* Noop to trigger CI

* idk

* add feature to pipeline

* Corrupt test to see if it is actually being run

* Revert change

* Doc for conf type

* Review

* Imports

* ...

* Remove debug for kitchen-sink

* Move test

* Fix imports

* I must have already tried this one...
2023-08-05 14:15:03 +00:00
Kian Paimani 817c97d65d add doc-only substrate entry point crate (#14581)
* add doc-only substrate entry point crate

* document a few more things

* add more

* fix width

* Update primitives/io/src/lib.rs

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>

* add link

* update cargo toml file

* fix sp-io docs

* improve

* small update

* add license

* satisfy license job

* add a line about FRAME

* CI happy now

* make CI more happy

* Let the check run for the whole workspace

* Forward the substrate node again as default run

* update binary names

* upate verison test

* Fix fix fix

* Fix

* rename to substrate-node in more places

* Revert "rename to substrate-node in more places"

This reverts commit 66960f84a1b6f1f7c638b4040e28e9fbabb8adf5.

* fix

* Fix build pipeline

* Fix properly plus add some docs

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
2023-07-19 18:36:50 +00:00
Bastian Köcher 5eb816d7a6 Removal of execution strategies (#14387)
* Start

* More work!

* Moar

* More changes

* More fixes

* More worrk

* More fixes

* More fixes to make it compile

* Adds `NoOffchainStorage`

* Pass the extensions

* Small basti making small progress

* Fix merge errors and remove `ExecutionContext`

* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`

Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.

* Fix compilation

* Register the global extensions inside runtime api instance

* Fixes

* Fix `generate_initial_session_keys` by passing the keystore extension

* Fix the grandpa tests

* Fix more tests

* Fix more tests

* Don't set any heap pages if there isn't an override

* Fix small fallout

* FMT

* Fix tests

* More tests

* Offchain worker custom extensions

* More fixes

* Make offchain tx pool creation reusable

Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.

* Fixes

* Fixes

* Set offchain transaction pool in BABE before using it in the runtime

* Add the `offchain_tx_pool` to Grandpa as well

* Fix the nodes

* Print some error when using the old warnings

* Fix merge issues

* Fix compilation

* Rename `babe_link`

* Rename to `offchain_tx_pool_factory`

* Cleanup

* FMT

* Fix benchmark name

* Fix `try-runtime`

* Remove `--execution` CLI args

* Make clippy happy

* Forward bls functions

* Fix docs

* Update UI tests

* Update client/api/src/execution_extensions.rs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/cli/src/params/import_params.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Pass the offchain storage to the MMR RPC

* Update client/api/src/execution_extensions.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Review comments

* Fixes

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-07-11 14:21:38 +00:00
Bastian Köcher a2b01c061b CI: Do not fail on junit upload failure (#14545)
* CI: Do not fail on junit upload failure

* fix yaml for test-linux-stable

---------

Co-authored-by: alvicsam <alvicsam@gmail.com>
2023-07-11 11:26:44 +02:00
Liam Aharon cfe5e8a8df run tests with --experimental flag (#14502) 2023-07-06 21:35:55 +00:00
Bastian Köcher 53f4b4df76 Run all tests with try-runtime feature enabled (#14396)
* Run all tests with `try-runtime` feature enabled

There are some tests that are guarded behind the `try-runtime` feature which are currently not being
executed. This pull requests changes this and enables the feature for the test run.

* Add some docs

* Rename variable
2023-06-16 12:16:19 +02:00
Mira Ressel b78473d28a cargo-hfuzz ci: specify manifest patches via cargo --config (#14372)
This is less brittle than locally editing Cargo.toml, which breaks when
our devs submit PRs such as #13992 that also temporarily add a patch
section.
2023-06-14 13:54:58 +02:00
Oleg Plakida 982db4309c Change unit test upload step (#14124)
Co-authored-by: parity-processbot <>
2023-05-16 11:24:54 +01:00
Bastian Köcher a8cd277d1c Re-enable node-cli tests in CI and fix them (#14093)
With the recent change of re-running flaky tests this should be fine!
2023-05-08 16:19:38 +02:00
Oleg Plakida 71a77c7054 Add flawky tests report (#13654) 2023-05-05 13:16:52 +01:00
Bastian Köcher e2547f5064 Improve handling of unset StorageVersion (#13417)
* Improve handling of unset `StorageVersion`

When a user is forgetting to set the storage version in a pallet and calls
`current_storage_version` to compare it against the `on_chain_storage_version` it will now fail to
compile the code. Before the pallet macro just returned `StorageVersion::default()` for
`current_storage_version` leading to potential issues with migrations. Besides that it also checks
in `post_upgrade` that the pallet storage version was upgraded and thus, no migration was missed.

* Use correct `Cargo.lock`

* Fixes

* Fix test

* Update frame/support/test/tests/pallet.rs

* Ensure we don't set a storage version when the pallet is missing the attribute

* Fix merge conflict

* Update frame/support/procedural/src/pallet/expand/hooks.rs

Co-authored-by: Roman Useinov <roman.useinov@gmail.com>

* Update frame/support/procedural/src/pallet/expand/hooks.rs

Co-authored-by: Roman Useinov <roman.useinov@gmail.com>

* Fix compilation

* Do not run everything with `try-runtime`

* Fix test

* Apply suggestions from code review

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix `no-metadata-docs`

---------

Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
2023-05-04 21:18:53 +02:00
Vladimir Istyufeev 93165bc4d2 CI: cargo-check-benches: don't merge in master if base ref isn't master (#14071) 2023-05-04 14:32:16 +04:00
Vladimir Istyufeev bedb497d55 CI: migrate to Google Cloud (#13994) 2023-05-02 13:34:14 +04:00
Alexander Samusev b46a512e1a [ci] Add message to cargo-deny (#14001)
* [ci] Add message to cargo-deny

* fix cargo-deny-licenses
2023-04-25 10:15:57 +02:00
Mira Ressel 9544ec765d replace unused cargo-deny check with a new cargo-deny-licenses job (#13956)
* ci: remove the cargo-deny job

It's been broken and disabled for quite a while, and per internal
discussion there doesn't seem to be any interest in fixing it and
actually using the job output for anything.

* ci: add new cargo-deny-licenses job

It'll run on all PRs and fail if external dependencies with unsuitable
licenses are detected.
2023-04-20 14:03:05 +02:00
Mira Ressel 84c9e2f63a invoke clippy with --locked (#13882) 2023-04-12 12:05:11 +02:00
Mira Ressel 4e32ae1fbe use stable rust toolchain in ci (#13830)
cargo-fmt stays on the nightly pipeline; our fmt config uses a heap of
unstable features.
2023-04-11 17:02:50 +02:00
Vladimir Istyufeev 9c9492476c CI: rephrase RUSTFLAGS (#13735) 2023-03-28 15:41:10 +04:00
Vladimir Istyufeev 551367d4cb CI: Investigate why RUSTFLAGS differs on master and PRs (#13686) 2023-03-28 04:00:18 +04:00
Mira Ressel 6213161e33 add a new ci job to fuzz sp-arithmetic (#13673)
as requested in https://github.com/paritytech/ci_cd/issues/761
2023-03-27 07:23:53 +02:00
Mira Ressel b117dbdf08 refactor cargo-check-benches logic
* fix fetch refspec
* assume base is master if the api call fails
* merge master into pr branch rather than the other way around; same
  effect but imo cleaner and more readable
* don't re-fetch pr branch, we want to test the version our pipeline is
  running on
2023-03-16 15:57:05 +01:00
Mira Ressel 3a9e37c0a7 fix BASE lookup in cargo-check-benches 2023-03-16 15:57:05 +01:00
Mira Ressel 8945c1bd42 timestamp gitlab ci job outputs (#13605)
* timestamp gitlab ci job outputs

Based on previous work by @alvicsam in #13047.

* inline timestamp script

Some of our jobs don't check out the substrate repo.

* include .timestamp in pipelines overriding the default before_script

Still not including it in the zombienet jobs, they have their own
timestamping anyway.

* move timestamp.yml to shared pipeline repo

https://gitlab.parity.io/parity/infrastructure/ci_cd/shared
2023-03-15 16:37:34 +00:00
Mira Ressel 84638524e7 Automatically format ci pipeline specs with prettier (#13441)
* [ci] Deduplicate variables: sections in pipeline specs

The prettier yaml parser doesn't like these.

* [ci] provide git clean filter to format pipeline specs

* [ci] Reformat pipeline specs with prettier
2023-02-24 15:02:30 +00:00
Alexander Samusev d5e1fcc471 [ci] Move publish-crates-locally to publish stage (#13395) 2023-02-16 12:09:32 +01:00
Bastian Köcher 1da71f0226 CI: Rewrite check-each-crate in python (#13238)
* CI: Rewrite `check-each-crate` in python

This is a much better readable version of the script and it should also work on Macos and not
siltently fail ;)

* Fix dumb bugs and print everything to stderr

* Don't buffer Python output

* 🤦

* 🤦 🤦

* Use check all until we have more macos runners

* Update scripts/ci/gitlab/check-each-crate.py

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-01-25 22:49:03 +00:00
JP 21141f5d4c More improvements for the crate publishing pipeline (#13153)
* more improvements for the crate publishing pipeline

* move default definitions to the publish-crates script

* add script to check the crate publishing pipeline at the start

* fix yaml references

* move more variables to .crates-publishing-pipeline

* separate .crates-publishing-pipeline from .crates-publishing-variables

* clean up redundant and unused code
2023-01-16 14:03:46 +00:00
Vladimir Istyufeev 3af93b8a3a CI: Code mark to request a pipeline failure (#13139) 2023-01-13 20:21:44 +04:00
Alexander Samusev 3e0a1904b7 [ci] add job switcher (#12922) 2022-12-14 12:36:33 +01:00
João Paulo Silva de Souza 8751f88fc7 Implement crate publishing on CI (#12768)
* implement crate publishing from CI

* fix indentation

* use resource_group for job exclusivity

ensure that at most one instance of the publish-crates job is running at any given time to prevent race conditions

* correct publish = false

* Remove YAML anchors as GitLab's `extends:` doesn't need it

* Temporarily force cache upload for the new jobs

* Revert `RUSTY_CACHIER_FORCE_UPLOAD`

* pin libp2p-tcp=0.37.0 for sc-telemetry

* Revert "pin libp2p-tcp=0.37.0 for sc-telemetry"

This reverts commit 29146bfad6c31e8cf0e2f17ad92a71bb81a373af.

* always collect generated crates

* increase timeout for publish-crates-template

* Force upload the new job cache again

* Revert "Force upload the new job cache again"

This reverts commit 5a5feee1b2c51fdef768b25a76be4c3949ec1c99.

* reformat

* improve timeout explanation

* s/usual/average

Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
2022-12-07 18:08:48 +00:00
Alexander Theißen 32578cb010 Remove sandboxing host function interface (#12852)
* Remove sandboxing interface

* Remove unused struct
2022-12-07 12:48:30 +00:00
João Paulo Silva de Souza eae3299ba5 Check all crates (#12709)
* check all crates individually

It's relevant to check workspace crates individually because otherwise their compilation problems
due to feature misconfigurations won't be caught, as exemplified by
https://github.com/paritytech/substrate/issues/12705

* adapt to lack of multiple macos runners

https://github.com/paritytech/substrate/pull/12709#discussion_r1022868752

* fix cancel-pipeline-cargo-check-each-crate-macos

* fix cargo-check-each-crate-macos again

* time command execution

* fix YAML anchors

* add explanation for rounding division

* ensure the minimum of one crate per group

* collect artifacts for pipeline stopper

* revert hardcoded crates_per_group

* re-add crates_per_group=1
2022-11-21 12:49:53 +01:00
Alexander Samusev e649108464 [ci] Add DAG for build-rustdoc and check-dependent-project (#12687)
* [ci] Debug ci runner

* try gha

* allow mac jobs fail

* add dags

* install protoc

* fix protobuf name

* fix dags

* remove allow fail for mac jobs

* remove gha

* adjust cargo-check-macos
2022-11-14 15:55:57 +01:00
Daniel Shiposha 73a65711e8 fix: construct_runtime multiple features (#12594)
* fix: construct_runtime multiple features

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-11-02 00:10:43 +01:00
Alexander Samusev 86744b6b16 [ci] cargo-check-benches against different base branches (#12557) 2022-10-27 10:54:07 +02:00
Oliver Tale-Yazdi 62d72f9258 CI: Enable debug assertions in Wasmer sandbox test (#12540)
* CI: Enable debug assertions in Wasmer sandbox test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix feature dependant import

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2022-10-20 20:57:00 +02:00
Vlad f1b34ccd5a Fix cargo-check-benches job (#12301) 2022-09-20 12:15:48 +03:00
Vlad bba345f8ed Use --locked for Cargo in CI everywhere (#12270) 2022-09-15 14:21:17 +03:00
Vlad 66e9f4925a Don't run cargo-check-benches for master based downstream staging pipelines (#12233) 2022-09-12 20:48:10 +03:00
Bastian Köcher 5647e71947 construct_runtime: Fix generation of types behind features (#12229)
* construct_runtime: Fix generation of types behind features

With the recent addition of supporting features in `construct_runtime!` there was a bug overseen.
The `AllPalletsWithSystem` etc type declarations would be declared twice when a certain was enabled.
The problem was that in the macro we didn't feature gate the types that should be declared when
there is no feature enabled. This pull request now takes care of feature gating this type behind
`all(#( not(feature) ))`. So, these types will only be enabled if no of the configured features is enabled.

* Fix tests

* FMT
2022-09-10 21:50:59 +02:00
Vlad 6c5d4668d0 Fully remove rusty-cachier from the node-bench-regression-guard job (#11904)
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
2022-09-08 15:35:44 +03:00
Bastian Köcher 9d75f3e3e7 Fix benchmarks and adds CI to test them (#12068)
* Fix benchmarks and adds CI to test them

Instead of waiting for benchmarks failing in Polkadot CI, we also can just test them in Substrate :P

* Do not overflow
2022-08-19 13:07:14 +02:00
Alexander Samusev db2d66e59b [ci] Improve cancel-pipeline job (#12008)
* [WIP][ci] Improve cancel-pipeline job

* fix job name

* test that fail works

* debug cancel-pipeline

* remove artifacts-false from cancel-pipeline jobs

* split cancel pipeline jobs

* fail test-linux-stable 2/3

* fail test-linux-stable 3/3

* fail cargo-check-benches 1/2

* fail cargo-check-benches 2/2

* fail test-linux-stable-int

* fail cargo-check-subkey

* fail check-tracing

* fail check-tracing

* fix pipeline
2022-08-12 11:03:27 +02:00
Nikos Kontakis 103f770e75 Rename node-runtime to node-kitchensink-runtime (#11930)
* Rename node=runtime to kithensink-runtime

* Undo md formatting
2022-08-02 15:25:52 +00:00
Alexander Samusev 2e8249dc17 [ci] chores: remove cargo install nextest (#11961) 2022-08-02 15:01:15 +02:00
Alexander Samusev 151c5d3fd9 [ci] remove cargo-check-nixos job (#11873)
* [ci] remove cargo-check-nixos job

* remove shell.nix
2022-07-26 11:37:54 +02:00
Alexander Samusev 3753d70e45 [ci] fix node-bench-regression-guard job (#11901)
* [Do not merge] [ci] debug node-bench-regression-guard

* debug

* fix artifacts path

* add debug job

* debug

* debug

* fix job, return pipeline
2022-07-25 11:15:47 +02:00
Alexander Samusev ed346d97c5 [ci] improvments to make pipeline faster (#11829)
* [DO NOT MERGE] test-linux-stable parallel on 3 ci nodes

* add debug message

* adjust rusty-cachier

* empty commit

* move test-linux-stable to test.yml

* make cargo-check-benches and test-wasmer-sandbox parallel

* fix comment

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* if to case

* use case instead if in cargo-check-benches

* format

* add comments to output

* add comment

* add quotes

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

Co-authored-by: parity-processbot <>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2022-07-15 10:44:46 +02:00
Vlad b357ecd467 Explain why rusty-cachier is put first (#11740) 2022-06-23 17:54:15 +03:00