mirror of
https://github.com/pezkuwichain/revive.git
synced 2026-10-20 18:56:17 +00:00
## Description
Improves overall time spent for our CI checks to complete. For cache
hits, total time decreases from ~70min to ~29min.
## Overview
```mermaid
gantt
title CI on a PR — BEFORE ~70m vs NOW ~29m
dateFormat HH:mm
axisFormat %M
section BEFORE ~70m
[release] build-wasm 20m :r1, 00:00, 20m
[release] build-x86_64-apple-darwin 17m :r2, 00:00, 17m
[release] build-x86_64-pc-windows-msvc 11m :r3, 00:00, 11m
[release] build-x86_64-unknown-linux-musl 10m :r4, 00:00, 10m
[release] build-aarch64-apple-darwin 8m :r5, 00:00, 8m
[release] create-macos-universal 14s :r6, 00:17, 1m
[test-wasm] own duplicate wasm build 19m :w1, 00:00, 19m
[test-wasm] test matrix 3m :w2, 00:19, 3m
[test] required 12m :crit, t1, 00:00, 12m
[differential-tests] build matrix waits 8m for release :crit, t2, 00:12, 8m
[differential-tests] macos-x86_64 compile and hash 44m :crit, t3, 00:20, 44m
[differential-tests] windows-x86_64 compile and hash 26m :t4, 00:20, 26m
[differential-tests] linux-x86_64 compile and hash 20m :t5, 00:20, 20m
[differential-tests] macos-arm64 compile and hash 15m :t6, 00:20, 15m
[differential-tests] run-e2e-tests parallel 43m :t7, 00:22, 43m
[differential-tests] compare-hashes 5m :crit, t8, 01:04, 5m
[hardhat] use-hardhat chained 1m :crit, t9, 01:09, 1m
section NOW ~29m
[release] build-windows 3m :nr1, 00:00, 3m
[release] build-x86_64-apple-darwin 7m :nr2, 00:00, 7m
[release] build-linux 2m :nr3, 00:00, 2m
[release] build-aarch64-apple-darwin 2m :nr4, 00:00, 2m
[release] create-macos-universal 23s :nr5, 00:07, 1m
[release] build-wasm cache hit ~11s :nr6, 00:15, 1m
[test-wasm] shared wasm build 15m :nw1, 00:00, 15m
[test-wasm] test matrix 3m :nw2, 00:15, 3m
[test] required 6m :crit, nt1, 00:00, 6m
[test] check-changes 2s parallel :nt2, 00:00, 1m
[differential-tests] per-target cache hits ~1m 30s :crit, nt3, 00:06, 2m
[differential-tests] windows-x86_64 compile and hash 19m :crit, nt4, 00:08, 19m
[differential-tests] linux-x86_64 compile and hash 16m :nt5, 00:08, 16m
[differential-tests] macos-arm64 compile and hash 13m :nt6, 00:08, 13m
[differential-tests] run-e2e-tests 16m :nt7, 00:08, 16m
[differential-tests] compare-hashes 2m :crit, nt8, 00:27, 2m
[hardhat] use-hardhat parallel 1m :nt9, 00:07, 1m
```
## Measurements
| Metric | Before | Now | Improvement |
|---|---|---|---|
| **PR CI wall-clock** | 70m 15s | 28m 47s | **−41m 28s (−59%)** |
| `test` job (required check) | 12m 10s | 6m 17s | −5m 53s (faster LLVM
download) |
| `compile-and-hash` build phase | ~7m 30s (waiting on release builds) |
~30s (builds already available) to ~1m 30s (if waiting for macos) |
−6-7m (per-target build split + faster LLVM download) |
| `run-e2e-tests` build phase | ~7m 30s (waiting on release builds) |
~30s (`linux-x86_64` already available) | −7m (per-target build split) |
| Longest `compile-and-hash` job | 44m 05s (`macos-x86_64`) | 19m 08s
(`windows-x86_64`) | −24m 57s (`macos-x86_64` excluded on PRs) |
| `compile-and-hash` matrix start time | 20m 13s | 7m 54s | −12m 19s
(per-target build split + faster LLVM download) |
| `run-e2e-tests` start time | 21m 50s | 7m 36s | −14m 14s (per-target
build split + faster LLVM download) |
| `run-e2e-tests` duration | 43m 28s | 16m 22s | −27m 06s (concurrency
bump + flakiness fix) |
## Changes
| Change | Before | Now |
|---|---|---|
| **Per-target build sub-workflows** | <ul><li>Getting a reusable build
depended on all builds completing.</li><li>Even if e.g. a macOS build is
cached, the waiting workflow with macOS runners could be queued for a
long time just to get the status and cache of the other workflow, since
macOS runners are more scarce than e.g. Ubuntu.</li></ul> |
<ul><li>Split umbrella `reusable-build.yml` into
`reusable-build-{linux,macos,windows,wasm}.yml`.</li><li>Consumers can
wait on a specific target and start earlier instead of blocking on the
whole matrix.</li><li>E.g. E2E and the compile-and-hash matrix start
earlier (gate only on the targets needed).</li></ul> |
| **Bottleneck macOS Intel only running on `main`** | macOS Intel
runners are 2-3× slower than macOS ARM for the contract compilation
tests. Ran on each PR, being the bottleneck. | This build is now
excluded from the matrix for PR runs, and only included for pushes to
`main`. |
| **E2E tests use higher concurrency** | `run-e2e-tests` duration:
~43min.</li></ul> | <ul><li>Uses higher number of concurrent tasks, made
possible by retester flakiness fix.</li><li>`run-e2e-tests` duration:
~16min.</li></ul> |
| **Faster LLVM downloads** | Building resolc took ~8-13min (~16 min on
macOS intel). | `get-llvm/action.yml` uses more efficient version
detection. Building resolc now takes ~2min (~4-7 min on macOS intel). |
| **Wasm tests use cached artifacts** | `test-wasm.yml` ran its own
duplicate build. | `test-wasm.yml` uses the reusable Wasm build. |
| **Skip unnecessary polkadot-sdk package** | For E2E tests,
`polkadot-omni-node` was unconditionally downloaded (~3-4min). | Only
`pallet-revive-eth-rpc` and `revive-dev-node` are downloaded from the
cache. |
| **Cache `report-processor`** | Only `retester` was cached. |
`report-processor` is cached for the compilations and export hashes
jobs. |
| **Hardhat unblocked from differential tests** | Started after the
differential tests succeeded in order to reuse the Linux build. | Starts
as soon as the required `test` job succeeds, using the reusable Linux
build. |
| **Conditionally skip workflows** | <ul><li>Some workflows
conditionally skipped for md-only changes.</li><li>Tooling-only changes
also ran e.g. `test-wasm.yml` and `differential-tests.yml`.</ul></li> |
<ul><li>Most workflows are skipped for doc-only
updates.</li><li>Tooling-only updates trigger only the respective
tooling test (Hardhat), apart from the required `test`
job.</li><li>Differential tests are skipped for js-only
updates.</li></ul> |
| **Miscellaneous** | <ul><li>Used actions running on soon-deprecated
Node 20.</li><li>Difficult to debug Playwright `test-wasm`
failures.</li></ul> | <ul><li>Upgraded actions that used Node
20.</li><li>Added Playwright debug traces uploaded on `test-wasm`
failures.</li></ul> |
## Resolved Issues
Closes #533
218 lines
7.3 KiB
YAML
218 lines
7.3 KiB
YAML
name: Verify Reproducible Builds
|
|
|
|
on:
|
|
workflow_call:
|
|
|
|
env:
|
|
IS_PUSH_TO_MAIN: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
|
|
# The compiler modes to use for compilation by retester.
|
|
MODES: "Y M0 S+, Y M3 S+, Y Mz S+"
|
|
SOLC_VERSION: "0.8.35"
|
|
|
|
jobs:
|
|
build-linux:
|
|
uses: ./.github/workflows/reusable-build-linux.yml
|
|
with:
|
|
retention_days: 1
|
|
|
|
build-macos:
|
|
uses: ./.github/workflows/reusable-build-macos.yml
|
|
with:
|
|
retention_days: 1
|
|
|
|
build-windows:
|
|
uses: ./.github/workflows/reusable-build-windows.yml
|
|
with:
|
|
retention_days: 1
|
|
|
|
build-matrix:
|
|
runs-on: ubuntu-24.04
|
|
outputs:
|
|
entries: ${{ steps.matrix.outputs.entries }}
|
|
steps:
|
|
- name: Build Matrix
|
|
id: matrix
|
|
run: |
|
|
entries='[
|
|
{
|
|
"platform": "linux-x86_64",
|
|
"target": "x86_64-unknown-linux-musl",
|
|
"runner": "ubuntu-24.04"
|
|
},
|
|
{
|
|
"platform": "macos-arm64",
|
|
"target": "universal-apple-darwin",
|
|
"runner": "macos-15"
|
|
},
|
|
${{ env.IS_PUSH_TO_MAIN == 'true' && '{
|
|
"platform": "macos-x86_64",
|
|
"target": "universal-apple-darwin",
|
|
"runner": "macos-15-intel"
|
|
},' || '' }}
|
|
{
|
|
"platform": "windows-x86_64",
|
|
"target": "x86_64-pc-windows-msvc",
|
|
"runner": "windows-2022"
|
|
}
|
|
]'
|
|
|
|
echo "Matrix:"
|
|
jq . <<< "$entries"
|
|
echo "entries=$(jq -c . <<< "$entries")" >> "$GITHUB_OUTPUT"
|
|
|
|
compile-and-export-hashes-native:
|
|
needs: [build-matrix, build-linux, build-macos, build-windows]
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include: ${{ fromJSON(needs.build-matrix.outputs.entries) }}
|
|
runs-on: ${{ matrix.runner }}
|
|
steps:
|
|
# For some deeply nested contracts in `resolc-compiler-tests`, Windows
|
|
# will generate a "Filename too long" error. This step enables long paths.
|
|
- name: Enable Git Long Paths (Windows)
|
|
if: ${{ runner.os == 'Windows' }}
|
|
run: git config --system core.longpaths true
|
|
|
|
- name: Checkout revive
|
|
uses: actions/checkout@v6
|
|
with:
|
|
path: revive
|
|
submodules: recursive
|
|
|
|
- name: Download Binary
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: resolc-${{ matrix.target }}
|
|
path: resolc-bin
|
|
|
|
- name: Make resolc Executable
|
|
shell: bash
|
|
run: chmod +x "resolc-bin/resolc-${{ matrix.target }}${{ runner.os == 'Windows' && '.exe' || '' }}"
|
|
|
|
- name: Compile Contracts
|
|
id: compile
|
|
uses: ./revive/revive-differential-tests/.github/actions/compile-contracts
|
|
with:
|
|
revive-differential-tests-path: revive/revive-differential-tests
|
|
resolc-path: resolc-bin/resolc-${{ matrix.target }}${{ runner.os == 'Windows' && '.exe' || '' }}
|
|
solc-version: ${{ env.SOLC_VERSION }}
|
|
modes: ${{ env.MODES }}
|
|
upload-artifact-name: compilation-report-${{ matrix.platform }}
|
|
|
|
- name: Export Bytecode Hashes
|
|
uses: ./revive/revive-differential-tests/.github/actions/export-bytecode-hashes
|
|
with:
|
|
revive-differential-tests-path: revive/revive-differential-tests
|
|
report-path: ${{ steps.compile.outputs.report-path }}
|
|
remove-prefix: ${{ steps.compile.outputs.contracts-source-prefix }}
|
|
platform-label: ${{ matrix.platform }}
|
|
upload-artifact-name: hashes-${{ matrix.platform }}
|
|
|
|
compare-hashes:
|
|
needs: compile-and-export-hashes-native
|
|
runs-on: ubuntu-24.04
|
|
steps:
|
|
- name: Checkout revive
|
|
uses: actions/checkout@v6
|
|
with:
|
|
path: revive
|
|
submodules: true
|
|
|
|
- name: Compare Bytecode Hashes
|
|
uses: ./revive/revive-differential-tests/.github/actions/compare-bytecode-hashes
|
|
with:
|
|
revive-differential-tests-path: revive/revive-differential-tests
|
|
hash-artifact-names: >-
|
|
hashes-linux-x86_64,
|
|
hashes-macos-arm64,
|
|
${{ env.IS_PUSH_TO_MAIN == 'true' && 'hashes-macos-x86_64,' || '' }}
|
|
hashes-windows-x86_64
|
|
modes: ${{ env.MODES }}
|
|
upload-artifact-name: hash-comparison-result
|
|
|
|
run-e2e-tests:
|
|
needs: build-linux
|
|
runs-on: parity-large
|
|
timeout-minutes: 60
|
|
env:
|
|
# Commit SHA that produced the polkadot-sdk crate pinned in Cargo.toml,
|
|
# obtained from the crate's .cargo_vcs_info.json via:
|
|
# curl -sL https://crates.io/api/v1/crates/polkadot-sdk/<VERSION>/download | tar xzO '*/.cargo_vcs_info.json'
|
|
# Current version: 2603.0.0
|
|
POLKADOT_SDK_REF: "c56b086a3cfcb3a48b75fd1ef9c77ad9b7408b95"
|
|
RUSTFLAGS: "-Awarnings"
|
|
steps:
|
|
- name: Checkout revive
|
|
uses: actions/checkout@v6
|
|
with:
|
|
path: revive
|
|
submodules: recursive
|
|
|
|
- name: Download Binary
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: resolc-x86_64-unknown-linux-musl
|
|
path: ./resolc-bin
|
|
|
|
- name: Make resolc Executable
|
|
shell: bash
|
|
run: chmod +x ./resolc-bin/resolc-x86_64-unknown-linux-musl
|
|
|
|
- name: Checkout Polkadot SDK
|
|
uses: actions/checkout@v6
|
|
with:
|
|
repository: paritytech/polkadot-sdk
|
|
ref: ${{ env.POLKADOT_SDK_REF }}
|
|
path: polkadot-sdk
|
|
submodules: recursive
|
|
|
|
- name: Install Dependencies
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y build-essential pkg-config libssl-dev protobuf-compiler llvm clang libclang-dev
|
|
|
|
- name: Download forklift
|
|
run: |
|
|
curl -fsSLo forklift https://github.com/paritytech/forklift/releases/download/0.15.0/forklift_0.15.0_linux_amd64
|
|
chmod +x forklift
|
|
sudo mv forklift /usr/local/bin/
|
|
|
|
- name: Configure forklift
|
|
run: |
|
|
mkdir -p .forklift
|
|
cp polkadot-sdk/.forklift/config.toml .forklift/config.toml
|
|
|
|
- name: Set Up Rust Toolchain
|
|
uses: actions-rust-lang/setup-rust-toolchain@v1
|
|
with:
|
|
toolchain: "1.90.0"
|
|
target: "wasm32-unknown-unknown"
|
|
components: "rust-src"
|
|
rustflags: ""
|
|
|
|
- name: Run Differential E2E Tests
|
|
uses: ./revive/revive-differential-tests/.github/actions/run-differential-tests
|
|
with:
|
|
revive-differential-tests-path: revive/revive-differential-tests
|
|
polkadot-sdk-path: ./polkadot-sdk
|
|
cargo-command: "forklift cargo"
|
|
platform: revive-dev-node-polkavm-resolc
|
|
resolc-path: ./resolc-bin/resolc-x86_64-unknown-linux-musl
|
|
solc-version: ${{ env.SOLC_VERSION }}
|
|
expectations-file-path: ./revive/.github/assets/revive-dev-node-polkavm-resolc.json
|
|
# Use a sane value which is fast enough to make the tests run fast but not too
|
|
# fast that it overwhelms the node.
|
|
concurrency-number-of-concurrent-tasks: 1000
|
|
|
|
# Sentinel job that depends on the jobs we want to allow completing (instead of
|
|
# being canceled once the other job chains complete). Thus, run-e2e-tests
|
|
# will always run even if compare-hashes finishes first, and vice versa.
|
|
wait-for-jobs:
|
|
if: ${{ always() }}
|
|
needs: [compare-hashes, run-e2e-tests]
|
|
runs-on: ubuntu-24.04
|
|
steps:
|
|
- name: Finish
|
|
run: echo "All jobs finished"
|