Files
revive/.github/workflows/reusable-build-wasm.yml
T
LJ 229ce55bd6 Reduce total CI duration (#536)
## Description

Improves overall time spent for our CI checks to complete. For cache
hits, total time decreases from ~70min to ~29min.

## Overview

```mermaid
gantt
    title CI on a PR — BEFORE ~70m vs NOW ~29m
    dateFormat HH:mm
    axisFormat %M

    section BEFORE ~70m
    [release] build-wasm 20m                      :r1, 00:00, 20m
    [release] build-x86_64-apple-darwin 17m       :r2, 00:00, 17m
    [release] build-x86_64-pc-windows-msvc 11m    :r3, 00:00, 11m
    [release] build-x86_64-unknown-linux-musl 10m :r4, 00:00, 10m
    [release] build-aarch64-apple-darwin 8m       :r5, 00:00, 8m
    [release] create-macos-universal 14s          :r6, 00:17, 1m
    [test-wasm] own duplicate wasm build 19m      :w1, 00:00, 19m
    [test-wasm] test matrix 3m                    :w2, 00:19, 3m
    [test] required 12m                                            :crit, t1, 00:00, 12m
    [differential-tests] build matrix waits 8m for release         :crit, t2, 00:12, 8m
    [differential-tests] macos-x86_64 compile and hash 44m         :crit, t3, 00:20, 44m
    [differential-tests] windows-x86_64 compile and hash 26m       :t4, 00:20, 26m
    [differential-tests] linux-x86_64 compile and hash 20m         :t5, 00:20, 20m
    [differential-tests] macos-arm64 compile and hash 15m          :t6, 00:20, 15m
    [differential-tests] run-e2e-tests parallel 43m                :t7, 00:22, 43m
    [differential-tests] compare-hashes 5m                         :crit, t8, 01:04, 5m
    [hardhat] use-hardhat chained 1m                               :crit, t9, 01:09, 1m

    section NOW ~29m
    [release] build-windows 3m              :nr1, 00:00, 3m
    [release] build-x86_64-apple-darwin 7m  :nr2, 00:00, 7m
    [release] build-linux 2m                :nr3, 00:00, 2m
    [release] build-aarch64-apple-darwin 2m :nr4, 00:00, 2m
    [release] create-macos-universal 23s    :nr5, 00:07, 1m
    [release] build-wasm cache hit ~11s     :nr6, 00:15, 1m
    [test-wasm] shared wasm build 15m       :nw1, 00:00, 15m
    [test-wasm] test matrix 3m              :nw2, 00:15, 3m
    [test] required 6m                                              :crit, nt1, 00:00, 6m
    [test] check-changes 2s parallel                                :nt2, 00:00, 1m
    [differential-tests] per-target cache hits ~1m 30s              :crit, nt3, 00:06, 2m
    [differential-tests] windows-x86_64 compile and hash 19m        :crit, nt4, 00:08, 19m
    [differential-tests] linux-x86_64 compile and hash 16m          :nt5, 00:08, 16m
    [differential-tests] macos-arm64 compile and hash 13m           :nt6, 00:08, 13m
    [differential-tests] run-e2e-tests 16m                          :nt7, 00:08, 16m
    [differential-tests] compare-hashes 2m                          :crit, nt8, 00:27, 2m
    [hardhat] use-hardhat parallel 1m                               :nt9, 00:07, 1m
```

## Measurements

| Metric | Before | Now | Improvement |
|---|---|---|---|
| **PR CI wall-clock** | 70m 15s | 28m 47s | **−41m 28s (−59%)** |
| `test` job (required check) | 12m 10s | 6m 17s | −5m 53s (faster LLVM
download) |
| `compile-and-hash` build phase | ~7m 30s (waiting on release builds) |
~30s (builds already available) to ~1m 30s (if waiting for macos) |
−6-7m (per-target build split + faster LLVM download) |
| `run-e2e-tests` build phase | ~7m 30s (waiting on release builds) |
~30s (`linux-x86_64` already available) | −7m (per-target build split) |
| Longest `compile-and-hash` job | 44m 05s (`macos-x86_64`) | 19m 08s
(`windows-x86_64`) | −24m 57s (`macos-x86_64` excluded on PRs) |
| `compile-and-hash` matrix start time | 20m 13s | 7m 54s | −12m 19s
(per-target build split + faster LLVM download) |
| `run-e2e-tests` start time | 21m 50s | 7m 36s | −14m 14s (per-target
build split + faster LLVM download) |
| `run-e2e-tests` duration | 43m 28s | 16m 22s | −27m 06s (concurrency
bump + flakiness fix) |

## Changes

| Change | Before | Now |
|---|---|---|
| **Per-target build sub-workflows** | <ul><li>Getting a reusable build
depended on all builds completing.</li><li>Even if e.g. a macOS build is
cached, the waiting workflow with macOS runners could be queued for a
long time just to get the status and cache of the other workflow, since
macOS runners are more scarce than e.g. Ubuntu.</li></ul> |
<ul><li>Split umbrella `reusable-build.yml` into
`reusable-build-{linux,macos,windows,wasm}.yml`.</li><li>Consumers can
wait on a specific target and start earlier instead of blocking on the
whole matrix.</li><li>E.g. E2E and the compile-and-hash matrix start
earlier (gate only on the targets needed).</li></ul> |
| **Bottleneck macOS Intel only running on `main`** | macOS Intel
runners are 2-3× slower than macOS ARM for the contract compilation
tests. Ran on each PR, being the bottleneck. | This build is now
excluded from the matrix for PR runs, and only included for pushes to
`main`. |
| **E2E tests use higher concurrency** | `run-e2e-tests` duration:
~43min.</li></ul> | <ul><li>Uses higher number of concurrent tasks, made
possible by retester flakiness fix.</li><li>`run-e2e-tests` duration:
~16min.</li></ul> |
| **Faster LLVM downloads** | Building resolc took ~8-13min (~16 min on
macOS intel). | `get-llvm/action.yml` uses more efficient version
detection. Building resolc now takes ~2min (~4-7 min on macOS intel). |
| **Wasm tests use cached artifacts** | `test-wasm.yml` ran its own
duplicate build. | `test-wasm.yml` uses the reusable Wasm build. |
| **Skip unnecessary polkadot-sdk package** | For E2E tests,
`polkadot-omni-node` was unconditionally downloaded (~3-4min). | Only
`pallet-revive-eth-rpc` and `revive-dev-node` are downloaded from the
cache. |
| **Cache `report-processor`** | Only `retester` was cached. |
`report-processor` is cached for the compilations and export hashes
jobs. |
| **Hardhat unblocked from differential tests** | Started after the
differential tests succeeded in order to reuse the Linux build. | Starts
as soon as the required `test` job succeeds, using the reusable Linux
build. |
| **Conditionally skip workflows** | <ul><li>Some workflows
conditionally skipped for md-only changes.</li><li>Tooling-only changes
also ran e.g. `test-wasm.yml` and `differential-tests.yml`.</ul></li> |
<ul><li>Most workflows are skipped for doc-only
updates.</li><li>Tooling-only updates trigger only the respective
tooling test (Hardhat), apart from the required `test`
job.</li><li>Differential tests are skipped for js-only
updates.</li></ul> |
| **Miscellaneous** | <ul><li>Used actions running on soon-deprecated
Node 20.</li><li>Difficult to debug Playwright `test-wasm`
failures.</li></ul> | <ul><li>Upgraded actions that used Node
20.</li><li>Added Playwright debug traces uploaded on `test-wasm`
failures.</li></ul> |

## Resolved Issues

Closes #533
2026-06-05 09:59:17 +02:00

182 lines
6.7 KiB
YAML

name: Reusable Build — Wasm
on:
workflow_call:
inputs:
is_release:
description: "Whether this is a release build"
required: true
type: boolean
retention_days:
description: "Artifact retention days"
required: false
type: number
default: 1
outputs:
resolc_web_js_url:
value: ${{ jobs.build.outputs.resolc_web_js_url }}
resolc_web_js_sha:
value: ${{ jobs.build.outputs.resolc_web_js_sha }}
env:
CARGO_TERM_COLOR: always
EMSCRIPTEN_TARGET: wasm32-unknown-emscripten
HOST_TARGET: x86_64-unknown-linux-gnu
jobs:
build:
name: Build Wasm
runs-on: ubuntu-24.04
outputs:
resolc_web_js_url: ${{ steps.upload-web-js.outputs.artifact-url }}
resolc_web_js_sha: ${{ steps.upload-web-js.outputs.artifact-digest }}
concurrency:
# Use `reusable-build` and not `${{ github.workflow }}` here so that callers share the same concurrency group.
group: reusable-build-wasm32-unknown-emscripten-${{ github.sha }}
cancel-in-progress: false
steps:
- name: Checkout revive
uses: actions/checkout@v6
- name: Check Build Cache
id: cache
uses: actions/cache@v5
with:
path: resolc-${{ env.EMSCRIPTEN_TARGET }}
key: build-${{ env.EMSCRIPTEN_TARGET }}-${{ github.sha }}
- name: Set Up Rust Toolchain
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
# Pin to 1.92.0 until LLVM Wasm libraries are rebuilt with -fwasm-exceptions
# See: https://github.com/paritytech/revive/pull/463
toolchain: "1.92.0"
target: ${{ env.EMSCRIPTEN_TARGET }}
rustflags: ""
- name: Download Host LLVM
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
uses: ./.github/actions/get-llvm
with:
target: ${{ env.HOST_TARGET }}
- name: Download Wasm LLVM
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
uses: ./.github/actions/get-llvm
with:
target: ${{ env.EMSCRIPTEN_TARGET }}
- name: Download EMSDK
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
uses: ./.github/actions/get-emsdk
- name: Build
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
env:
# Consumed by js/emscripten/build.js.
# For nightly Wasm releases, this URI is currently broken (see https://github.com/paritytech/revive/issues/542).
RELEASE_RESOLC_WASM_URI: ${{ inputs.is_release && format('https://github.com/paritytech/revive/releases/download/{0}/resolc.wasm', github.ref_name) || '' }}
run: |
export LLVM_SYS_221_PREFIX=$PWD/llvm-${{ env.HOST_TARGET }}
export REVIVE_LLVM_TARGET_PREFIX=$PWD/llvm-${{ env.EMSCRIPTEN_TARGET }}
source emsdk/emsdk_env.sh
make install-wasm
chmod -x ./target/${{ env.EMSCRIPTEN_TARGET }}/release/resolc.wasm
- name: Move Artifacts
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
run: |
mkdir -p resolc-${{ env.EMSCRIPTEN_TARGET }}
mv ./target/${{ env.EMSCRIPTEN_TARGET }}/release/resolc.js ./resolc-${{ env.EMSCRIPTEN_TARGET }}/
mv ./target/${{ env.EMSCRIPTEN_TARGET }}/release/resolc.wasm ./resolc-${{ env.EMSCRIPTEN_TARGET }}/
mv ./target/${{ env.EMSCRIPTEN_TARGET }}/release/resolc_web.js ./resolc-${{ env.EMSCRIPTEN_TARGET }}/
- name: Set Up Node.js
uses: actions/setup-node@v6
with:
node-version: "24"
- name: Basic Sanity Check
run: |
mkdir -p solc
curl -sSLo solc/soljson.js https://github.com/argotorg/solidity/releases/download/v0.8.35/soljson.js
node -e "
// Make sure to require './solc/soljson' and not 'solc/soljson' in order to use
// the explicitly downloaded solc, rather than resolving an npm package potentially
// installed in prior workflow steps.
const soljson = require('./solc/soljson');
const createRevive = require('./resolc-${{ env.EMSCRIPTEN_TARGET }}/resolc.js');
const compiler = createRevive();
compiler.soljson = soljson;
const standardJsonInput =
{
language: 'Solidity',
sources: {
'MyContract.sol': {
content: 'pragma solidity ^0.8.0; contract MyContract { function greet() public pure returns (string memory) { return \'Hello\'; } }',
},
},
settings: {
optimizer: { enabled: false },
outputSelection: {
'*': {
'*': ['evm.bytecode']
}
}
}
};
compiler.writeToStdin(JSON.stringify(standardJsonInput));
compiler.callMain(['--standard-json']);
const stdout = compiler.readFromStdout();
const stderr = compiler.readFromStderr();
if (stderr) { console.error(stderr); process.exit(1); }
let out = JSON.parse(stdout);
let bytecode = out.contracts['MyContract.sol']['MyContract'].evm.bytecode.object
console.log(bytecode);
if(!bytecode.startsWith('50564d')) { process.exit(1); }
"
- name: Upload Bundle Artifact (Release)
if: ${{ inputs.is_release }}
uses: actions/upload-artifact@v6
with:
name: resolc-${{ env.EMSCRIPTEN_TARGET }}
path: resolc-${{ env.EMSCRIPTEN_TARGET }}/*
retention-days: ${{ inputs.retention_days }}
# There is no way to upload several files as several artifacts with a single upload-artifact step.
# It's needed to have resolc_web.js separately for nightly builds for resolc-bin repo.
# https://github.com/actions/upload-artifact/issues/331
- name: Upload resolc.js (Nightly)
if: ${{ !inputs.is_release }}
uses: actions/upload-artifact@v6
with:
name: resolc.js
path: resolc-${{ env.EMSCRIPTEN_TARGET }}/resolc.js
retention-days: ${{ inputs.retention_days }}
- name: Upload resolc.wasm (Nightly)
if: ${{ !inputs.is_release }}
uses: actions/upload-artifact@v6
with:
name: resolc.wasm
path: resolc-${{ env.EMSCRIPTEN_TARGET }}/resolc.wasm
retention-days: ${{ inputs.retention_days }}
- name: Upload resolc_web.js (Nightly)
if: ${{ !inputs.is_release }}
id: upload-web-js
uses: actions/upload-artifact@v6
with:
name: resolc_web.js
path: resolc-${{ env.EMSCRIPTEN_TARGET }}/resolc_web.js
retention-days: ${{ inputs.retention_days }}