Files
revive/.github/workflows/differential-tests.yml
T
LJ 733e01c78e Implement verification of reproducible builds and run differential E2E tests (#468)
## Description

This adds verification for reproducible builds (via
[revive-differential-tests](https://github.com/paritytech/revive-differential-tests)
testing framework) by:

1. Compiling
[contracts](https://github.com/paritytech/resolc-compiler-tests/tree/main/fixtures/solidity)
and generating hashes from each bytecode
    * With the resolc builds:
      * linux musl
      * macos universal
      * windows
      * (future todo: wasm32 emscripten)
    * With optimization levels `0`, `3`, `z` for each contract
2. Exporting the hashes
3. Comparing the hashes
    * For each optimization level:
* Checks if each hash for a specific contract is identical across the
platforms

This PR also runs the differential E2E tests (via `retester`) with the
resolc linux musl build.

All of the above only runs if our `test` workflow succeeds.

## Related

This PR initially implemented the above in this repo but has now been
implemented into the separate testing framework instead. The most
relevant PRs implemented alongside this PR as prerequisites are:

* `revive-differential-tests`:
  * [Generate bytecode hashes included in the report
](https://github.com/paritytech/revive-differential-tests/pull/238)
* [Fix handling of resolc's optimizer `mode` and allow setting solc
optimizer
enabled](https://github.com/paritytech/revive-differential-tests/pull/243)
* [Fix solc and resolc optimization
settings](https://github.com/paritytech/revive-differential-tests/pull/253)
* [Implement a `compile` context/subcommand for standalone compilation
mode](https://github.com/paritytech/revive-differential-tests/pull/239)
* [Implement support for `--mode` CLI argument for
`compile`](https://github.com/paritytech/revive-differential-tests/pull/246)
* [Implement `export-hashes` and `compare-hashes` subcommands for report
processor
](https://github.com/paritytech/revive-differential-tests/pull/250)
* [Implement GH actions for compiling, exporting and comparing
hashes](https://github.com/paritytech/revive-differential-tests/pull/251)
* [Gate zombienet-sdk behind `cfg(unix)` to fix Windows
builds](https://github.com/paritytech/revive-differential-tests/pull/254)
* [Link library-using contracts with dummy addresses for `--compile`
command](https://github.com/paritytech/revive-differential-tests/pull/267)
* `revive`:
* [Build LLVM for Windows using MSVC-compatible
Clang](https://github.com/paritytech/revive/pull/497)
* [Apply LLVM patches for deterministic operand order and correct
peephole optimization for `minsize`
](https://github.com/paritytech/revive/pull/500)
* Investigation of comparison mismatches in third-party dependencies:
  * `llvm-project`:
* [`SelectionDAG` type legalization produces different operand ordering
for `i256` multiply across
platforms](https://github.com/llvm/llvm-project/issues/190148)
  * `solidity`:
* [Windows absolute paths in standard JSON break import resolution and
library linking](https://github.com/argotorg/solidity/issues/16579)

## Notes

### Failed compilations

Since
[resolc-compiler-tests](https://github.com/paritytech/resolc-compiler-tests)
currently includes contracts that are expected to fail resolc
compilation (e.g. due to unsupported instructions, contracts that are
expected to work with solc for evm/revm), the compilation workflow job
is not expected to fail if compilation fails for a contract.

Instead, if a contract compiles with one build but not with another, a
hash will be missing and will thereby generate a mismatch when comparing
all hashes, failing at the verification step.

## Resolved Issues

Closes [#175](https://github.com/paritytech/revive/issues/175)
2026-04-14 16:58:58 +02:00

177 lines
6.1 KiB
YAML

name: Verify Reproducible Builds
on:
workflow_call:
env:
# The compiler modes to use for compilation by retester.
MODES: "Y M0 S+, Y M3 S+, Y Mz S+"
SOLC_VERSION: "0.8.34"
jobs:
build:
uses: ./.github/workflows/reusable-build.yml
with:
is_release: false
retention_days: 1
compile-and-export-hashes-native:
needs: build
strategy:
fail-fast: false
matrix:
include:
- platform: linux-x86_64
target: x86_64-unknown-linux-musl
runner: ubuntu-24.04
- platform: macos-arm64
target: universal-apple-darwin
runner: macos-15
- platform: macos-x86_64
target: universal-apple-darwin
runner: macos-15-intel
- platform: windows-x86_64
target: x86_64-pc-windows-msvc
runner: windows-2022
runs-on: ${{ matrix.runner }}
steps:
# For some deeply nested contracts in `resolc-compiler-tests`, Windows
# will generate a "Filename too long" error. This step enables long paths.
- name: Enable Git Long Paths (Windows)
if: ${{ runner.os == 'Windows' }}
run: git config --system core.longpaths true
- name: Checkout revive
uses: actions/checkout@v4
with:
path: revive
submodules: recursive
- name: Download Binary
uses: actions/download-artifact@v4
with:
name: resolc-${{ matrix.target }}
path: resolc-bin
- name: Make resolc Executable
shell: bash
run: chmod +x "resolc-bin/resolc-${{ matrix.target }}${{ runner.os == 'Windows' && '.exe' || '' }}"
- name: Compile Contracts
id: compile
uses: ./revive/revive-differential-tests/.github/actions/compile-contracts
with:
revive-differential-tests-path: revive/revive-differential-tests
resolc-path: resolc-bin/resolc-${{ matrix.target }}${{ runner.os == 'Windows' && '.exe' || '' }}
solc-version: ${{ env.SOLC_VERSION }}
modes: ${{ env.MODES }}
upload-artifact-name: compilation-report-${{ matrix.platform }}
- name: Export Bytecode Hashes
uses: ./revive/revive-differential-tests/.github/actions/export-bytecode-hashes
with:
revive-differential-tests-path: revive/revive-differential-tests
report-path: ${{ steps.compile.outputs.report-path }}
remove-prefix: ${{ steps.compile.outputs.contracts-source-prefix }}
platform-label: ${{ matrix.platform }}
upload-artifact-name: hashes-${{ matrix.platform }}
compare-hashes:
needs: compile-and-export-hashes-native
runs-on: ubuntu-24.04
steps:
- name: Checkout revive
uses: actions/checkout@v4
with:
path: revive
submodules: true
- name: Compare Bytecode Hashes
uses: ./revive/revive-differential-tests/.github/actions/compare-bytecode-hashes
with:
revive-differential-tests-path: revive/revive-differential-tests
hash-artifact-names: "hashes-linux-x86_64, hashes-macos-arm64, hashes-macos-x86_64, hashes-windows-x86_64"
modes: ${{ env.MODES }}
upload-artifact-name: hash-comparison-result
run-e2e-tests:
needs: build
runs-on: parity-large
timeout-minutes: 75
env:
# Commit SHA that produced the polkadot-sdk crate pinned in Cargo.toml,
# obtained from the crate's .cargo_vcs_info.json via:
# curl -sL https://crates.io/api/v1/crates/polkadot-sdk/<VERSION>/download | tar xzO '*/.cargo_vcs_info.json'
# Current version: 2603.0.0
POLKADOT_SDK_REF: "c56b086a3cfcb3a48b75fd1ef9c77ad9b7408b95"
RUSTFLAGS: "-Awarnings"
steps:
- name: Checkout revive
uses: actions/checkout@v4
with:
path: revive
submodules: recursive
- name: Download Binary
uses: actions/download-artifact@v4
with:
name: resolc-x86_64-unknown-linux-musl
path: ./resolc-bin
- name: Make resolc Executable
shell: bash
run: chmod +x ./resolc-bin/resolc-x86_64-unknown-linux-musl
- name: Checkout Polkadot SDK
uses: actions/checkout@v4
with:
repository: paritytech/polkadot-sdk
ref: ${{ env.POLKADOT_SDK_REF }}
path: polkadot-sdk
submodules: recursive
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install -y build-essential pkg-config libssl-dev protobuf-compiler llvm clang libclang-dev
- name: Download forklift
run: |
curl -fsSLo forklift https://github.com/paritytech/forklift/releases/download/0.15.0/forklift_0.15.0_linux_amd64
chmod +x forklift
sudo mv forklift /usr/local/bin/
- name: Configure forklift
run: |
mkdir -p .forklift
cp polkadot-sdk/.forklift/config.toml .forklift/config.toml
- name: Set Up Rust Toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: "1.90.0"
target: "wasm32-unknown-unknown"
components: "rust-src"
rustflags: ""
- name: Run Differential E2E Tests
uses: ./revive/revive-differential-tests/.github/actions/run-differential-tests
with:
revive-differential-tests-path: revive/revive-differential-tests
polkadot-sdk-path: ./polkadot-sdk
cargo-command: "forklift cargo"
platform: revive-dev-node-polkavm-resolc
resolc-path: ./resolc-bin/resolc-x86_64-unknown-linux-musl
solc-version: ${{ env.SOLC_VERSION }}
expectations-file-path: ./revive/.github/assets/revive-dev-node-polkavm-resolc.json
# Sentinel job that depends on the jobs we want to allow completing (instead of
# being canceled once the other job chains complete). Thus, run-e2e-tests
# will always run even if compare-hashes finishes first, and vice versa.
wait-for-jobs:
if: ${{ always() }}
needs: [compare-hashes, run-e2e-tests]
runs-on: ubuntu-24.04
steps:
- run: echo "All jobs finished"